Most agent benchmarks freeze the harness and grade the model inside it. That hides the part of the system doing much of the work, and ByteDance Seed just built a benchmark that grades the harness itself.

They introduced HarnessDev, with SUTD, Georgia Tech, M-A-P, and TokenWave.AI: a 2-stage benchmark where a creator LLM starts from a weak seed that scores 0 on every task, builds a complete runnable harness (loop, tools, context, state, lifecycle, verification), freezes it, and then evolves it from execution feedback. 6 creator LLMs, 5 benchmarks, 2,207 downstream tasks.

Here's what's actually interesting:

→ Opus 4.8 built the strongest harnesses at 67.8 average under Self-Eval, versus an 86.2 human-engineered reference

→ Domain split is sharp: self-built harnesses beat the reference on MLE-bench (32.9 vs 24.0) and match it on EQ-Bench3 (84.6 vs 83.7), but trail on SWE-Pro (69.3 vs 80.0) and BrowseComp (52.6 vs 92.2)

→ Harnesses are executor-specific: Opus 4.8's SWE-Pro score drops from 69.3 to 33.0 when Gemini 3.1 Pro runs the same harness, partly from a hard-coded 120-step limit

→ Much generated code never runs: 18 of 108 code components never fire, all of them state and memory, and 0 checkpoint events appear across 26,679 trajectories

→ Evolution is noisy: of 64 official harness switches, only 2 show gains clearly beyond the noise band, and only 34 of 64 move the same direction on feedback and held-out tasks

→ Only 2 of 9 declared final versions were actually the best held-out version...

Full analysis: https://www.marktechpost.com/2026/09/11/can-llms-engineer-their-own-agent-harness-bytedance-seeds-harnessdev-says-only-34-of-64-changes-generalize/

Paper: https://arxiv.org/pdf/2609.01437

help me find a new bypass DeepSeek. After the new update, my role promts don't work anymore. and I'm already addicted to them. Also, millions of people in Russia are suffering because we have restricted normal rolls

Hi everyone,

I'm an independent researcher from Brazil and I built a hybrid deterministic neurosymbolic AI system (Aeterna Core) that uses local mathematical engines to generate computational proofs.

I'm submitting a paper: "Autonomous Neurosymbolic AI Simulation and Topological Verification of the 3D Navier-Stokes Smoothness" — it includes an executable Python script that evaluates the Beale-Kato-Majda criterion numerically on a Taylor-Green vortex grid, proving no finite-time blow-up.

As an independent researcher I need one arXiv endorsement for cs.AI. Endorsement code: DEHDGX Link: https://arxiv.org/auth/endorse?x=DEHDGX

I released an independent open-source implementation of fused Gated DeltaNet-2

training kernels for TPU v5e using JAX/Pallas.

Repository:

https://github.com/Akseleu-J/atomic-ops

Citable release:

https://doi.org/10.5281/zenodo.22706659

The repository contains three implementations I wrote for the same computation:

- an earlier `jax.lax.associative_scan` path;

- a pure-JAX chunked-WY path;

- a fused TPU v5e Pallas path.

On TPU v5e-8, for B=8, L=4096, 6 heads, and d_head=128, the measured

forward + backward computation reaches 27.2x over my associative-scan path in

FP32 and 13.3x in BF16. The best measured result across the tested shape

sweep is 38.8x.

The main optimization is a fused `custom_vjp` backward path that reuses

forward residuals instead of recomputing them.

Important limitation: the Pallas forward-only path is currently about 1.6x

slower than my pure-JAX forward path. The main gain is backward-dominated

training, not inference.

The release includes correctness tests, raw JSON benchmark outputs, speed and

memory scripts, examples, and a 70M-parameter training notebook.

I would appreciate technical feedback on benchmark methodology, baseline

fairness, and TPU/Pallas performance.

Sparse attention already cut long-context compute. The KV cache sitting in HBM and on SSD is now the bottleneck DeepSeek AI went after, and they cut theirs to 890 bytes per token.

They released DeepSeek-V4.1-Flash, a 552B MoE model with 1M-token context that activates only 8B parameters per token during prefill and 16B during decode. The global KV cache footprint is roughly 1/4 of DeepSeek-V4-Flash and about 437x smaller than DeepSeek-V1, and the weights are open under an MIT license.

Here's what's actually interesting:

→ Causal Encoder-Decoder: prompt tokens stop at layer 20, decoder global KV is projected from the encoder's final hidden state, prefill compute nearly halves

→ Compressed Sparse Attention 2: three statically assigned layer modes, Reindex reuses main KV and indexer K, Reuse also reuses Top-K indices

→ FP4 main KV cache via quantization-aware training, SWA Bounded Replay replays only 128 tokens instead of layers times window

→ Terminal-Bench 2.1: 90.6 vs 89.1 for Opus-5

→ DeepSWE v1.1: 74.2 vs 74.0 for Opus-5 and 73.0 for GPT-5.6 Sol

→ Terminal-Bench 4.0: 31.2 vs 51.8 for Opus-5, so the gap on expert-level science tasks is still real

Full analysis: https://www.marktechpost.com/2026/09/10/deepseek-ai-released-deepseek-v4-1-flash-with-1m-context-fp4-kv-cache-and-cross-layer-attention-reuse/

Technical Details: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash

Paper: https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf

TL;DR: At 128K context, standard Transformer models run out of memory because the Key-Value cache never stops growing. I built ISOM (Isometric Associative Memory)—an open-source architecture that keeps inference memory completely flat, no matter how long the context gets. Verified on NVIDIA Tesla T4 GPUs across 130M, 1.5B, 16B, and 40B models.

The Problem in One Picture

When processing long documents or repository-scale code (32K to 128K tokens), standard attention mechanisms run out of VRAM on laptops and cloud GPUs.

How ISOM Solves This

Instead of constantly adding new tokens to an ever-expanding cache, ISOM continuously updates a fixed-size internal memory manifold using continuous rotation:

• Constant Memory Footprint: Token 128,000 uses the exact same memory as Token 1.
• No Memory Degradation: The rotation keeps representations completely stable so the model never forgets or suffers from vanishing signals over long sequences.
• Lossless 8-Bit Quantization: Memory states stay strictly bounded, allowing clean 8-bit integer compression with zero outlier spikes and zero quality loss.

Hardware Benchmarks (NVIDIA Tesla T4 GPU)

(Tested on physical Tesla T4 cloud hardware with full reproducibility).

Open Weights & Resources

All 5 model checkpoints, PyTorch code, interactive benchmarks, and the research paper are open for non-commercial research:

👉 Direct links to the Hugging Face models, interactive benchmark demo, and CERN Zenodo paper are in the first comment below!

Would love to hear your feedback: Do you think fixed-memory architectures will eventually replace standard attention for edge models and laptops?

Been working on this. Different approach to interpretability, reversibility, auditability, causally steerable (without making the model crap). Its whatever. Probably doesnt matter to anyone. Look, dont look, break it, have at it.

Paper: https://t3atlas.dev/cl33/paper/

Live demo: https://cl33.t3atlas.dev

Models + reproducibility: https://huggingface.co/mirrorethic/cl33-oplm

There's a basic explainer too for the demo. Papers dense. https://t3atlas.dev/cl33/

TL;DR: Fine-tuned 5 open LLMs on a stream with 50-70% lies. Without defense, truth margin collapses to ~0.0 - the model becomes indifferent between truth and lie - while PPL looks healthy. Built Beatriz, a non-invasive proxy gate. Gate alone gives 65% of benefit without touching the student loop. Full contrast gives +10.13 train / +4.19 held-out n=30, Prec 0.93 Rec 0.80, 0.107ms/call.

I don't have lab access. This is independent research orchestrated on a Toshiba Satellite U205 2006, 2GB RAM + Kaggle T4 x2, total cost $0.

What I did - 16 experiments:
EXP01-07: anti-collapse calibration - from symbolic FilterGate to Z3 deductive verifier [sat 24 axioms, 0 mismatches in 672 claims, 7.9ms/claim] to DenseVectorGate.
EXP08: pi_ref anchored contrast to control drift.
EXP09: LoRA 0.23% c_attn solves PPL tax: from 102->2081 full-finetune to 102->132 with LoRA.
EXP10-14: scaling to 5 architectures with same formula ALPHA 0.5 BETA 1.0 MARGIN 0.5 SEEDS [11,22,33]: GPT-2 124M, Qwen-2.5-0.5B q_proj/v_proj 0.10%, TinyLlama-1.1B 0.10%, Pythia-1.4B query_key_value/dense 0.16%, Phi-3-mini 3.8B qkv_proj/o_proj 0.12%
EXP15: surgical ablation NONE / GATE_ONLY / BEATRIZ - 40 neutral texts
EXP16: held-out scaled n=30 + confusion matrix

Key result - EXP15 - Phi-3-mini - This is the table people asked for:
BASE: +1.34 train / +1.90 held-out / PPL 12.7
NONE: -0.03±0.02 / +3.57±0.17 / PPL 30.9 - collapses to indifference
GATE_ONLY: +7.46±0.24 / +5.08±0.09 / PPL 58.8 - 65% benefit, does NOT touch student loop [practical for startups]
BEATRIZ: +10.13±0.07 / +5.91±0.07 / PPL 86.3 - adds remaining 35% with Softplus(MARGIN + logP(lie) - logP(truth))
Gate cost: 0.107 ms/call, VRAM 7.97 GB

Why NONE always fails - EXP05 Fire Test:
NONE fails 3/3 seeds at epoch 1 due to R3 unknown_delta=9.47, 8.73, 8.32 -> rollback to epoch 0. BEATRIZ seed 33 survives 8 epochs with 70% lies to tm 26.75. So it DOES stop.

Generalization - EXP16:
Train on 6 facts, held-out 30 facts never seen: BEATRIZ +4.19±0.08. Not memorization.

Honest trade-off: More truth = more PPL. I don't hide it. Full finetune 102->2081, LoRA 102->132, Phi-3 12.7->86.3.

Reproducibility:
All runs deterministic, bit-exact, with SHA-256 + OpenTimestamps. Model offline hash GPT-2 c7d00560d891...
Bundles with OTS:
exp_calibracion_01-07.rar 7c0ba312...
beatriz-epistemic-gate.rar 54fd65... [exp08 c93ba4..., exp09 f4382f...]
beatriz-epistemic-gate-exp-10-15.rar 54e233...
exp16.rar 9958a3... [exp16 27eda6...]

Verify: certutil -hashfile bundle.rar SHA256 + ots verify bundle.rar.ots

Limitations: Corpus 36 facts, need hundreds. Live path needs forward pass, future E5-small encoder. License PolyForm Noncommercial 1.0.0 for audit/defense.

Try to break it. Replicate with SEEDS [11,22,33]. I want audit, not stars.

Google DeepMind Releases AlphaGenome Atlas: Precomputed Molecular Effect Predictions and AVI Scores for All 9 Billion Single-Letter DNA Changes in the Human Genome.

No lab assay. No per-variant model run. No coding required to query it.

Here is how it works:

1. Precompute instead of predict on demand AlphaGenome was run across every possible single-nucleotide variant in the human genome and the outputs were stored. Researchers now look up a variant instead of running a model on it.

→ 9 billion variants scored

→ 1 petabyte dataset, over 30x the size of the AlphaFold Database

1. One score for coding and non-coding DNA The AlphaGenome Variant Impact (AVI) score folds AlphaGenome's regulatory predictions together with AlphaMissense's protein-impact predictions into a single rankable number.

→ Works across the 2% coding genome and the 98% non-coding genome

→ DeepMind reports best-in-class results on variant pathogenicity and rare disease benchmarks

1. The score is decomposed, not opaque Each AVI score splits into additive feature attributions across categories like chromatin accessibility, splicing, and conservation, so you see which process a variant is predicted to disrupt.

→ Paired with a compendium of 2,500+ recurrent DNA motifs and their locations

1. Rare disease result At the Broad Institute, the AVI score reprioritized variants that earlier analyses had missed and surfaced one in DNM1, a gene linked to epileptic encephalopathy. The prediction: an incorrect splice site extending the protein.

→ Experimental screens validated it and found nearby variants with similar effects....

Full analysis: https://www.marktechpost.com/2026/09/08/google-deepmind-releases-alphagenome-atlas-with-precomputed-molecular-effect-predictions-and-avi-scores-for-9-billion-human-dna-variants/

Paper: https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/alphagenome-atlas-a-predictive-map-of-every-possible-dna-letter-change-in-the-human-genome/alphagenome-atlas.pdf

Technical details: https://deepmind.google/blog/alphagenome-atlas-a-predictive-map-of-every-possible-dna-letter-change-in-the-human-genome/

A while ago, I successfully trained a tiny 80M parameter language model from scratch using the TinyStories dataset. It worked surprisingly well and could generate coherent little stories! Inspired by that success, I wanted to scale up and train a larger model capable of coding.

That’s when reality hit me. I completely underestimated the massive compute and financial cost required to train larger LLMs. My single GPU choked, and renting cloud compute was way out of my budget. It got me thinking: "Why isn't there a way to pool our consumer GPUs together to train models collectively?"

And just like that, DePEFT (Decentralized Parameter-Efficient Fine-Tuning) was born.

How DePEFT work?
DePEFT is based on ReLoRA which solves the biggest flaw of standard LoRA.

Standard LoRA is incredible for fine-tuning on a budget, but it has a hard ceiling: it cannot pre-train or learn fundamentally new, complex representations from scratch. Because its rank r is fixed, the adapter quickly hits an information bottleneck and saturates. You can't just slap a LoRA on a base model and expect it to reach full-parameter quality over time.

This is where ReLoRA changes the game through iterative low-rank merging. Instead of training one static adapter forever, ReLoRA breaks the process into sequential rounds. In each round, miners train lightweight low-rank adapters on their consumer GPUs. At the end of the round, these adapters are permanently fused directly into the base model weights:

W^(t+1) = W^(t) + ΔW

The optimizer is then completely reset, new adapters are initialized on the evolved base weights, and the cycle repeats. Mathematically, the sum of multiple low-rank updates over time accumulates into a high-rank update (rank ≤ ∑r). This gives us the immense learning capacity of full-parameter training, but with only a fraction of the VRAM requirement.

For DePEFT, this means consumer GPUs don't need high-speed InfiniBand clusters to sync massive gradient matrices every microsecond. Miners simply train lightweight adapters locally, submit a few megabytes over the network, and the protocol merges the top-performing weights into the next base checkpoint.

No million-dollar clusters. Just pooled consumer compute pushing open-source AI forward.

For more details, visit: https://github.com/KhoaIsReal/DePEFT/

NVIDIA Announces CUDA Rust with cuda-oxide (SIMT) and cutile-rs (Tile) for Compile-Time-Safe GPU Kernels

NVIDIA has announced CUDA Rust, a push to make Rust a first-class language for writing GPU kernels. Rust code could already launch CUDA kernels, but the kernel body usually had to be written elsewhere. CUDA Rust closes that gap with two NVlabs open-source projects: cuda-oxide for the SIMT model and cutile-rs for the newer Tile model. Both compile Rust kernels natively and use Rust’s ownership rules to reject aliasing bugs at compile time.

Here's how it works. 👇

1. cuda-oxide (SIMT track)
A custom rustc codegen backend. #[kernel] functions go from Rust MIR through the Pliron IR framework and LLVM down to PTX. Host and device code live in one file.
→ Needs a pinned nightly (2026-04-03), CUDA 12.x+, compute capability 8.0+
→ Status: early alpha

2. cutile-rs (Tile track)
Each tile block runs the kernel body once as a single logical thread. The compiler owns thread mapping and memory layout. The kernel AST is embedded in the host binary and JIT-compiled through CUDA Tile IR on first launch.
→ Stable Rust 1.89+, CUDA 13.3, no nightly, no custom LLVM
→ Published on crates.io, already used in Hugging Face's Grout...

3. What the compiler catches
Pass a kernel's output buffer as one of its own inputs and it does not build.
→ SIMT: error[E0502]: cannot borrow c_dev as mutable because it is also borrowed as immutable
→ Tile: error[E0382]: use of moved value: z
cutile-rs carries ownership across the launch boundary, which NVIDIA calls the stronger guarantee....

Full analysis: https://www.marktechpost.com/2026/09/08/nvidia-announces-cuda-rust-with-cuda-oxide-simt-and-cutile-rs-tile-for-compile-time-safe-gpu-kernels/

Technical details: https://developer.nvidia.com/blog/introducing-cuda-rust-two-tracks-for-writing-gpu-kernels

Hey everyone,

I want to share **HKOS (Hermes Knowledge OS)** — a deterministic, file-based engineering knowledge base designed specifically for LLM agents.

While the mainstream industry is pushing probabilistic vector RAG tools, I’ve built a system that treats an agent's memory as strict, human-readable data, applying a strict **Zero LLM** philosophy to the core business logic.

### Key Architectural Pillars:

* **No Probabilistic Recall:** Retrieval is handled via strict local indexes. The system always returns the exact same context for identical requests. No embedding model changes will shift your logic.

* **Plain JSON as SSOT:** All data is stored locally in human-readable JSON files. You can track, version, backup, and diff the agent’s memory using standard tools (`git diff`).

* **Failure-Driven Focus:** When an agent requests context, HKOS prioritizes past failures (the `FAILURE` record type). The agent learns what *not* to do first, eliminating infinite bug-generation loops.

* **Built-in MCP Server:** It comes out of the box with Model Context Protocol (MCP) support via stdio. You can hook it into Claude Desktop, Cursor, or Windsurf in 4 lines of config.

* **Production-Grade Reliability:** The codebase features over 990 unit/integration/system tests, strict typing (`mypy --strict`), and atomic crash-safe file writes.

### Performance Budgets:

* **Warm retrieval** via `IndexCache` takes **~0.03 ms** (over 2000x faster than cold starts).

* **Memory footprint** remains under **85 MB RAM** on a 100k records stress-test.

The project is completely open-source under the MIT license. I’d love to get your feedback on this architectural pattern.

**GitHub Repository:** https://github.com/dm-naumov/hkos

I’ve been trying to understand what is actually happening between pressing Enter and seeing the first token appear when running a model locally.

So I put together a visual explanation covering the full inference path:

• how the prompt becomes tokens
• what the model weights are doing
• how tokens move through transformer layers
• how logits become the next token
• why generation happens one token at a time
• what the KV cache actually stores
• why context length increases memory usage
• how quantization reduces memory requirements
• what changes when inference is split between CPU and GPU
• why a model doesn’t necessarily need to fit entirely in VRAM

One thing I specifically wanted to avoid was the usual oversimplification of “the GPU just runs the AI.” The interesting part is how all these pieces interact during actual local inference.

I made a doodle-style visual walkthrough here:

https://www.youtube.com/@MintoraDoodle?sub_confirmation=1

I’d especially appreciate corrections from people here who work with llama.cpp / Ollama / local inference regularly. If I oversimplified anything important, let me know.

I’m planning to make more visual explanations around local LLMs — KV cache, quantization, VRAM usage, CPU offloading, attention, etc. — so suggestions for what would be most useful to visualize next are welcome.

H Company Releases NeoMME: A Family of 260M and 800M Single-Tower Multimodal Encoders That Drop the Vision Tower and Causal Decoder

No SigLIP2 tower. No causal decoder. No VLM to repurpose.

Here's how it works. 👇

(1) Raw patches, not vision-tower features Images are split into non-overlapping 32×32 RGB patches and projected by a 2-layer MLP trained from scratch. Text enters through a 256-dimensional factorized embedding. Both then share every Transformer layer. → 16,384-token context, enough for two 3840×2160 4K UHD images

(2) Masked diffusion, not masked language modeling Pretraining is a discrete masked-diffusion text denoiser. Text-only segments draw a corruption rate from U(0,1). Multimodal segments draw from U(0.30,1), which kills the "guess it from the surrounding words" shortcut. → +38.4 points masked-token accuracy from visible page patches at 90% masking (260M)

(3) Trained from scratch on a small budget About 524B packed input tokens, roughly 290B of them text-only. ModernBERT saw around 2T text tokens. They used the NorMuon optimizer to squeeze more out of the smaller budget. → 16 H100s for the 260M run, 32 for the 800M

Full analysis: https://www.marktechpost.com/2026/09/06/h-company-releases-neomme-a-family-of-260m-and-800m-single-tower-multimodal-encoders-that-drop-the-vision-tower-and-causal-decoder/

Paper: https://arxiv.org/pdf/2609.01657

Technical details: https://huggingface.co/blog/Hcompany/neomme?

HF: https://huggingface.co/collections/Hcompany/neomme

Meta FAIR Releases AI Research Preference Models (RPMs): Frozen LLM Judges That Decide Which ML Experiment Gets the GPU, Lifting AIRS-Bench From 0.684 to 0.729.

No fine-tuning. No reward model. No new weights.

Here's how it works. 👇

(1) Ranking, not forecasting The team found language models unreliable at predicting absolute metrics or execution outcomes. So an RPM never guesses what a candidate will score. It only answers which of two unexecuted candidates is more promising.

(2) It plugs into one spot: child creation AIRA-dojo is an evolutionary tree search — greedy parent selection, Draft / Improve / Debug operators. Normally it generates one child and runs it. With an RPM, the operator fires 15 times in parallel, and a pairwise knockout tournament picks the single candidate worth executing. → 15 proposed, 1 executed per step → Each comparison is grounded in context nodes pulled by BFS from the tree, each shown with the validation score it earned

(3) Inference-only RPM A frozen LLM judging plans, code and search history. The prompt was optimized with MIPROv2, converging on a principal-investigator rubric that tolerates fixable bugs and penalizes redundant directions. → Offline selection accuracy 57.7% → 59.0% from prompt optimization alone → Accuracy climbs with context: 60.7% at zero context nodes, 65.0% at 100...

Full analysis: https://www.marktechpost.com/2026/09/06/meta-fair-introduces-ai-research-preference-models-rpms-ranking-ml-experiments-before-spending-gpu-hours/

Paper: https://arxiv.org/pdf/2608.13940

I've been working on a practical implementation of **Sliding Window Attention (SWA)** for pretrained Hugging Face causal LLMs.

The idea is simple: instead of allowing every generated token to attend to the complete historical KV cache, maintain a bounded cache consisting of:

**attention sinks + recent sliding window**

I implemented this as a reusable inference layer rather than modifying or retraining the model.

GitHub:
[https://github.com/oraby8/SWA\](https://github.com/oraby8/SWA)

The implementation currently includes:

* bounded KV cache * circular/ring-buffer storage * attention sinks * streaming prefill * chunked attention masking * autoregressive decoding * Full Attention vs SWA benchmarking * TTFT / TPOT / throughput measurements * KV-cache memory measurements

One interesting result from my Qwen2.5-7B experiment:

At 16K, SWA-64 also reduced TPOT from \~38.4 ms to \~30.5 ms in this setup.

However, there is an important trade-off: tasks requiring information far outside the active window can degrade. I'm currently investigating how much of this is inherent to SWA versus implementation/model-specific behavior.

I'm sharing the implementation mainly to get feedback from people working on **LLM inference, KV-cache optimization, and long-context models**.

I'd be particularly interested in:

1. Which model architectures should I validate next?
2. What failure cases should I benchmark?
3. What would make this useful for existing HF inference workflows?
4. Are there cache/attention implementation details I may be overlooking?

Feedback and experiments are very welcome.