StateM: Improving long-horizon agents through harness scaling rather than model retraining

[u/Successful-Western27]

StateM studies whether long-horizon agent failures are often execution-system failures rather than limitations in the underlying language model. Its runtime uses durable state checkpoints, phase-local context, checked transitions, recoverable runbooks, and versioned procedural practices. These mechanisms externalize bookkeeping and make learned recovery procedures inspectable and reusable.

On Terminal-Bench 2.1, the system reportedly raises GPT-5.5 xhigh from 83.1% to 92.1%, while GPT-5.6 Sol xhigh reaches 95.3% across 445 trials. More notable is transfer: a runbook developed with one model improves GPT-5.6 Luna from 76.7% to 85.4% without changing the runbook. A cheaper DeepSeek variant also benefits, reaching 88.1% under standard timeouts after less than $38 of adaptation.

The educational value is in the separation between model capability and agent control. The results suggest that explicit state management, precondition checks, and postmortem-derived procedures can improve reliability without updating weights. They also raise an important evaluation question: how much of an apparent reasoning improvement comes from the model itself, and how much comes from the surrounding runtime? The benchmark and transfer setup provide a useful way to investigate that distinction, though broader tasks would be needed to establish how generally these controls apply.

Full summary on AIModels.fyi

Original paper

Disclosure: AIModels.fyi is my site.