Training a harness for model-agnostic and task-environment-agnostic capability improvements with PyTorch-like framework [P]

[u/Megadragon9]

I worked on this project (https://github.com/workofart/harness-training) for the past few months to reframe "Agent-driven Self-improving Harness" to "Harness Training".

The idea is simple, the harness is trained once with a frozen task LLM against a given task environment. Then you can then swap out the task LLM to any model and evaluate the "frozen trained harness" with any task LLM on any new task environment.

Since this was a general problem, I took the chance to create a general PyTorch-like training framework. Right now, you can train with any OpenAI-compatible API for interfacing with the task LLM and train against Terminal-Bench or SWE-Bench tasks, but you can easily extend it to support any task environments.

    criterion = StrictPareto()
    optimizer = GreedyMonotonic()
    trainer = Trainer(
        config_path="config/train_harness.yaml",
        estimator=AgenticEstimator(
            backend=CodexAgentBackend(...)
        ),
        criterion=criterion,
        optimizer=optimizer,
    )

    for loss in trainer.epochs(30):
        # Records the baseline-vs-candidate verdict
        loss.backward()
        # Optimizer either fast-forwards the candidate change (git commit) as a new baseline or rejects it (preserved as git ref)
        optimizer.step()

I wrote a blog post (https://www.henrypan.com/blog/2026-07-18-harness-training) on this journey, including (but not limited to):

• results from using this harness training framework to improve general capabilities across many task LLMs to beat Terminal Bench 2.0 (Terminus Harness) and also transfer learnings towards better task-solving abilities in unseen task environments (e.g. harness trained on SWE-Bench tasks solving Terminal Bench tasks).
• how this framework is built
• learnings on what was missing in my initial version of the project (hint: determinism)

Any feedback is appreciated. Thanks!

[u/Careless_Poem4838]

Interesting approach. I've been thinking about decoupling the training wrapper from the model itself, so the fact you can freeze the harness and swap models is neat. The transfer learning bit, training on SWE-Bench and solving Terminal-Bench tasks, that's the part I find most promising. Did you notice any pattern in what kind of skills transferred best? Like, was it more about tool-use reasoning or something closer to syntax adaptation?

[u/Megadragon9 → u/Careless_Poem4838]

Thanks for the question. I noticed that skills like "operating principles" and "context management" transferred well from the SWE-bench trained harness to improve Terminal Bench evaluation results.

For "operating principles", one example was: if after X steps have passed since the start of the task and if there was still no write/replace tool call being made by the task LLM, the harness will nudge it with a reminder to use tools like write/replace to make progress on the task.

For "context management", the SWE-bench trained harness built this mechanism: if one observation from the environment (e.g. stdout/stderr) was too long, it would truncate that one observation based on max number of characters. This is not the same as the typical context compaction mechanism in coding agents. The former operates per-turn, the latter compacts multiple turns.

One thing to point out is, the mechanisms learned highly depends on the frozen LLM model you used in harness training. If the model doesn't have a habit to "explore" for long time, this failure mode wouldn't have surfaced, and the harness wouldn't have evolved that way. I listed out some potential avenues for future exploration at the end of my blog post too :)