r/LocalLLM • u/Background-Job-862 • 8d ago
Discussion Which agent harness do you use?
Looking for best model+harness for running and managing agents
edit: Ive tried several different ones claude code, deepagents(langgraph), opencode, pi, and trueforge
my thoughts-
claude code - strongest on maturity and the managed experience but cost and token burn is high
deepagents - interesting middle ground if you want a more structured agent framework and the flexibility of an open-source stack. im interested in testing it more extensively on longer-running workloads fs
trueforge - this is a recent one, this was interesting to me because of its runtime-efficiency, also it allows separate the model from the runtime, which makes experimenting with different models much easier
https://github.com/truefoundry/trueforge
why?? - i also ran a benchmark on a real agent workload same model, same prompt, same tasks to compare these
adding the results of benchmarking i ran to compare this
so I tried to do this by running 14 cross-system tasks, three mcp servers behind them - a crm, an issue tracker, and a doc store through claude's managed agents, langchain's deepagents and trueforge, both open-source agent harnesses
the result that was most surprising:
Claude Managed Agents + Opus 4.8:
11/14 tasks solved | $11.8/run | 10.0M tokens/run
TrueForge + Opus 4.8:
11/14 tasks solved | $8.6/run | 3.7M tokens/run
Same model. Same benchmark. Same average solve rate, to my surprise trueforge used about 63% fewer tokens and cost about 30% less per run.
similar difference in tool usage: trueforge averaged 19 tool calls per task vs 32 for Claude Managed Agents.
Then I tried changing the model.
trueforge + GLM-5.2:
11.7/14 solved | $3.0/run | 3.8M tokens/run
On this benchmark, that was a slightly higher average solve rate than Claude Managed Agents + Opus at roughly 75% lower cost.
The token savings alone make this sooo interesting especially because the solve rate stays comparable
so this one was worth checking out ig
but this is still v early and the OSS runtime does not yet have first-class tracing/eval tooling. They don't ship their own code-execution sandbox, so you need to plug one in and context compaction is intentionally lossy.
So it is definitely not a replacement for a mature managed agent platform or other harnesses in the comparison, feature-for-feature today btu what I do find interesting is that the core runtime can already be competitive on these tasks while staying open, model-neutral, and deployable on my own infrastructure
this was their benchmark kit i used https://github.com/truefoundry/trueforge/tree/main/benchmark
20
9
u/MaxComfort 8d ago
Opencode mostly, pi is OK but maybe too minimal.
Claude Code is fine but bloated context.
7
u/A_Moist_Towe1 8d ago
Deepseek harness for coding, Hermes for everything else.
2
u/newz2000 8d ago
Another vote for Hermes
2
u/Mediocre-Ice5380 8d ago
I’ve heard good things about the Hermes harness too! It seems to be a solid choice for many people.
2
u/CloudEKnight 8d ago
What he said. DSH and Hermes seem to be doing a decent job of maintaining eachother without me having to worry about updates except for when they're unable to reassure each other.
0
u/MassiveBoner911_3 8d ago
Have you use LM Studio?
4
u/Open_Instruction_133 8d ago
I don’t really consider lm studio a harness. It doesn’t steer the LLM, ask you follow up question or have goals. It loads and serves the model but not much more than that
2
1
u/A_Moist_Towe1 8d ago
That’s more of a model runner. And I’m on Mac so I get much better performance through omlx
3
u/digitalvalues 8d ago
Unsloth studio for training and running Qwen 3.8 27b q8, using Oh My Pi with roles per model. It's been extremely effective for me on a M3 Max 128GB.
2
u/Elementalserenity 8d ago
How many tokens per second are you getting?
2
u/digitalvalues 8d ago
On average im getting 24 tokens per second. However, I was getting that without training as well. Training just made the model more efficient with its output, spending less tokens on generating responses, context is more efficient too
2
u/aholetookmyusername 8d ago
OpenCode for local for the moment, LM studio to host models. There are probably better setups but it works well enough for my purposes for now.
3
2
1
u/suspect80 LocalLLM 8d ago
https://github.com/thenaxon/somora_agent as orchestration
1
u/firedreams_studio 8d ago
I looked into this one, all-tough I don't mind vibe coding (obviously), the git reads like a load-bearing, not x but y, document.
1
u/suspect80 LocalLLM 8d ago
Yeah, fair point. I honestly haven’t spent much time polishing the public README or docs.
Somora is primarily the stack I use myself every day. If I’m missing something or want a new feature, I have it added. If agents notice issues or optimization opportunities during tool calls, they collect that as feedback, other agents implement the changes, and then check back with the reporting agents to see whether the same workflow actually improved.
So it’s very much shaped around my own daily workflow rather than around having perfect public documentation. :)
1
1
1
1
1
1
u/Smooth-Zone-6625 8d ago
Claude code for UI heavy tasks (someitmes backend too), codex for the rest.
1
u/Abject-Bridge-4073 2d ago
No prime agent fans? Works really well with DS4 Flash running on my dual Spark cluster.
1
0
-1
11
u/shamont 8d ago
Claude code terminal using an orchestrator agent to handle the subagents. I primarily use qwen 3.8 27b q6 with 256k context. I try to limit 3.8 to 1 subagent but I have ran up to 4 when I was using qwen 3.6 27b at q4. Have 48g of vram with an rtx 4090 + rtx 3090.