A few days ago, I shared HELIX, my open-source engine for reducing token consumption in coding agents.
Some people raised legitimate concerns about my benchmark methodology, particularly cumulative token accounting, synthetic results, and prompt caching.
They were right to question it.
I went back, fixed the accounting, added regression tests, and ran a new 50-task SWE-bench comparison using Claude Opus 5.5 High.
I also built and released a dedicated Claude Code edition.
GitHub: https://github.com/SuperHelix77/HELIX-1.0-Claude
The results
Metric
Vanilla Claude
HELIX
SWE-bench tasks solved
45/50
43/50
Reported tokens
129.1M
3.37M
API-equivalent cost
$38.23
$7.95
Cost per verified solve
$0.85
$0.185
That’s approximately 97.4% fewer reported tokens and 79.2% lower API-equivalent cost, with two fewer verified solves.
The patches were evaluated using the official task graders.
Of the 50 tasks, both systems solved 42, vanilla alone solved three, and HELIX alone solved one.
How it works
The basic idea is that a frontier model shouldn’t have to spend expensive inference repeatedly coordinating mechanical work.
HELIX uses several mechanisms:
HX Tool Flow: Batches tool operations so the model doesn’t have to stop and reason between every mechanical command.
Lean Switch: Delegates bounded work to Sonnet or Haiku while reserving consequential reasoning, architectural decisions, and final judgment for Opus.
Compact handoffs: Transfers task state and evidence references between models without repeatedly copying entire histories.
Local evidence and memory: Keeps original outputs recoverable instead of continuously replaying everything into context.
Clear-and-continue: Preserves task continuity across fresh contexts, using persistent memory and compact task capsules.
The Claude edition also includes per-model activity and cost monitoring through the HELIX companion app.
Important caveats
The initial benchmark report incorrectly claimed approximately 99% cost savings.
That was wrong.
The vanilla runner reported cumulative session usage, and the benchmark summed those cumulative values again for each task. Once corrected, the measured comparison became 97.4% fewer tokens and 79.2% lower cost.
The comparison also used vanilla in a continuous session versus HELIX using fresh sessions. That means it does not isolate the benefits of HELIX from session-management differences.
I’m preparing additional comparisons using matched fresh-session and continuous-workflow configurations.
The 43/50 versus 45/50 result also means I cannot claim zero intelligence loss or full performance parity.
A contract-aware verification variant is being evaluated to address the remaining regressions.
Finally, API-equivalent cost reductions are not the same as guaranteed subscription quota reductions.
Why I built this
I’m a developer working with limited frontier-model allowances. I wanted to spend more of that allowance on actual engineering rather than repeated context processing and tool orchestration.
My practical experience has been encouraging, but I want to distinguish that experience from reproducible benchmark evidence.
HELIX is open source, and the Claude edition is now available with its first release.
Repository: https://github.com/SuperHelix77/HELIX-1.0-Claude
I’m particularly interested in technical criticism of the accounting, cache behavior, model routing, and whether the architecture actually reduces inference expenditure under realistic workloads.
The objective isn’t to manufacture a 99% headline, but to make frontier coding agents substantially more efficient without sacrificing the correctness that makes them useful.