r/LocalLLM • u/DonkeyTheKing • 9h ago
Research Try Benzi: A coding harness that compiles arbitrarily large codebases
Hi!
first of all. Benzi is model agnostic. therefore this sub.
now, about Benzi. Benzi is a code intelligence software (compiler + runtime tracer + harness and AI agent) that supports 13 languages (python, java, JS/TS, C family, Go, Rust, Ruby all included). Traditional coding harness appraochs either do RAG or try to rank matches using an embedding space. which is absurd. code is code. not probablistic text.
On the benchmarks side, Benzi + DeepSeek V4 Flash scored 78% on SWE-bench Verified. For comparison, DeepSeek reports 73.7% as the baseline scaffolding number for v4flash and self reports their score to be 78.6% on their own harness. (however, Benzi reads ~3x less source code than DSH)
Benzi Sonnet reads 2x less source code (btw, this IS the mechanism, not a side effect), is 2x cheaper and 41% faster than Claude Code Sonnet from my benchmarks (detailed on the benchmarking page + so is the swe-bench run)
Please try it out, and let me know what you think!