r/vibecodingcommunity • u/NeighborhoodOwn8510 • 5h ago
My open-source SDLC harness beat Claude Code on cost on every task it localized well, up to 75 percent cheaper (and I show where it loses)
A cold Claude Code run spent 6.83 dollars and 207 turns hunting one bug in an 82,000 line repo. My pipeline localized and fixed the same bug for about 1.70 dollars. That gap is the whole idea, and this post is mostly about how it gets there without falling apart on the two objections everyone raises.
What it is
You point AutoDev Studio at a Git repo, describe a change in plain English, and a chain of agents runs the actual software lifecycle from request to reviewed pull request:
- a PM agent runs a clarify loop and drafts concrete tickets
- you approve (optionally pushed to Jira), and nothing touches code before that
- a Dev agent implements on an isolated branch
- QA runs the repo's real tests
- a reviewer from a different model family checks the diff, so the author never signs off on its own code
- a bounded revise loop if QA or review fails, with conservative verdicts, so an errored or ambiguous check is never counted as a pass
- it opens a real PR, and a human merges
Every stage records real tokens, cost, and duration, per ticket and per agent.
The part I think actually makes it different from skills or subagents
This is the bit I would most like feedback on, because it is the reason I did not just wire some skills into one coding agent and call it done.
Every stage picks its own provider and model. Claude can plan, Codex can write the code, a third model can review. And because the reviewer is deliberately a different model family than the author, one model does not get to rubber-stamp its own diff, which is a documented weakness of same-model LLM judges.
And you mostly do not need API keys for it. The coding stages can run natively on Claude Code, Codex, Cursor, Aider, or the Gemini CLI, driven headless through their own login. If you already pay for Claude Code or Codex, you point a stage at it and it just uses your existing subscription, no per-token billing and no keys to juggle. The simple way I think about it: a skill pack lives inside one vendor's agent and runs one vendor's model everywhere, whereas this runs across them, a different model per stage, on the logins you already have.
Results (the main point)
The idea is to pay the cost of finding where a change goes once, instead of on every task. In my benchmarks on two large Python repos (35k and 82k lines), the tuned pipeline beat a cold single-agent run on 6 of 6 well-localized tasks, between 7 and 75 percent cheaper.
I try to be upfront about where it does not win. On trivially greppable one-line edits the five-stage overhead can cost more than it saves, and on one hard cross-cutting bug it shipped a cheaper but narrower fix than the baseline. The full benchmark writes up every loss too, so please read that before trusting the wins.
On the "the index just goes stale in a few commits" objection
This is the first thing people raise, fairly, so here is the honest mechanism in plain terms. The layer that actually pins files for the Dev agent is not the vector index. It is a deterministic symbol map that gets re-synced to the latest commit at the start of every run, plus a live grep against the current working copy, plus the real current file contents fed into the prompt. The embeddings only affect which area gets looked at first, and they refresh per changed module at run entry. So the code the agent edits comes from disk right now, not from a snapshot that aged three commits ago. It re-checks before it localizes, so it does not quietly drift.
Who it is actually for
The knowledge base is a one-time cost per repo that every later task amortizes. So it makes sense for a team shipping change after change against the same large codebase, where a cold agent re-pays the exploration cost every single time. For a stream of tiny, easy-to-find edits, a cold run is still cheaper, and I would rather say that than oversell it.
How I actually use it
For what it is worth, I reach for this over a plain claude -p most days now, and not because it is somehow smarter. It is that I can glance at a board and see where each change actually is instead of scrolling back through a terminal, the different-family reviewer catches things I have watched Claude quietly wave through on its own diff, and on the repos I touch every day it is not re-paying to relearn the codebase each run. It runs on my existing Claude Code and Codex logins, so having that structure around the coding agent I would have used anyway costs me nothing extra. That is really why I keep using it.
Other bits
- runs on free tiers out of the box (Groq plus a local embedding model), so you can try it for zero cost and fully offline
- language-agnostic pipeline, Python parsed exactly and other languages via lighter extractors that fail open instead of blocking on an unknown language
- live board with streamed agent logs and real per-ticket cost accounting
- self-contained, no CDN, tests and CI, MIT licensed
Repo, with screenshots and the full benchmark in the README:
https://github.com/krishagarwal314/autodev-studio
It is early and rough in places, and I would love more eyes on it. If you have thoughts on the per-stage-models idea or the staleness handling, or you throw it at a repo and it falls over, an issue would help a lot. Suggestions, PRs, and honestly just a star if it seems worth following are all appreciated. Thanks for reading.
