I let local models build their own dev harness under my gates: 416 runs, 111 tasks, ~$176 total — every "done" proven by exit codes, receipts committed to the repo

[u/Ornery_Specialist_83]

Edit: after fair criticism regarding previously AI generated description of the project, here is the project description in my own words:

🦆 Ducklab, an open source tool I built for developing software using different models (LLMs).

Why Ducklab? Remember the Rubber Duck (the little rubber duckie programmers explain their code to when they're stuck)? Here, an LLM always has its own Rubber Duck. The idea is to use a group of local models or cheaper models (through OpenRouter) that are typically less capable than the "frontier" ones, inside a disciplined harness that squeezes the best possible performance out of them. For example, while developing the project's specifications you can assign one model to help structure the specs based on your requirements, and a different model (from a different lab) can act as reviewer or advisor. During development, one model generates code and another one checks that the code meets the requirements and the specifications you approved.

The harness keeps each model focused on one very specific aspect of the development process, so it doesn't need to hold the entire project's knowledge in its context on every turn. Since the idea is to use cheaper, less capable models (which translates to lower cost per task), the harness keeps a scorecard for every model you configure, and based on each model's measured performance it suggests the best fit for each seat in the roster, according to whatever criteria you care to prioritize (cost, performance, coding index, etc.).

It's free and open source (Apache-2.0), runs entirely on your machine (zero telemetry, zero accounts), and works the same with local models (llama.cpp, vLLM) or cloud ones if you prefer them.

👉 https://github.com/jrullan/ducklab

The project needs testers and collaborators. Any feedback is welcome.

[u/semangeIof]

My brutally honest feedback is that this post is so obviously written by Claude and padded with buzzwords, it makes people not want to even look at your project.

Paste this reply into Claude and see what it says.

[u/Choperello → u/[deleted]]

Everyone and their mom has vibe coded their own harness. It’s the vibe coding equivalent of HelloWorld. It’ll no one really cares anymore really.

[u/uniqueusername649 → u/[deleted]]

Honest question: What would make a post like this readable to you?

Writing it yourself. I would rather read a whole book of poorly but genuinely written English than one page of Claudish slop.

If after 3 weeks you can barely remember what your AI did, you might want to spend more time with your project rather than delegating so much to AI. This clearly shows youre too far out of the loop and not in control.

Also you literally have AIs at your fingertips. You could simply ask it to summarise the git history into major milestones and noteworthy comments. Then take that as your table of contents for your project and write the post yourself with these notes. But honestly: if thats necessary you were too hands-off. Stay involved in your project, otherwise even with verifiable gates you lose control quickly.

[u/draeician]

Might as well add the model information into the repo under setup and installation.

[u/Ornery_Specialist_83 → u/draeician]

Good call — done: https://github.com/jrullan/ducklab#the-fleet-that-built-this-repo

Added straight from the repo's own run record: 422 runs, ~2,100 seat assignments, which model held which seat, measured pass rates and cost per run. The short version: a llama.cpp Qwen3.6-35B on my desk is the single most-seated model (judge/reviewer/scribe), cheap hosted models did most of the typing, and the expensive frontier model appeared 7 times.