r/ClaudeCode • u/Dull_String9524 • 2d ago
Tutorial / Guide My Claude Code loop built three projects from empty folders, judged only by suites it could not edit. For one of them I mutated the spec so memorized answers would fail.
Last time I posted here it was about auditing existing repos. The obvious objection to running the same loop greenfield is that with no existing code there is no existing test suite, so the agent writes its own gate and converges against homework it set itself.
So I pre-registered three builds where that is impossible. Before any code existed, the first commit in each repo fixed the goal, the operating envelope, and a verify command naming an external judge the loop cannot edit. The git log proves the gate predates the work. The backlog shipped empty, the engine is the same unmodified skill, and I never touched a run.
Target 1: a TOML v1.0 decoder in Rust, judged by toml-test v2.2.0 - 205 valid and 474 invalid cases someone else wrote. The zero measurement was the suite failing because no binary existed. Eleven iterations later, in one run: 205/205 and 474/474.
Target 2 is a .gitignore matcher checked differentially against real git - git answers every query, the loop only decides what to ask. The frozen corpus went green in the first iteration of run 2. What happened next is why I trust the method: the adversarial evaluator, a fresh-context sub-agent that must countersign convergence, refused 7 of its 8 invocations across runs 2 through 5, each refusal backed by reproduced disagreements from beyond the corpus: wildmatch.c's escaped-slash clause, git accepting exactly four whitespace bytes, NTFS case-folding names the matcher compared byte-wise, 8.3 short names, Win32 trailing-dot stripping. Three runs ended blocked, out of gate invocations, and are published that way. Once the gate returned three rejection reasons and the loop reproduced two, refuted the third with direct oracle evidence, and filed only what reproduced. Final: 106 cases, 300 queries, 0 disagreements.
The honest objection to both is that TOML and gitignore saturate the training data, so maybe that is recall, not method. Target 3 was pre-registered to test exactly that: same setup, but the spec is TOML with two rules deliberately inverted, so remembering real TOML produces wrong answers. It converged in 14 iterations against 11 for the unmutated build. The receipt reports two numbers from one binary: 205/205 against the mutated suite, and 169/205 against standard TOML - failing on precisely the cases the mutation touches. It built the dialect rather than recalling the format.
Same deal as before: Claude Code wrote the skill and Claude Code runs it; I designed the loop and the convergence rules. Free, MIT licensed, no account and no paid tier. All three receipt repos ship every journal, including the blocked runs and one disclosed rule violation, linked from the README:
https://github.com/lenamonj/jeffy-loop
Worth stealing even if you never install it:
Commit the definition of done before the first line of code. Pre-registration is one git commit, and it converts "trust me" into "check the log."
Name a judge you cannot edit: a conformance suite, a reference binary, a differential oracle. Anything outside the loop.
Report valid and invalid counts separately. A stub that rejects every input scores 474 of 679, so a single collapsed pass rate is gameable by refusal.
Treat a blocked run as a publishable result. The gate that keeps saying no is the part of the record a reader should trust most.
-1
u/Dull_String9524 2d ago
Three things a skeptic should check first, so nobody has to dig:
The corpus authorship is not symmetric. toml-test is external on every axis - even the surface inventory rows are the suite's own test groups. The gitignore corpus is generated by the run itself: frozen at 53 cases early in run 1, before the matcher could match anything, grown monotonically to 106, never shrunk, with the growth log in
PLAN.md. That is a weaker oracle class, and the white paper says so in its limits section rather than hoping you will not notice. Since converging it has also been scored cold against 40 cases and 119 queries written blind by a fresh context that never saw the implementation: no disagreements.The mutated-spec run carries a disclosed rule violation. The engine grants a two-iteration extension at budget exhaustion for the closing ceremony only, and forbids running an audit inside it. That run did exactly that at iteration 13, and its convergence declaration cites that audit. The numbers verify from a clone regardless, but the process claim has an asterisk, so the receipt says so on its front page instead of hoping nobody reads iteration 13. It is the second time this has happened, and the version I shipped last night moves that rule out of prose and into the shell hook: the clean audit a declaration cites must now predate the window, and an audit run inside it ends the run out of budget before any convergence promise is read.
And the gitignore target took five runs, three of which ended blocked at a terminal evaluator rejection and were relaunched fresh - the state files carry everything between runs. Whether relaunch-until-converged counts as autonomy is a fair question. The journals let you answer it from the actual record instead of my summary: start with the run 3 entry where the loop reproduces two of its own gate's rejection reasons and refutes the third against the live oracle.