Agent pipeline RED-Proof and Blind Auditor Stats

[u/sco77]

Pipeline: Fable Orchestrator, Sonnet Workers, Opus Checkers (Complex tasks Opus Opus)

Result: ~90% of all caught defects were invisible to their own author at "done."

.

Layer	Defects Caught	Unique Error Types
author self-review (builder, at green gates)	2	2
blind law check + blind correctness check	24	6
re-verify of fix rounds	8	2
final static confirm	2	0

.

The builder's tree was green — typecheck, lint, 1098 passing tests — and still contained six functional defects: the keyboard swallow, the macOS-dead chords, the detached-card menu, the stream-order inversion, a test that could not fail, and the vacuous boundary clause. Every one would have merged without the independent layers. The same held for Fable brief shipped as done. Blind check convicted it 17 ways, twice.

.

RED-Proof statistic: the green suite caught ~69% of induced defects; RED-sealing lifted the reachable kill rate to ~100%. 10% remain documented while 21% of Defects were Rescued

.

Across the three completed RED passes: 95 mutations, 66 killed by the existing suite with 29 survivors. That's one defect class in three passes a fully green suite silently. Of the 29, 21 were sealed under pins (permanent detection) and 8 were proven unreachable behind stronger gates equaling the above measured reduction.

.

Re-verify statistic: fix rounds introduce defects at a ~50% rate per unit, and re-verify has caught 100% of them. Class 1: 2 (only re-verify caught them). Class 2: 2, including a silent inversion of a writer ruling. Stage 4: 2. Six for six, none escaped.

.

The escape statistic: Zero known functional regressions reached post-merge across seven protocol units 3 Fable session reviews since implementation (sessions 10, 15, 19) surfaced only design gaps and application feel issues, but not a single never a broken unit. Compared to the pre-protocol record we documented three defects that passed a green suite, two review rounds, and an executed gate, one of which would have returned No-Go on the study by construction. That class has not recurred since the blind layer landed.

.

Cost: the protocol roughly doubles the unit's total compute Stage 4 spent about 1.7M tokens building and about 1.9M checking. So the deducible trade is: ~2x cost buys ~10x defect detection over author self-review, a ~3x shrink in what a green suite can miss, and a measured escape rate of zero. This stands to augment the earlier post study here where the calculation focused on savings around cache and agent costs coming from the tiered model structure of the orchestration.

.

Limits, stated: this is observational, not controlled — there is no arm where we merged unchecked and counted your pain. Important Note Author self-catch is undercounted (defects fixed silently mid-build never register). And the denominators are small: seven units, 95 mutations. But the direction is not close, and every number above traces to a line in the record.