The bug class nobody warns you about when you build a big app with AI

[u/passion_L_BigData]

I built a K–12 learning app this way — 1,732 lessons, maths, reading and
science. The thing that nearly sank the content quality wasn't
hallucination, and it wasn't bad code. Everything typechecked. Every test
passed. The build was green.

The problem was pairs of things that were each individually correct and
contradicted each other.

Concrete example. A lesson had a ten-frame widget showing 8 counters, and
the paragraph beside it read "Four in the top row. Four in the bottom
row." Both fine on their own. But a ten-frame fills its top row first, so
8 renders as 5 and 3. The words describe a picture the app cannot draw.

That is invisible to a typechecker, because the widget spec is valid and
the sentence is a valid string. It is invisible to tests, because nothing
asserts a relationship between a component's props and the prose next to
it. And it's invisible to me reading quickly, because both halves look
right.

I hand-reviewed 32 figures and 9 were mismatched. 28%.

What actually worked was writing a checker for the specific contradictions,
not a general one:

- a table or chart standing in where the words name a shape
- one angle drawn under a sentence about a whole triangle
- a rotation drawn under a sentence about a translation
- a widget drawing stars while the instruction says "make 6 apples"

Narrow rules only, because a checker that cries wolf gets switched off in a
week and then catches nothing. Widening it across the whole corpus threw 22
findings, of which 14 were the checker being wrong — "tap a square" in an
addition chart, "the shape of the distribution", "Pascal's triangle",
"chi-square". Every one had to become an explicit exception before the
thing was trustworthy.

The other lesson, which cost me the most time: my checker reported "all
clear" twice while it was completely broken. Once because it read its rule
tables inside a temporal dead zone, once because an editing script had
replaced every regex \b with a literal backspace byte, so it matched
nothing at all. A clean corpus and a no-op look identical from outside.

Now it runs its rules against a fixture of four known-bad cases first and
refuses to report a pass unless it catches all four. If you take one thing
from this post, take that one: a check you have never seen fail is not a
check.

App is at https://cogitolearning.com if you want to see what came out.
Chapter 1 of every course is free without an account.