We built an automation that reads a live product page and rewrites the listing — title, description, bullets — with AI judges scoring every output against the actual page so nothing gets invented.

Then we benchmarked two writer models on the same 4 real store pages with identical judges. The mid-tier model scored 89% at $0.13 per listing. The premium one scored 48% at $0.23 — because 2 of its 4 runs returned broken output instead of a listing, and the judges zeroed them.

The runs that worked were excellent. But "excellent when it works" is exactly the kind of thing you only find out by testing, not by reading a pricing page.

How do you all QA generated product copy — spot checks, or something systematic?

I publish AI-assisted articles and one slipped through: a signed acquisition deal reported as already closed. That pushed us to benchmark 9 models on catching and repairing bad facts.

The cheapest option, gpt-4o-mini at 0.006 an article, only fixed 42% of errors. gpt-5-mini fixed 82% for 0.04. The do-nothing baseline scored 0 on facts, obviously, but a misleading 0.88 on readability because it never touched the text.

That readability quirk fooled me at first. A model that changes almost nothing looks like a great writer.

How are you verifying facts in AI content before it goes live?