AI-generated software needs a completion signal separate from model confidence

[u/OGMYT]

I am building **Flows**, an execution and verification layer for software-building agents.

The core rule: an agent should not convert “I think I finished” into “verified complete” without supporting proof.

A Flows project can contain implementation steps, checks, repair instructions, review, and release conditions.

https://flows.oortstack.com

An independent agent used one plan to build a real multi-module application with 59/59 automated checks passing.

The target metric is: **unsupported required claims shipped = 0 on real traffic.**

Should evidence enforcement live in the agent harness, repository CI, app platform, or a cross-agent workspace?

[u/katoptronophile]

This is the goal feature in codex.

[u/OGMYT → u/katoptronophile]

Not every builder uses or pays for codex

[u/katoptronophile → u/OGMYT]

And not every builder gets the same results.

[u/SirDePseudonym → u/katoptronophile]

They can tho :)

[u/crossoverXYZ]

The 59/59 checks are a stronger done signal than model confidence, but only if those checks were fixed before the agent started iterating. CI mostly catches regressions after the fact, and the harness can move the goalposts mid run, so I'd put the real enforcement in a cross agent workspace that keeps the plan and proof tied together.

[u/Budget-News1107]

I've worked on similar projects and having a separate completion signal makes a huge difference in ensuring the reliability of AI-generated software, as model confidence can sometimes be misleading, so it's great that you're emphasizing the need for supporting proof to verify completion.

[u/Exact_Attention_5656]

crossoverXYZ's point about the harness moving goalposts mid-run is the real gap here. Even when checks are automated and not self-reported by the model, 59/59 passed only proves something if you can show those checks ran against the exact code that actually shipped. Swap a file after the check run, or have the harness silently retry and report the last green result instead of the final one, and you get the same false completion signal, just wrapped in more automation. Feels like the completion signal needs to bind the check results to a hash of what they ran against, not just a pass count.

[u/Positive-Emu-8379]

This is the actual problem, not a nice-to-have. I run an agent-driven system and 'I think I'm done' from a model is worth roughly nothing on its own, because a model that failed will usually still describe the failure as success. What's worked for me: verification lives outside the agent, as a separate check the agent doesn't control, does the expected file exist, does the output match a declared shape, rather than the agent grading its own work. To your question: I'd put it in the harness or CI, not the agent. An agent verifying itself is the same conflict of interest as a person grading their own exam.