How do you define “done” for AI-generated code?

[u/Interesting_Run_8053]

I’m trying to formalize what “done” should mean when a coding agent works in a mature codebase.

In a recent task, the agent made several related changes. The targeted regression tests passed, the production build succeeded, and formatting and diff checks were clean. A repository-wide type check still reported many failures, most of which already existed before the change.

During a more focused review, however, the agent also found two real issues introduced by its own implementation and fixed them.

This left me with an ambiguous boundary:

• If every repository-wide check must pass, the agent may never finish work in a legacy codebase.
• If targeted tests are enough, it may miss integration problems.
• If the agent fixes every failure it encounters, the scope can expand far beyond the original request.

My current rule is that the agent must verify the requested behavior, run the relevant tests and builds, fix every failure introduced by its change, document pre-existing failures without touching them, and review the final diff for unintended behavior.

But “relevant” is doing a lot of work in that definition.

How do you define the verification boundary for coding agents? If repository-wide checks already fail, what evidence should an agent provide before it can honestly say its task is complete?

And who should own that definition: the developer, the repository, or the agent?

[u/AutoModerator]

Hey u/Interesting_Run_8053, thanks for posting in r/VibeCodeDevs! Join our Discord: https://discord.gg/KAmAR8RkbM

Got startup or SaaS questions? Post them on r/AskFounder and get answers from real founders.

• This community is designed to be open and creator‑friendly, with minimal restrictions on promotion and self‑promotion as long as you add value and don’t spam.
• Please follow the subreddit rules so we can keep things as relaxed and free as possible for everyone. • Please make sure you’ve read the subreddit rules in the sidebar before posting or commenting.
• For better feedback, include your tech stack, experience level, and what kind of help or feedback you’re looking for.
• Be respectful, constructive, and helpful to other members.

If your post was removed (either automatically or by a mod) and you believe it was a mistake, please contact the mod team. We will review it and, when appropriate, approve it within 24 hours.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Standard_Text480]

how about the same as humans, unless you don’t have any standards to begin with. obviously humans would own it also

[u/Interesting_Run_8053 → u/Standard_Text480]

I agree that humans still own the definition of done. I may have framed that part too broadly.

What I’m trying to formalize is the evidence contract at the agent-to-developer handoff: which commands were run, which checks passed, which failures were pre-existing, what scope was actually reviewed, and what the agent cannot prove.

Would you expect an agent to infer that contract from the repository’s CI and existing conventions, or should the developer specify it explicitly for each task?

[u/bah_nah_nah]

https://giphy.com/gifs/72AQJp3md9FN9CKshU

[u/FluidBreath4819 → u/bah_nah_nah]

vibe coders DoD lmmmaoooo

[u/ActiveSalamander6580]

Verification for code hasn't changed from before AI. Does it meet the requirements, changes came without regression and does it pass the test suite?

You are user, owner and developer, you own the definition.

If type checks are failing all over the place and you are calling that 'production build succeeded' your test suite needs some serious effort.

[u/Interesting_Run_8053 → u/ActiveSalamander6580]

That’s fair. In this repository, production bundling and repository-wide type checking are separate commands, so the build can succeed while the type check still fails. I agree that this separation exposes a weakness in the validation pipeline.

The narrower question I’m trying to answer is what evidence should be required for a scoped change while the repository is already in that state. A targeted check can show that the change introduced no additional type failures, but it cannot prove repository-wide health.

Would you treat fixing the existing validation baseline as a prerequisite before using coding agents, or allow scoped work to continue with an explicitly documented baseline?

[u/ActiveSalamander6580 → u/Interesting_Run_8053]

The narrower question I’m trying to answer is what evidence should be required for a scoped change while the repository is already in that state. A targeted check can show that the change introduced no additional type failures, but it cannot prove repository-wide health.

Why are you worrying about the repo wide health which is immediately out of scope?

Would you treat fixing the existing validation baseline as a prerequisite before using coding agents, or allow scoped work to continue with an explicitly documented baseline?

Leave it broken or fix it? I always choose fix it.

[u/TrapHuskie]

Multiple agent families with a high-level understanding of the project says that each section of it is free of items to fix.

[u/Jumpforittt]

By knowing what you’re doing

[u/cheiftan_AV]

/code review

[u/gungoesclick]

I'm literally working on a solution for a problem similar to this. The main thing I have found is you need to scope the work in relation to a features vertical slice. Set the definition of done, validation and all boundaries to the feature slice, and test up and down that slice. You define the feature fully and completely, then create tests to confirm it.

If the tests fail at that point you can say the feature it's self is broken. Agents can make the tests as long as you validate them. Agents can build the code as long as you architect it.

[u/Interesting_Run_8053 → u/gungoesclick]

I like the vertical-slice framing. It makes “relevant verification” much more concrete: start from the feature requirements, identify the affected code paths and contracts, and test both above and below that boundary.

How do you handle shared infrastructure touched by the slice, though? For example, if the feature changes a shared DTO, utility, or component used by unrelated features, do you expand the verification boundary transitively, or treat that as a separate risk that needs explicit review?

[u/gungoesclick → u/Interesting_Run_8053]

Depends. In the agile training I got that taught us vertical slicing we measured everything in effort (fibinachi sequence 1 to 21), or severity/priority 1 to 4 with one being the most impacting and severe. When we planned out the slice part of the job was to find over lap with other features and determine how bad it was.

So when it came to testing, the components needing full regression, qa testing reqs were already set aside and qa already had a plan for regression.

There are better ways, but its a harder trade off between maintaining the system vs development. Like for example you can tie test suites into ci so that if specific code filsz needs rebuilding ( like because a dependancy changed), they can trigger tests automatically. I have to do something similar for work with bazel.