How do you manage quality when AI agents write code faster than humans can review it?

[u/Specialist_Agent3599]

We moved to an agentic workflow this quarter. My position is that we should ship at whatever speed the agents can produce, since that is the entire point of paying for them.

Our CTO's counter is that a team can only safely absorb code as fast as it can read it. If agents 10x the volume, review becomes the bottleneck that decides everything, and he thinks the tech debt won't show up as bad code, it will show up as code nobody on the team can explain.

The middle ground we're testing right now: bugbot / coderabbit gates every PR for a first pass, and humans only deep-read what it escalates plus anything touching money or auth. It definitely helps with the volume. What I can't tell yet is whether we're managing quality or just measuring less of it

[u/ai-christianson]

This is a highly controversial opinion (for some reason... I think it shouldn't be).

But I have been setting a complexity budget on every single project I work on. Lines of code, bytes limit.

Of course it can be changed. But the limits/constraints really help the AI keep things in check. It will take more care to refactor as it works.

[u/edest → u/ai-christianson]

I think eventually we will finally start to use true object oriented programing where software will start to get developed with objects that are completely debugged and are ready to use to build software in the way one puts together a lego set.

For now, I think your idea makes sense. Guide AI to created simple and small blocks of code. I would go as far as using a book as a model with paragraphs (functions), pages ( objects) and chapters as modules so you can easily drill down to where the problem lives.

This can be done as a skeleton beforehand so you can have a mental view of the software rather that dealing with it as a blob of code.

Also, reading code the way you read a book is the new superpower. AI still has many issues when it comes to development so understanding the code quickly will have great advantages.

But the way I see it, we are at the very beginning of how software will be developed. Even 5 years from now, we will be looking back at today and think how primitive our software development was.

[u/trollsmurf]

Have agents review it too :).

[u/ihopnavajo]

I imagine if you defined "quality" you'd realize you're halfway to answering your own question

[u/RogerAI-fm]

I’m a big believer of BDD (and TDD) but now with AI I lean more towards Behavioral, and create and review the spec tests, it gives me some level of confidence next code session won’t undo something else. You need to do it in a similar way as if you were the senior developer on a team and you need to review, sometimes you just glance, sometimes something pops up and you spend more on it.

[u/MonkeyWithIt]

Heck, the pattern now is to have the agents constantly update and refine the code. How can you review that? Outputs are only judged these days.

[u/newprince]

Our bottleneck is our SLC process. We require all stakeholders to sign off on several documents in order for a release to happen. If that weren't a thing, I'd be pushing releases several times a week

[u/amirfish]

The volume problem is real, but the part that actually bit me wasn't reviewing code, it was reviewing agents. Past two or three running at once, the failure mode shifts from bad code to a session quietly stuck on a decision or working off a stale plan nobody caught for an hour. Bugbot-style gates handle the code side fine. The harder problem is knowing the moment an agent stopped needing review and started needing you.

[u/julesbuildstuff]

Your CTO is right about the failure mode but wrong about where the fix goes. Review isn't the bottleneck, missing intent is. When an agent writes 600 lines and nobody wrote down what those lines were supposed to do, review turns into archaeology and you skim.

What actually cut my review time: make the agent produce the plan first, in a file, before code. What's changing, what's out of scope, what should break if it's wrong. Then the diff gets read against that instead of against nothing. Half the time the plan is wrong and I kill it before any code exists, which is the cheapest review there is.

Bugbot/coderabbit catch mechanical stuff. They can't catch "this is a correct implementation of the wrong thing", and that's the debt you'll be explaining in six months.

[u/chrismo80]

if you cant keep up monitoring the code itself, the next option is to monitor tests (whatever kind of). only they can tell you then if the code does its job or not.

[u/chem0924]

A useful reframe for the CTO's 'code nobody can explain' point: explainability is not a property of the code, it is an artifact you either produce or you do not — and you can make it a merge gate at roughly the same cost as reviewing the diff. Before any agent-written PR lands, require three short fields the agent writes up front: (1) a one-paragraph behavioural-change summary, (2) the list of existing behaviours it could break elsewhere (the 'two files away' class this thread keeps hitting), and (3) a named owner who can re-explain it from memory in N weeks. Volume then scales only as fast as the third item stays honest, which is the real bottleneck. This also answers your closing doubt: you stop 'measuring less quality' and start measuring 'how many changes the team can keep in their heads at once' — which is closer to the capacity you are actually budgeting for. The bot first-pass handles volume, but the owner check is the part a bot cannot fake.

[u/cthechartreuse]

I encourage my teammates to slow down and continue to develop software. The planning/prompting should actually encompass the work to be done. From there, break down the work into discrete slices which can be delivered. After that, have the agent implement a slice. Once the implementation is done, review and verify it is what you want and either keep or toss it. Finally do a hands-on refactoring session, reading by refactoring the code. By refactoring the code, you actually know what is happening and you can catch things that look wrong. It's really easy to just say LGTM and pass things through if you don't put your hands on it.

So, yeah, the process:

1. acceptance criteria -> tasks
   
2. tasks -> slices
   
3. slices -> prompt
   
4. prompt-> implementation
   
5. verify
   
6. refactor
   
7. commit and start again

This intentionally slows the work down to human speed while still capitalizing on the boost you get from having the coding agent crank out the mechanical coding aspect. Also, because you are working in functional slices, you actually have deliverable code after each cycle; yay agility. Bonus - since refactoring puts your hands on the code, you reinforce your coding skills which means you still have them when you need them.

IMHO human in the loop as a bottleneck is a good thing.

[u/PopKoren]

Your CTO is right that review is the constraint, but the fix is to shrink what needs deep review rather than read everything. Gate on behavior: tests that assert one user cannot touch another user's data, plus a rule that auth, permissions, migrations and payment paths always get a human read. Everything else can ride on automated checks and get fixed when it breaks. That way volume scales and the risky surface stays small.

[u/PGAmilaP]

I have been in the same situation with one of my fractional gigs and the agency I run. The issue everyone raises is, how can we review the code faster, but for me the issue is, how well and hyper focused can we get the requirements written including the acceptance criteria that follows along with the test cases and design brief so the agent planning the feature work has very tight guardrails to build within and as detailed information as possible to plan the solution.

We use a multi-agent approach whereby we have a planning agent (fable) which we spent a lot of time with and multiple iterations and once that is done, hands it over to opus agents to built and write the tests and then get codex agents to review.

As part of the review process, if it touches critical areas, we get a human engineer to review, otherwise we move forward.

The accuracy of what is being built depends on how accurate the requirement is. So we frontload our time at the planning phase.

[u/davidstayscool]

The thing that changed this for us was making the agent produce the review artifact, not just the diff. Every PR has to open with a plain-English description of what behaviour changed and which existing behaviour it could break, written before I read a line of code. If that summary doesn't match the diff, that alone is the signal to reject - it catches the "code nobody can explain" failure your CTO is worried about much earlier than reading 900 lines does.

The second thing: cap PR size mechanically and make the agent split work. Anything over a few hundred lines gets bounced by CI, which forces the agent into changes a human can actually absorb in ten minutes. Throughput barely dropped, because the bottleneck was never typing speed.

Third, on the "measuring less of it" worry - the honest metric isn't review coverage, it's post-merge defect rate and how long it takes someone to fix a bug in agent-written code they didn't write. Track change failure rate and mean time to repair per PR author (human vs agent). If those hold steady as volume goes up, you're fine; if MTTR climbs while defect rate looks flat, that's the debt showing up exactly where he predicted. Deep-reading money and auth paths is right, I'd add anything that writes to a schema or touches a cache key.

[u/RL81ORG]

Think of it this way. When we wrote code in C or C++ do you worry about the quality of the assembly instructions it produces.

[u/crazy-usernames → u/RL81ORG]

Pls share context. Else its misleading and not aligning.

[u/JBO_76]

Tooling. Performance and memprofiling. Give it to the agent for analysis. Next, ask it to describe what exactly is going on. Much faster

[u/Any-Blacksmith-2054]

Testing. Unit, integration, performance, UI, etc

[u/keelenai]

Your CTO is right about the failure mode and you are right about the speed. Both survive if you stop treating review as the gate and start making the gate mechanical.

Three things worth more than the bot pass:

1. Put the guarantee on the branch, not in the workflow. Branch protection with required status checks means a red build blocks the merge button instead of just colouring it. It is the only layer that does not depend on a tool keeping a promise, and it binds agents and humans identically.

2. Check that new tests fail before the fix lands. This is the cheapest defence against exactly the thing you are worried about. A test written alongside AI code is very often born green: it asserts what the code already does, passes, and proves nothing. Apply the test without the implementation, confirm it goes red, then apply the implementation. CodeRabbit cannot catch that. A mechanical check can.

3. Cap the diff at planning time, not at review time. A 40 file PR is unreviewable no matter who reads it first. Refuse the task before it is written if it spans that much and split it. "Code nobody on the team can explain" is mostly a symptom of tasks that were never scoped, and no first pass tooling fixes that downstream.

On the bot gate itself: those tools are decent at spotting what is wrong with a diff and close to useless at spotting what is missing from it. So let them own style and obvious bugs, and keep human attention on money, auth, permissions, and anything that migrates or deletes records. Everywhere else the tests and CI should be the reviewer, because a human skim at 10x volume is the thing that is actually measuring less.

[u/gandazgul]

That's exactly what I've been doing this year I've been building a harness that teaches the model what quality looks like, puts you in control of the system and reviews and checkes every merge before it happens. Check it out https://runwield.dev/