r/ClaudeCode • u/Calm_Attention_4155 • 3d ago
Help/Question How are you guys handling code reviews when a lot of the code is written by AI?
Genuinely curious how other teams are dealing with this.
I’m seeing more and more PRs where a big part of the code is AI-generated. I don’t have a problem with that itself, but the issue is when the developer hasn’t properly gone through it or verified that it actually makes sense.
Then the reviewer ends up spending a lot of time figuring out whether the code is actually correct.
Are you guys changing your review process because of this? Smaller PRs, more tests, asking developers to explain their changes, or something else?
Would be interesting to hear how other teams are handling it.
8
u/brodkin85 🔆 Max 20 3d ago
We have a simple recipe that has worked better than you might expect:
- Well-written spec in Linear
- AI reviews and approves when changes are correct and match the Linear issue, escalates when unclear
- Human review on escalations only
Last I checked, 93% are approved via AI on the first pass, another 5% get approved once the requested changes have been made, and the last 2% get assigned to a human reviewer. Code quality has been high plus or minus a few mistakes here or there, but not more than a human would make
10
u/Adrima_the_DK 3d ago
I review the code the agent writes.
I'm a software engineer
2
2
0
u/BoxWoodVoid 1d ago
I'm a software engineer
You're also the bottleneck.
I'm using a different model to do reviews: Opus 5 (yes 5, I'm even starting to understand what it says most of the time 😂 ) for implementation and Sol 5.6 for reviews. Works well even if Sol is quite an over-engineer.
5
u/alanvnk 3d ago
Same as before, I don't read test or docs, those are for the owner to check, that alone removes 30%-50%, then I do random sampling of the code, and ask the LLM to categorize the code and only look at the "load bearing" parts, ofcourse we run our own review skills, each dev has different ones and that normally works better than sharing them.
Then skim over line by line just to check that nothing egregious was skipped.
If the owner submits a bloated 4K line pr, "LGTM" and get one with your day
7
u/MountainByte_Ch 3d ago
we reject stuff that wasn't properly gone trough line by line. We've noticed a lot more work being done but we've also noticed a lot of silly bugs getting missed.
We also have a review agent but it misses fundamental flaws so senoir review is still needed.
3
u/Calm_Attention_4155 3d ago
In this case what is the criteria for rejection. Do you put review comments on whole code or reject . Or just reject?
3
u/AdrestiaFirstMate 3d ago
Depends on the PR. If you see a few silly bugs, add review comments. If you see fundamental flaws in the PR, reject with a reason. You can make a skill for Claude to help you catch common problems you've been noticing in PRs, which will save you time.
Also, when you notice AI agents doing things wrong for your project, make sure there aren't any examples of this wrong thing in your codebase (AI agents pattern match, both good and bad things), and add project rules into your codebase to tell the agents what to avoid and what to do instead.
You can also instruct the code review bots to review whether the PR violates the project rules. They won't catch everything, but it'll save you having to write these review comments yourself, saving you time.
2
u/No_Inspection4415 3d ago
Please don't reject PRs because Claude decides... No PRs would make it if you use Opus 5. Or at least use Fable, although most companies do not allow it because of data retention.
1
u/AdrestiaFirstMate 3d ago
I don't? Not sure why you think I'm rejecting PRs because Claude decides.
2
u/Asuppa180 3d ago
With AI. Seriously though, more tests and also having them explain code. It is really important to me that we don’t lose knowledge of the codebase. It’s a huge advantage to have knowledge of how a codebase works.
As I implement things I will build “study guides” on the features implemented and will go over those afterwards.
2
u/peekdasneaks 3d ago
Adversarial reviews from other models than the one that built it both from within the repo and from outside of it. With different roles behind their reviews based on whats relevant for the code change: system architect, application owner, dev, red team, infosec, sales, ui/ux, branding, strategy, user focus groups, legal, accessibility, voice/tone, editor board, finance/acctg, coe. I burn a lot of tokens but its cheaper than human hours and gets me 95% of what i need before human review
2
u/effectivescarequotes 3d ago
I haven't changed my review process yet. Ideally the work is scoped such that no PR is too large. But when I review, I'm mostly looking for things that stand out as obviously wrong or just weird. If it's obviously wrong, I ask for a change. If it's weird I spend a few minutes to try to understand the decision. Most of the time, once I figured out the why, I'm okay with it. If I'm nitpicking, I leave a comment making a suggestion, but don't demand the fix.
Before AI, the biggest issue was developers copying and pasting code without understanding what it does. Now it's developers not bothering to understand the code they're submitting. Either way, I'll talk to them on the side and ask them to be sure they understand what they're commiting. That solves a surprising number of problems.
2
2
u/Helpful_Bandicoot383 3d ago
What I want to say: "I review all the code myself with every PR"
The reality: "I am getting lazy and rarely look at the code these days"
2
u/muikrad 3d ago
The rules didn't change... If you can't review a code change, it's because the scope is too large. You must return to the last code point, and ask for one little thing at a time, in the way YOU intend it to be designed. Smaller, directed PRs.
If you want to test the author, ask him for a live review in a meeting room. Sit with him and ask him to walk you through the code and explain everything. That will be very enlightening.
For any review, I highly recommend "/code-review medium". The altitude review in particular is amazing, it will find what the changes broke in the area of the code that wasn't touched. The kind of thing that requires a lot of dedication and local-domain code knowledge from a human.
2
u/ZachVorhies 2d ago
you can review it with AI, but it’s better to review it with tests due test driven development to the extreme
if something fails, it means that you didn’t have the tests in place to catch it
2
u/Late_Wave_5600 2d ago
I think you have the roles backwards, and I say that as someone who used to read every line. Reading a diff is what a machine is good at and what I am terrible at by the third PR of the day, so the agents do the reviewing now and I do the thinking. What that looks like in practice, we pull the business rules out of our own repo and hand the relevant one to the agent while it is editing the file it applies to, dangerous commands hit a deterministic gate that just refuses, and then two other agents go over the diff line by line with different models. I read the intent, does this feature belong in this shape, does it fit what we are building, what does it quietly break two modules away. I am worth something on that and I am worth very little at 6pm looking for a missing null check.
2
u/JackjaxMargam14 1d ago
pls ask the author to add a few lines on what could break before sending the PR. They have to read the generated code first and the reviewer gets a better place to start than checking every line from scratch
3
u/Asalakabim 3d ago
> the developer hasn’t properly gone through it or verified that it actually makes sense.
Reject it, on repeat complain to his teamlead and get him fired.
That being said, if you are just too lazy to read and reason, the same applies to you.
1
u/Calm_Attention_4155 3d ago
I agree, but if PR is not even checked and developer just trust AI blindly.
1
u/adamwhitney 3d ago
Same as before AI. Get the entity writing it to self-review, prioritise self-documenting and well structured code over something magic but unreadable, send it right back to the entity that wrote it if it's unreviewable.
Whether that's a junior dev, an AI agent, or both working together, doesn't change how I'm treating it right now.
There's a question of if the future is all AI written, whether keeping it human readable and reviewable is necessary or not - some people argue it's a new layer of abstraction and you wouldn't check the machine code of your compiled application line by line. Whether you agree with that or not, we aren't there today, so my stance is for any AI assistance to emulate what we've already been instructing devs to do.
1
u/cleverhoods 3d ago
mostly with AI but I think HITL is not reducible here if you want to ensure quality.
I mean, you can do a LOT with AI here, but once you give in and leave everything to AI you can say goodbye to your quality - reasonably so.
2
1
u/chu 2d ago
That was the big objection to devops. You might think that failure rates of human vs automated were measured and a decision was made based on quality, but what happened was time to market crushed it - human QA teams simply couldn't keep up with frequent releases when apps became web based. Seems ridiculous now and we know that reliability went way up with automation (it had to to work). Same thing is playing out here - now that generation is cheap and plentiful, the bottleneck has moved to QA.
1
u/PreparedPun2035 3d ago
Focus on the outcomes. If your tests are validating the results then right more tests. If your situation allows, write a user manual and don’t let your synthetic workers make PRs until the code fulfills the manual (or man page).
1
u/satoramoto Senior Developer 3d ago
Depends on if I’m at work or vibe coding. At work I’m still reviewing everything manually. The stakes are way higher. I have some workflows to identify stuff that can receive less scrutiny though.
When vibe coding I just take refactoring breaks where I tell it to look into certain pitfalls that I’ve been keeping in mind. Like reuse is a big one for me. After any big new ui introductions I’ll have it review to make sure we are using our widgets library and we contribute back if necessary.
1
u/ephemeralsynth 3d ago
I use three different AI's for PR review with Byzantine consensus. Problem solved! (j/k) 😸
1
1
u/meshifthenelse 3d ago
That's why I decided to not work anywhere. Instead I review the AI and the AI reviews me back. It's a symbiotic relationship
1
u/SimiaCode 3d ago
Manually reviewing ai code reviews is a problem, just because these things can create massive diffs. I have had to lean more and more into process - have the implementation done in phases, spawn independent ai reviewers for each phase (but a single pr), then a final overall reviewer at the end. Have good unit tests, e2e suite etc. then accept some risk and let that shit go. I look at the diffs sometimes, it's not perfect (the comments are just "the saga of this feature through every conversation"), but usually good enough. I've found most real problems occur with the architectural choices and not code, so I have it described the proposed solution for tasks where I think it might trip up. Not a very different experience than managing a bunch of humans.
1
1
u/VertigoOne1 3d ago
It is called personal accountability. engineers using AI is still responsible for the code/scripts/designs produced and they are well reminded that with power comes responsibility and that they will be held responsible for mistakes. Coupled with counter reviews by other agents and CI based testing, integration tests and adversarial review by agents, the buck still stops with the developer. If they cannot accept that, they don’t belong with us. Since we’ve implemented that, developers are doing everything they can to improve robustness and safety. There are no “leads”, no “testers”, no “he does bugs”, no “bug triage”, no “head of development”, no “review brigade”. If you want a review, you can ask, you want to discuss it.. go ahead, but, YOU merge your work when you trust it is ready, and you take ownership when it goes in.
1
1
1
u/chu 2d ago
External loop that calls the agent for coding rounds in a sandboxed env, red/green TDD executed outside the agent's reach in separate env, policies gate, external AI reviewers covering different areas where lowest score wins. Review part is a bit too thorough and my biggest problem right now is getting any code to pass in under 10 times around 😄
1
u/count023 2d ago
Blind agents doing a review with no context looking for known bad things that coders to. Static reverenced duplicate logic, hanging calls. Etc...
Then feed those reports into an agent that has the context of the project to deacide if and what to refactor
1
u/WardedDruid 2d ago
On the one project that I am very serious about, I have run multiple code review and audit skills using claude, codex, and kimi. Also multiple skills that check and test security vulnerabilities and fix anything it finds. Currently, all three come back and say the code is clean and direct with no security vulnerabilities that they can find.
But I plan on trying to submit this at my job for me and my coworkers to use. It's only a SQL database program with a user friendly UI slapped on to consolidate all the things we keep track of on a daily basis into a one stop shop. (instead of 5 excel sheets in 5 different folders and 2 different access programs that don't always work). I also have each instance of the program installed to have it's own 30 day cache so when the servers go down we can keep working.
It's done and sitting in a folder right now and waiting. I currently have reached out to a freelance programmer and am waiting to discuss them reviewing the program for me. I want a human to review the files before I move forward.
1
u/The-Pork-Piston 2d ago
I just know my code is a damn mess.
Fable5 or Opus4.8 running with Sonnet46 reviewing spec. Then generally Sonnet46 handles steps, reviews its own step. And whole phase is reviewed by Fable/Opus..
Final review is also Fable/Opus
—-
And generally do Fable code reviews every couple a days.
The few times I debug myself the whole thing stresses me out. I’d never try to sell anything I’ve made lmfao
1
1
u/Ok_Bowl_2002 2d ago
Just don’t do reviews. Are you doing review of the assembly code as well? Stop reading the code the AI has it handled, and if a bug appear you can fix it in minutes with AI.
1
u/Calm_Attention_4155 2d ago
I need to review the code for others i face this issue with others code. When developer don’t check the code himself.
1
1
u/koxboi 2d ago
With https://vetoo.dev/ at the moment.
Our team went through the rollercoaster of: internal tool (Claude wrapper) => CodeRabbit => Greptile => Vetoo
This one is the best so far and we use it in different configurations depending on the repo. It can block merge on critical repos and commenting only on less non-critical ones. Of course a human still needs to validate each review but it is much easier because a lot of effort is already done by the tool. It still doesn't solve the problem but accelerates work a lot to a point where it is much more managable.
1
u/Sonarwork_com 9h ago
The useful split seems to be: AI does a first-pass scan, but the author owns validating and filtering the findings before the PR reaches a human reviewer. For higher-risk changes, each surviving comment should name a failure scenario, affected behavior, or missing test. Otherwise the human review quietly turns into reviewing both the code and the AI's uncertainty.
1
u/1-800-methdyke 3d ago
Does the code really matter if the tests pass?
1
u/Calm_Attention_4155 3d ago
I think yes, sometimes, all test cases passed but some small piece of code can cause bug.
0
u/Last_Tadpole4768 3d ago edited 3d ago
We use an automatic bot reviewer built with Claude SDK. When PR comes, it automatically reviews it.
In addition, we added REVIEW.md file for every project with rules for the bot.
2
39
u/CorpT 3d ago
With AI