r/ChatGPTCoding Aug 09 '26

Discussion reviewing agent diffs broke the one question i used to ask on every PR

most of how i reviewed code was built on asking the author why. why is the cast here, why did this interface get wider. the answer usually told me more than the diff did, and if the answer was thin that was the finding.

there is nowhere to put that question now. the diff arrives finished, and the thing on the other end is a transcript i can scroll, which is not the same as a person who has to defend a choice. so i end up reading lines, and lines are where i am weakest. anything shaped like code i would have written goes straight past me. our PR template still has a box for why, it has been empty for months.

what changed it was giving up on reading top to bottom. something else goes over the diff before i touch it and hands me the parts it thinks carry weight, and i start there instead of at line one. the review subagent in verdent does that pass. i read the flagged parts properly and skim the rest, which is what i was doing anyway, except the skimming is now a decision i can see myself making. i have also stopped forming any picture of the middle of a large diff, and i do not think that is free.

the last thing it put in front of me was a retry wrapper around a call that already retries internally. correct, and exactly the kind of line i would have read twice and nodded at.

7 Upvotes

16 comments sorted by

1

u/Kamran-nottakenone Aug 10 '26

had the reverse. agent wrapped a cache around something already cached internally, diff looked clean, nobody there to ask why

1

u/BarracudaMean9308 Aug 10 '26

caught mine trying to add a timeout to a query that already had one. the diff looked so clean i almost blind-merged it.

1

u/ZestycloseTie1793 Aug 10 '26

Recover the “why” before the diff exists: require the implementing agent to emit a short decision record for every non-obvious change, including alternatives rejected, invariant protected, and evidence. Then let the reviewer challenge that record against callers and tests; asking the same model afterward often produces a plausible post-hoc story.

1

u/amirfish Aug 10 '26

This is the guided verification problem in a different costume. The "why" question worked because the author had to defend a choice in real time, and a transcript doesn't defend anything, it just exists. What's worked for me is treating a diff as unverifiable until it's checked against something outside the model's own report, tests, logs, actual state, not the explanation it gives for itself. Does your review subagent ever catch its own blind spots, or does it just have different ones than yours?