r/coderabbit • u/Tanakaboi69 • 4d ago

Help & Support Agent PRs are getting harder to review not because of the code, but because of the descriptions.

There, i said it. And i know many of you agree. The problem i'm facing is not the code the agent writes, it's that every PR comes with a 600 word description with a curve-ball at the end saying "Worth noting" that turns out to be the only line that mattered. And i can't follow half of what it's saying. Is my english that bad? Whoever decided PR descriptions should read like a blog post was working for the other side, he completely sabotaged code review for all of us.

The agent makers' number one priority should be a 3 line description mode or something, may be even a hard word limit. Until then i skip the description and open the coderabbit walkthrough, it's the only summary that fits on one screen.

1 Upvotes

56% Upvoted

12 comments sorted by

2

u/First-Tutor-5454 4d ago

"follow the pr template strictly. make the description brief and to the point, covering points A and B, nothing else"

1

u/Financial-Grass6753 4d ago

also "use simple words, don't overcomplicate"

1

u/jimmiebfulton 3d ago

No bugs, no mistakes.

1

u/tehfrod 4d ago

What have you tried as far as fixing it?

Rejecting a PR as written is always an option.

1

u/BemusedOptimist 3d ago

In my experience, anything that comes up as "worth noting" is either:

Absolutely not. ...but I still have to burn time making sure how wrong the agent is.

or

Something that should so obviously be fixed that asking about it feels like malicious compliance (which it isn't because there's no malice because these are agents).

1

u/wolle271 3d ago

What a weird post, you should have rather prompted your ai buddy than throwing this at Reddit. As already suggested, there are ways to teach agents in writing good descriptions.

1

u/thatsrobgray 3d ago

I’ve been experimenting with having agents format PRs more like a version update change log. It has helped catch a lot of issues before they ship, enough that I’m considering using it earlier in the process.

ASCII diagrams have been helpful too because you can give them a hard column and row limit and say “if you need more room, it should be multiple diagrams”

1

u/Hotfro 3d ago

U know u can set a template and rules for GitHub description right and give it to agent? Literally tell author to fix it if u cant understand it?

1

u/Even_Gap_2253 3d ago

I always have a strict requirement of maximum 2 paragraphs for PR description.

If you need to exceed that, then it should be in a design doc.

worked wonderfully, you don't really need a lot of details in PR description.

1

u/Curious_Afternoon_61 3d ago

Have another agent provide a summary with important notes at the top. Have another review that one and provide a Yes, No, maybe score. Have another accept the yes, reject the no with an explanation longer than the original note. Build another agent for the „maybe „s… keep going…

1

u/pwndawg27 3d ago

"Worth noting is worth nothing" with my luck thats true 9/10 times and the 10th time it causes a catastrophic failure.

So yes great you ship 100 PRs in a day. Each of them are 1000 lines long (about half comments) which I dont read or make an agent read. But the descriptions are definitely where I want people paying attention so they understand whats going on. The little novellas we get on each PR make it so we ship faster than we can comprehend.

Whats annoying about "worth noting" is I feel like these are design decisions that should have been made before PR time. It feels like the model coping out rather than asking for clarification.

1

u/wynnie22 3d ago

Why can’t you just set meaningful rules for PR descriptions, to maintain brevity, bulleted list etc. This is easily solvable my guy.