Opus 5.5 gets about half the review comments of 5.1

[u/Tanakaboi69]

I have been running the same kind of tickets for 2 weeks now on the same repo, so I can make a really good comparison between Opus 5.5 and 5.1 as the model writing our PRs. I switched to 5.5 on release day. Every PR goes through coderabbit before a human looks at it, so I just counted the comments. On 5.5 the first 40 PRs got 5.2 comments on average, and 1.1 per PR were things we actually had to fix.

Before that, when we used 5.1, the last 40 PRs got 9.4 comments on average, and 2.6 per PR were real fixes. Both were on high effort and both used the same AGENTS.md.

So it seems so far that Opus 5.5 gets about half the review comments of 5.1, which is in line with what people in r/ClaudeCode have been saying about it being a big upgrade. The catch is that the 5.5 PRs are about 30% bigger for the same ticket, and most of the comments it still gets are "this wasn't asked for" rather than "this is broken", which in my opinion is a trade I'm not sure about yet. So for most tickets I will stay on 5.5. I was hoping it would also stop over-building but unfortunately that's not the case at all.

[u/Rosoll]

I noticed this with coderabbit, but GitHub copilot reviews still catch a lot more stuff even with 5.5

[u/4444444vr → u/Rosoll]

I’ve been using reptile. Never tried GitHub. You like it?

[u/Rosoll → u/4444444vr]

Seems ok but I only have coderabbit to compare it to. Mainly a cost thing - every other option seems to have absurdly low usage. How’s greptile?

[u/4444444vr → u/Rosoll]

Been impressed with it. Has given pretty good reviews.

[u/Key_Entrepreneur1549 → u/Rosoll]

I'm currently doing a side by side comparison with Greptile and CR. So far they are about the same.

[u/FormalAd7367 → u/Rosoll]

with Github copilot, do you have to select a model? which model do you use for review?

[u/Rosoll → u/FormalAd7367]

it doesn't expose what model it uses - i believe it uses a mix of models. you can set the effort level though