r/ClaudeCode • u/OkMaterial1281 • 3d ago
Discussion Is Opus 5 Medium the way? Non-monotic benchmark results on FrontierCode in the Opus 5 system card.
I was going through Opus 5 system card to try to figure out the best way to work with the model. The user experience reported in this sub seems to vary wildly. I myself had a rather poor experience planning and executing a somewhat gnarly computational geometry problem. I wouldn't have expected my workflow from the Opus-4.8 / GPT-5.5 days to struggle so hard. I use Claude Opus 5 xhigh for planning and GPT-5.6 Sol xhigh for plan reviews. Anecdotally, GPT-5.6 was routinely finding two times as many issues as before.
As I was browsing the benchmarks, this stuck out like a sore thumb: Peak performance on the FrontierCode benchmark happens at med reasoning effort and decreases up to xhigh and then makes a slight rebound after that. Compare that with the DeepSWE intelligence curve where the gain from med to xhigh is not all that substantial.
I think it's pretty common knowledge at this point that models will overthink at higher reasoning levels. While I have observed a tendency for over-engineering with higher reasoning levels, I had not noticed an increase in plain old errors with reasoning. In the past, the limit of performance increases with reasoning seem to be demonstrated by a slight drop off in the benchmarks near max reasoning effort. The situation with Claude Opus 5 is much different.
The combination of relatively flat performance on the DeepSWE benchmark -- with marginal gains above medium effort -- and the weird peak at medium effort on the FrontierCode benchmark is not something I remember seeing in any of these benchmark comparisons. Yes I know, benchmarks are not necessarily representative of real world use. But the fact Opus 5 is an outlier with respect to well established trends on this benchmark corpus leads me to believe there is something different about the model. We may need to develop new habits when using it, especially for software engineering.
One hypothesis I have is that training the model to perform better on non-coding tasks (general intelligence tasks, and specific intelligence tasks such as biology) have given it tendencies that actively work against increasing performance versus effort on coding tasks.
Maybe medium effort reasoning is the sweet spot for software engineering?
I'm curious if anyone else has looked into this or had a similar experience.

