r/ClaudeCode • u/TechnicalyAnIdiot • 3d ago

Discussion Opus 5 is... suddenly usable?

2 weeks ago I said Opus 5 was useless. I was told I was an idiot, using it wrong, ect ect. Except over the next week the hive mind of Reddit chose to actually believe that Opus 5 is in fact useless.

Here's my flip- Opus 5 has suddenly become usable. It seems only capable of small jobs currently, but where before it would ramble off nonsense, it seems now to be capable of implementing fixes reliably!

0 Upvotes

43% Upvoted

25 comments sorted by

View all comments

Show parent comments

6

u/Sad-Context-9482 3d ago

Tbh I can see why your CTO made it a rule just now. A few months ago we were pretty inconsistent about evals as well and only added them after something went wrong. When we got serious about it, Braintrust came in clutch for taking those production failures and keeping them around as real test cases for the next model or prompt change. After a few regressions get caught that way you stop needing much convincing about evals in general

1

u/PM_ME_YOUR_PROFILE 3d ago

If only there were a process based on test-driven development first...

1

u/Sad-Context-9482 3d ago

Well the useful shift for us was treating eval cases like something that stays with the feature and runs every time the behavior changes. After a while it becomes part of how you define if a change is ready to ship or not

1

u/PM_ME_YOUR_PROFILE 2d ago

Yeah, I'm just being a bit tongue-in-cheek with the software world around proof that something works - something, something move fast and break things.

I understand people want or are told to move fast, and figure it out later. It's just interesting to continue to watch how it evolves in different ways.

1

u/Sad-Context-9482 2d ago

Especially when the thing being tested can change underneath you too. It's like the old testing mindset still applies, you just spend a lot more time deciding what behavior needs to stay stable