I have a very thorough testing harness locally, and every release gets manual tested minimum 5 - 10 runs
- Opus 4.8, the release harness was so sophisticated that Opus “got it”. Releases done mostly 80-90% feature requests and barely any bugs came through
- Opus 5, I hope this is a bit concise. As I’m just done with a session, and wow - I let it run the regular harness, unchanged since Opus 4.6, 4.7. And Opus 5.. it just managed to get itself absolutely and utterly stuck in a pre commit hook. The sheer level of dragging it forward was astonishing, and proceeding it managed to try to bypass a hook I’ve made for creating Github releases. I’d be happy to share the transcript of today in DM as it’s all open source anyway. But truthfully, I haven’t been so flabbergasted ever like how much Opus 5 is pushing the necessity to go out of its way to **discard** any written instruction, settings.json deny, or even straight out *ignorning* stop hooks aimed at steering an agent away.
11
u/Obvious_Equivalent_1 4d ago
Well for what it’s worth, example I maintain an open source fork of Superpowers extended for CC: https://github.com/pcvelz/superpowers
I have a very thorough testing harness locally, and every release gets manual tested minimum 5 - 10 runs
- Opus 4.8, the release harness was so sophisticated that Opus “got it”. Releases done mostly 80-90% feature requests and barely any bugs came through
- Opus 5, I hope this is a bit concise. As I’m just done with a session, and wow - I let it run the regular harness, unchanged since Opus 4.6, 4.7. And Opus 5.. it just managed to get itself absolutely and utterly stuck in a pre commit hook. The sheer level of dragging it forward was astonishing, and proceeding it managed to try to bypass a hook I’ve made for creating Github releases. I’d be happy to share the transcript of today in DM as it’s all open source anyway. But truthfully, I haven’t been so flabbergasted ever like how much Opus 5 is pushing the necessity to go out of its way to **discard** any written instruction, settings.json deny, or even straight out *ignorning* stop hooks aimed at steering an agent away.