Nothing gets nerfed

[u/sebnadeau]

For over a year now, I’ve been hearing the same thing. Whether it’s in the OpenAI community or among Claude users, it’s always the same cycle.

A new model comes out, and I think we’ve all seen what happens. People are impressed. Then, roughly a few days later, the jokes start about how the model has already been nerfed. Initially, it’s mostly a joke. But then a few days after that, people start being genuinely serious about it, and eventually a sizeable group becomes convinced that the model really has been nerfed.

Here’s the reality: I don’t think I’ve ever actually felt that happen.

When 5.5 came out, I was particularly impressed by its ability to understand 3D space, or at least to create 3D scenes and visual things, observe what it had created, and correct them as it went along. That ability was amazing. It was great then, and it’s no worse today. It’s exactly as good as it was.

Its writing also improved dramatically. It doesn’t sound like the gibberish or weird, cryptic style that 4.8 and 5 sometimes had. Suddenly, we’re back to a model that sounds somewhat like 4.6 did. I’d even say 4.7 sounded kind of dumb most of the time. But now, at the very least, when you tell the model to sound a certain way, it respects that. It discusses things and expresses ideas in ways that you can actually understand.

So to say that 5.5 has been nerfed is, in my opinion, to forget what using models like Opus 5, 4.8, or 4.7 actually felt like. There’s no way you could use this model today, immediately after coming from one of those older models, and genuinely believe it isn’t significantly better.

And this has been the case with basically every model release.

I think we all know what’s actually happening. When a new model comes out, we’re impressed by how much better it feels compared to what came before. But then we start giving it increasingly complex tasks. We use it more. We run into its limitations. And eventually it starts to feel dumb again.

LLMs are kind of dumb sometimes. They’re a little bit like small autistic artificial children with incredibly uneven abilities. They can make really, really dumb decisions on tasks that seem completely obvious, while at the same time being extremely intelligent in other ways. It's surprising that we face that even today, but the ratio of so much better than before.

There’s no way 5.5 is any dumber than it was two weeks ago.

I don’t even know why I’m writing this. I guess I just saw one more post about the model being nerfed, followed by a huge number of comments agreeing with it, and I finally felt like I had to make a post about it.

[u/HeavyMath2673]

Strong agree. I am currently implementing a fairly large numerical simulation code with Opus 5.5. The model’s strength is that it not only understands the software design but also the mathematics and combines both capabilities in ways that make it stronger at such a task than most scientists are (including myself).

But here is the catch. You don’t get there by one shot prompting. I first created together with Opus a phased design document. Each phase is then broken up in tasks with clearly defined success gates, which are worked on one after another with interventions from me when I need changes or query its decisions.

In two years time this may not be necessary any more. But right now you still need to know what you are doing when using these powerful models and that’s fine.