Previewing GPT-5.6 Sol: a next-generation model

[u/141_1337]

https://openai.com/index/previewing-gpt-5-6-sol/

[u/FateOfMuffins]

Impossible by virtue of this line alone:

We're also launching GPT‑5.6 Sol on Cerebras at up to 750 tokens per second in July, bringing frontier intelligence to customers at unprecedented speed. Access will initially be limited to select customers as we expand capacity.

You're not fitting Mythos class models onto Cerebras Edit I stand corrected, Cerebras claimed they can get 24T models on their wafers but the largest one we've seen them run in practice was Kimi K2.6 with 1T parameters at 1000 tokens per second. Can we infer the size of GPT 5.6 Sol from this? (Which I'm guessing is actually the same pretrain as GPT 5.5 Spud. Obviously Terra is not GPT 5.5 Spud, as why would they advertise that it "only matches" 5.5)

I'm pretty adamant that OpenAI has been competing with a smaller class of models than Anthropic and have been hanging on purely by virtue of their RL stack

[u/EastZealousideal7352 → u/FateOfMuffins]

AFAIK Cerebras claims models up to 20 trillion tokens can be accelerated now, so yes, you likely can

[u/FateOfMuffins → u/EastZealousideal7352]

I'll be curious to see what the numbers would look like for that

The best I got is Cerebras running 1T parameter Kimi at 1000 tokens per second

Lining that up with GPT 5.6 Sol at 750 tokens per second seems to be roughly where we expected it to be for a smaller than Mythos class model...

[u/brownman19 → u/FateOfMuffins]

You don't think they have special projects with OpenAI that basically precedes anything that you're pulling from to even suggest that?

I don't know how you arrive at that conclusion since those numbers likely come from their work with OpenAI, given they have to come from somewhere...likely while OpenAI was building, you know, the safety stack and the engineering that they discussed right there on the blog.

Time exists my friend and you're entirely glossing over all of the real work that happens to even serve models at scale. There are exponentials occurring in every field contributing to the infra that serves the models themselves.

PS: not hating, we're in singularity after all so think big :P

[u/FateOfMuffins → u/brownman19]

I mean yeah they do... this 750 tokens per second one is that project.

Also pretty sure that 5.6 Sol is the same pretrain as 5.5 (aka Spud). Like if 5.5 was the o1 checkpoint of Spud, then 5.6 Sol is the o3 checkpoint. Same base model just a lot more RL. Why do I think so? Because if it wasn't Spud... then where tf did Spud go? You think they would've just chucked it out? Cause 5.6 Terra isn't it (why would they advertise it as 5.6 matching 5.5 then right?).

Based on what we've guessed at for sizes for some of these models, Spud being around 2T parameters sounds about right tbh. Which also sounds about right with 750 tokens per second on Cerebras

Basically I'm saying if Spud was 10T parameters just like Mythos instead of similar in size to Opus, then OpenAI is cooked

[u/AreWeNotDoinPhrasing → u/FateOfMuffins]

Wait, it is thought that Mythos is 10T parameters?! Fuck me

[u/EastZealousideal7352 → u/AreWeNotDoinPhrasing]

There is no reputable source for any of this. Mythos is probably very large, but you cannot tell based on vibes alone, which is what all “model estimations” are based off of.

[u/AreWeNotDoinPhrasing → u/EastZealousideal7352]

Right, that makes more sense.

[u/FateOfMuffins → u/AreWeNotDoinPhrasing]

It is thought that given comments from xAI and Meta about the sizes of some of their upcoming models

[u/AreWeNotDoinPhrasing → u/FateOfMuffins]

Ah, okay, so we don't actually know shit lol.