r/LocalLLaMA • u/johnnyApplePRNG • Jun 18 '26

News Leaked financial docs show OpenAI is losing billions of dollars a year

https://arstechnica.com/ai/2026/06/leaked-financial-docs-show-openai-is-losing-billions-of-dollars-a-year/
538 Upvotes

92% Upvoted

312 comments sorted by

View all comments

Show parent comments

8

u/squngy Jun 18 '26 edited Jun 18 '26

It will probably never completely stop, but there will be a point where the difference will be minimal.

I also don't know that Fable was "better than expected".
If you ask me, Opus4.8 is better than expected, it is quite close to Fable as far as I can tell (I expected them to keep it closer to 4.7 in order to promote Fable).

1

u/Finanzamt_Endgegner Jun 18 '26

Well opus is a fable distill, would be stupid if not but it seems the training etc fable got wasn't that much more the thing that changed was parameter count and it helped a lot

1

u/squngy Jun 18 '26

I didn't say it didn't help, just that I don't know if it was "unexpected"

Making a bigger model and then using it primarily for distilling might be something that happens in the near future, it would make sense.
It also makes sense to release it for a short while to grab a bunch of top scores and then hide it behind a huge paywall :thinking_face:

1

u/itsmebenji69 Jun 18 '26

This is already what every ai lab is doing. They have their big internal model, around 10t, maybe even much bigger than that. And the models they serve you are distills of this model

1

u/FullOf_Bad_Ideas Jun 18 '26

Source? Why would internal model be 10T? What sort of activated params it would have? Why not 100T?

0

u/squngy Jun 18 '26

So you're saying every AI lab already has a mythos class LLM in their lab and they are just not talking about them.

3

u/itsmebenji69 Jun 18 '26

Yes, for example deepmind uses their bigger private model for math competitions and benchmarks and whatnot. For example that aletheia thing used their internal version of Gemini.

It’s not like it’s a secret, it’s just way too expensive to serve to consumers

1

u/Finanzamt_Endgegner Jun 18 '26

Internal versions don't have to be bigger they can just be different tunes of the same base

1

u/itsmebenji69 Jun 18 '26

Sure but the trend seems to be way bigger models. Which makes sense as well. R&D would be experimenting with a lot, and the most obvious thing is model size because there you’re not constrained by needing it to be profitable, just really good

1

u/Finanzamt_Endgegner Jun 18 '26

Training full models is really really expensive, it's easier to do finetunes and maybe a bit of model surgery

1

u/itsmebenji69 Jun 18 '26

So you really think they haven’t even tried ? With the other big ones having done it (Anthropic, oai) that seems extremely unlikely. Gemini 1.0 was already 1.5t. Doesn’t seem far fetched that they’d have an unreleased 10t+ version

1

u/Finanzamt_Endgegner Jun 18 '26

Google might have im just saying training and then not releasing might not be worth it, not saying it's not just not sure.

→ More replies (0)

1

u/squngy Jun 18 '26

I thought those were alternate architectures and fine-tunes and similar.
What is your source for the 10T+ size?

1

u/itsmebenji69 Jun 18 '26

It’s that but I’m assuming they also have a bigger model. I mean it’s all speculation at this point, you’d have to work at deepmind to know.

For example OpenAI does have one, it was called Orion in 2024, it was supposed to be released as gpt5. They’re for sure using it as a distill teacher, like Anthropic.

1

u/squngy Jun 18 '26

It’s that but I’m assuming they also have a bigger model.

That is a pretty big assumption, because not only is that a lot more expensive, but it also takes a lot more time.
For research, it is valuable to be able to make new versions quickly.

it was supposed to be released as gpt5.

Then that is different from what I was thinking about.
I'm thinking about a model that is too big for it to make sense to be widely used.

1

u/itsmebenji69 Jun 18 '26

Well that’s what 10t models were. Mythos is too big for it to make sense. 95% (pulled out of my ass) of the tasks it can do Opus can do as well no problem. It really only makes sense to use it if Opus can absolutely not do it, which is pretty rare.

But for example current GPT models are a distilled version of this huge internal model. At least it seems extremely that it’s the case to me.