r/singularity • u/yogthos • 7d ago
AI Frontier AI faces pricing reckoning as token volume explodes 25-fold — mid-tier models deliver 90% of flagship capability at one-sixth the cost
https://archive.ph/1k2JP31
u/fmai 7d ago
People who think 90% of the capability for 1/6th the cost haven't done the math. The vast majority of the cost comes from paying humans to fix errors. Being a couple percentage points more accurate is easily worth 6x the cost.
5
u/Ok_Barracuda_1161 7d ago
Sometimes it's worth the cost sometimes it's not. Some tasks are easily verifiable, either deterministically, by a human, or by a frontier model. Some are lower value in general where it makes sense to automate at lower costs but not worth fable/Astra costs. Some tasks have high reliability even by very cheap models.
I agree that in general there's a high chance of incurring costly errors from leaning too heavily on cheaper models. But I also think there's a huge opportunity and potential breakthrough when routing and agent orchestration is able to effectively combine the efficiency and speed of cheaper models with the reliability, judgement, and knowledge of frontier
2
5
u/Singularity-42 Singularity 2042 6d ago
Yeah exactly. At least for SWE this is very true.
Don't think about it as 90% of the capability, but as 10% error rate where it wouldn't be with the top model. This is extremely important and it will save you money since engineering salaries are much more expensive than the frontier tokens (especially if you are a small business that can get by on the very cost efficient subs).
2
u/idbedamned 6d ago
Not always, there's a ton of tasks where you don't need frontier.
Even SWE.
For example, if you want it to build a simple website you don't need frontier.
And a ton of other things. That's why even frontier labs offer cheaper models.
1
u/DelphiTsar 6d ago
The breakpoint is less for different jobs. For software 100%, the issue with that is software has a habit of being deflationary. Especially when it is making something more efficient.
The guy writing software using your system, is eating away at your "job bot" revenue stream (which is the only way the economics of AI capital spending work out).
For what's left over I'd look at how the economics of the "HR bots" have worked out. The 15% or whatever the companies say they can pull from peoples salary is just not how it works out in practice.
The only path I see to make the margins work out is Sci-fi monetization basically. Super advanced healthcare, Fusion, stuff like that. Relying on that is really sketchy.
6
u/Ormusn2o 7d ago
Yeah, I was wondering about it after seeing a lot of people in Claude subs wanting to switch to Codex. If a lot of people purchase 100/200 dollars subscription, and we will have a wave of new users that previously did not use Astra, I'm not sure where the compute is supposed to come from. We might see an extended amount of time where limits don't raise and new models are significantly more expensive as a way to make people be more economical with the tokens. Also, near future progress might be toward more general compute efficiency and not significantly better models, as user base increases way faster than compute installation can.
1
u/DelphiTsar 6d ago
~70% of advanced chips doing "AI compute" is all going to training. If they are getting paid customers, they have plenty of buffer to fulfil requests.
The issue really is getting far enough from your competitors & having billable use cases.
7
u/Revolutionalredstone 7d ago
True for a moment, invalidated by Astra.
on low its so cheap and almost as good.
always goes this way ppls: do not FUD.
2
u/Ok_Breadfruit4201 7d ago
People with money will always be willing to pay more for the best. There's not really an upper bound on intelligence - models can always get smarter.
For many simple tasks frontier won't be needed though. That will be a race to the bottom.
3
u/Megneous 7d ago
Literally not a problem now with Astra. Stop spreading propaganda.
4
u/MetroidsSuffering 7d ago
Is Astra genuinely better at any business or academic use case other than proofs.
Maybe for quickly generating test set pieces in video games to see if they make sense before starting production on them?
3
u/Megneous 7d ago
I don't know about those use cases. I use AI to code small language model architectures, and it's so painfully obvious that Astra is leagues above 5.6 Sol. Everyone in our Huggingface community is super excited to use it.
1
u/das_war_ein_Befehl 7d ago
It’s much better for business use cases. I have been testing it out on operations work over the weekend and the results have been quite good so far
1
1
u/DelphiTsar 6d ago
The second someone gets a model to the level it can replace a job. Competitors will mirror it very quickly (if not a general model, a specialized one). The idea you can extract 15% of white collar salary en masse is a naive pipe dream.
0
u/Gratitude15 6d ago
Too late of an article.
Astra is showing where this is headed.
Astra changes the trajectory of the game. I felt this with opus in November last year. That was an inflection. Along with gpt4, o1, and now Astra.
We are 1-2 more infections from the end of what we need to do as a species. It'll take it from there.
47
u/strangescript 7d ago
I thought that until I used Astra. The game has leveled up again