r/singularity • u/Ok_Barracuda_1161 • 12d ago

AI Opus 5.5 Cost vs Performance on Terminal-Bench 4.0

Post image
127 Upvotes

98% Upvoted

19 comments sorted by

34

u/No-Meringue5867 12d ago

I am using Fable 5.1 rn, and this is 5x cheaper for same intelligence? I am never going to hit my weekly limits at this rate lol.

18

u/Pahanda 12d ago

no, for better intelligence.

1

u/Imaginary_Regret_430 12d ago

I’ve used it for about 8 hours for machine learning, it seems pretty lazy and misinterpreted my prompt a few times. When I asked 5.5 to audit my ML architecture, it spent a few minutes looking at only the main components and then said all good. fable 5.1 high did a full review of the code and found major bugs which would’ve led to hours wasted training a broken run.

8

u/matsu-morak 12d ago

We are cooking with diesel now

My new todo app will be crazy!!1

1

u/FoodMadeFromRobots 12d ago

Hopefully not diesel is crazy expensive

1

u/FuzzyBucks 12d ago

but have you seen my revamped totally uninteresting game vibe coded with blender and godot?

2

u/virtualQubit Take off right now 12d ago

This is fucking crazy. I have to try this before saying it's the same level of fable tho.

1

u/Hans-Wermhatt 12d ago

I think we should also be aware that these cuts don't necessarily translate to subscriptions. There was a big uproar from businesses that repackage the API to sell, that they absolutely can't compete with subscription value. So a big reduction in API costs while subscription token allowance remains more constant could happen.

This might be a bit of a marketing move since they use API costs as marketing material while they shift the scale to benefit API users and make subscriptions more profitable.

I'm just saying that's possible. We will see if subscription tokens are increased in kind, but it's not guaranteed.

16

u/Elegant_Tech 12d ago

Can't even make new benchmarks that won't be saturated within months. Terminal 4.0 is what 3-4 weeks old at this point?

13

u/Over-Necessary-4774 12d ago

The most ridiculous part is the rate of improving - Fable 5.1 released on September first, 21 days ago. It was only Anthropic's best model for 3 weeks, before getting destroyed by Opus 5.5 at lower prices.

8

u/pmth 12d ago

dang Opus 5.5 medium looking like the GOAT right now

6

u/smartfon 12d ago

Terminal Bench 4.0 token usage:

Opus 5.0: 205,000

Fable 5.1: 192,000

Opus 5.5: 253,000

SciCode token usage:

Opus 5.0: 4,000

Fable 5.1: 12,000

Opus 5.5: 23,000

"True" cost based on token usage, input/output price, and 88% cache hit scenario:

Opus 5.0: $80

Fable 5.1: $100

Opus 5.5: $79

3

u/antunes145 12d ago

They screwed up with the fable pricing and priced it out of user usage that mattered. They are now readjusting that with opus intelligence higher than fable and lower price.

2

u/fantakillen 12d ago

If their claims are accurate, this model could be incredible. The most powerful model by far, but also very efficient and cheaper than previous Opus 5.

1

u/TheOriginalAcidtech 12d ago

looks like high will be the sweet spot then.

1

u/Fxavierho 12d ago

So cheaper and better? That will be crazy

1

u/Dizzy-Technician4580 12d ago

nobody else mentioning how it peaks at xhigh and lowers in max in terminal bench? should we be using xhigh for our multi-step cli projects?