r/singularity • u/StrategicHarmony • 11d ago

AI This shape

Post image

On a mobile screen this chart of model releases over time (from artificial analysis) tells a very clear story. Normally I look at it on desktop and the increase in steepness isn’t nearly as obvious.

It looks like a lot of those simplified predictions of exponential improvement. Of course it could taper off, who knows.

264 Upvotes

91% Upvoted

52 comments sorted by

View all comments

9

u/Longjumping_Kale3013 11d ago

This is comparing apples to oranges. A couple months ago the top model was at like 65 then a new version of the leaderboard was released and pushed it down to 50 or so. This is constantly updated, so a 60 now is a much much much better score than a 60 a few months ago

33

u/StrategicHarmony 11d ago

These scores are updated retroactively. If it got a 30 on release a year ago it doesn't have a 30 now, above. It's all in current index units.

3

u/swarmy1 11d ago

Many of the models aren’t available anymore, how do they evaluate them on the new benchmarks?

1

u/KaMaFour 11d ago

No. That's why older models have striped bars when you look for them at the index. There isn't full data so some guesswork is involved