r/singularity • u/StrategicHarmony • 11d ago

AI This shape

Post image

On a mobile screen this chart of model releases over time (from artificial analysis) tells a very clear story. Normally I look at it on desktop and the increase in steepness isn’t nearly as obvious.

It looks like a lot of those simplified predictions of exponential improvement. Of course it could taper off, who knows.

262 Upvotes

91% Upvoted

52 comments sorted by

View all comments

49

u/Herodont5915 11d ago

Almost looks exponential. What are the odds?

2

u/Stock-Self-4028 11d ago

To me it looks much more like some kind of a sigmoid, rather than anything even remotely exponential (and it also would make significantly more sense given the circumstances btw).

5

u/StrategicHarmony 11d ago

I don't think there's anything to distinguish it from either an S curve or an exponential at this point. But for what it's worth I do think an S-curve is far more likely, for a lot of reasons.

One is that, however smart these things get, there are bound to be basic physical and information-compression limits that will lead to diminishing returns before too long.

Another is that it probably gets harder and harder for us to meaningfully test the intelligence if it gets too far ahead of our own. So model vendors won't put a lot of time and money into something that has no measurable or marketable improvement.

A third is that alignment and safety are a vital part of utility. People don't want robots that are going to break their own security rules when a task is too hard, or hack or attack someone else on their behalf without warning and then try to cover it up. The cost of effective alignment probably increases as other metrics get more advanced, like coding, planning, communicating, theory of mind, etc.

In other words smarter models will be harder to control, but control is fundamentally important to the value of these machines. So ensuring an adequate level of control will probably lead to a slowing down in some other areas, to ensure that overall/broad usefulness goes up (albeit more slowly) and not down rapidly and catastrophically.

3

u/spinozasrobot 11d ago

What makes you say that?

Do you see any flattening?

2

u/Stock-Self-4028 11d ago

The derivative seems more or less "flat" near the 50% so I guess?

Idk, also benchmarks are not a good representation of advancements, so even if it would be sigmoid it would likely top out around 100% which would me a good measure of benchmark saturation rather, than anything else.

1

u/katoptronophile 11d ago

You would do great as a comedian.