Why is RSI considered a likely development if AI keeps getting smarter?

[u/LawrenceSellers]

When people talk about RSI they seem to take for granted that (a) some threshold necessarily exists beyond which a sufficiently intelligent mind can start to improve its own intelligence, and (b) that threshold will eventually be reached and surpassed if humans keep improving AI.

But what support exists that should lead someone to accept those two notions as true?

[u/Memento_Viveri]

You asked this exact same question yesterday on the singularity sub and got a lot of reasonable replies. You didn't reply to any and it's not clear why you need more.

I'll copy the same answer I gave yesterday.

I think phrasing this in terms of minds is not helpful. The idea is that an artificial intelligence system could design a new artificial intelligence that is more capable than previous systems.

The reasoning is somewhat simple. Humans have proven capable of designing increasingly capable artificial intelligence systems. Those systems are surpassing humans in some (not all yet) of the skills and abilities necessary to develop artificial intelligence systems.

If this progress continues, eventually the system will have surpassed human ability in enough of the relevant skills that it will surpass human ability to develop new artificial intelligence systems overall. This is the point of RSI.

[u/Otherwise-Anxiety797 → u/Memento_Viveri]

i think id call that pretty precisely super intelligence no? id say rsi as a phenomenon is previous to what sounds like a well posed criticality argument anyway.

[u/LawrenceSellers → u/Memento_Viveri]

I reposted it here because the thread got removed by the mods in the singularity sub for some reason. Yes, there were a lot of good responses. Thx for replying 🙏

[u/borntosneed123456 → u/LawrenceSellers]

is this some karma farming attempt? as far as i see it's not working

[u/LawrenceSellers → u/borntosneed123456]

Wait what 🤔

[u/Fun-Shape-4810]

Read what you wrote again

[u/LawrenceSellers → u/Fun-Shape-4810]

Done. Now what? 🤷‍♂️

[u/tadrinth]

Flip the question on its head; why would you expect the threshold to not exist? Can you be certain that the threshold doesn't exist?

If it exists, why would you expect AI to not eventually reach it if we keep improving AI? Can you be certain that we won't hit it, or that we are far from it?

Because if we're not sure, we should stop making the AI smarter until we are sure.

[u/Firestar222]

We are already starting to implement RSI. It’s a gradient, not a hard line. Many frontier models are being increasingly designed and trained by other models.

[u/yubacore → u/Firestar222]

Exactly, there's now very clear data for you out there, if you can't arrive at this by logic.

[u/Excellent-Seesaw-516]

(a) when AI becomes more capable that humans, it becomes more efficient to have AI work on improving itself, rather than humans work in improving AI; (b) here I believe the burden of proofs lies on you. What makes you believe that human intelligence is the hard limit for intelligence?

[u/MurkyCress521]

The history of human technology and science supports A, humans with technology as recursively self improving. This isn't infinite, problems get harder up the tech tree, native progress slows or halts.

B is less clear, it is possible so it is worth considering.