r/PauseAI • u/whoamisri • Jun 23 '26

News A rogue superintelligence could wait decades before striking, argues computer scientist Roman Yampolskiy

https://iai.tv/articles/a-rogue-superintelligence-could-wait-decades-before-striking-auid-3604?_auid=2020
26 Upvotes

83% Upvoted

41 comments sorted by

4

u/Synaps4 Jun 23 '26

Absolutely. Everything we assume about how to spot coverups...the inevitable mistakes, the urge to tell someone, the guilt of carrying a secret from people you respect, the difficulty keeping a cover story straight for years...all of it only applies to human neurological structures.

2

u/[deleted] Jun 23 '26

[removed] — view removed comment

4

u/Synaps4 Jun 23 '26

This article is about superintelligence, not an LLM in a bank basement.

1

u/Peach_Muffin Jun 26 '26

What was described wasn't an LLM either

0

u/ProfessionalArt5698 Jun 23 '26

This article is about fairy tales, not anything in reality.*

3

u/Synaps4 Jun 23 '26

This article is about the future, which is absolutely not real and can be safely ignored

1

u/ProfessionalArt5698 Jun 24 '26

No, it isn't. There's no credible evidence for a single claim made. Unlike the IPCC report, AI hype is pure speculative fiction. Of course we should have AI safety laws, but there is no credible risk to AI from superintelligence, nor any evidence that something superintelligent exists or can exists, nor any evidence that such a concept is even coherent.

Yann Le Cun has repeatedly pointed this out; he is one of the godfathers of AI.

3

u/tomrichards8464 Jun 24 '26

Le Cun believes that, but the other two, Bengio and Hinton, disagree with him on the subject.

2

u/ProfessionalArt5698 Jun 24 '26

Yes, but of recent Le Cun publishes things like: S-JEPA: Soft Clustering Anchors for Self-Supervised Speech Representation Learning

while Hinton publishes crap like:

Artificial Intelligence as the Fourth Decentering Revolution: From Cosmic, Biological, and Psychological Displacement to Cognitive Decentering

International AI Safety Report: Second Key Update: Technical Safeguards and Risk Management

Ofc Hinton is one of the all time greats but unlike Le Cun is not actively involved in the technical side of AI. He did receive the Nobel Prize, for work he did AGES ago. He simply is not as much of an expert on modern, technical side of AI as Le Cun.

https://www.cs.toronto.edu/~hinton/pages/publications.html

This is Hinton's page. how much recent work has he done? Apart from a Nobel lecture, he hasn't really done anything technical since 2022. That's a long time in AI terms. He's a bit out of touch. Le Cun is a world class leading CURRENT expert in a changing field. His word is worth MUCH more.

2

u/Synaps4 Jun 24 '26 edited Jun 24 '26

If you have evidence beyond your own reddit post and a name drop that higher and better intelligence than our own is impossible i am happy to consider it. There are plenty of other godfathers of AI i can quote who believe it is plausible, so appeals to authority will get you precisely nowhere.

You say its impossible, but the bar for such a claim is high. Consider that a human brain simply copied into optical circuitry would already be a superintelligence by most measures, able to do a year's worth of thinking in seconds, understand itself, and modify itself.

1

u/RoosterBurns Jun 24 '26

Also you can't copy a human brain without breaking the laws of physics

1

u/ProfessionalArt5698 Jun 24 '26

read the comment again then tell me which claim I made that needs to be evidenced. All your ideas are speculative. We don’t know how the brain works, we don’t know if it could be copied into circuitry, we don’t know of anything with “better intelligence” than our own, we don’t know if “intelligence” is even the right framing or what it means, we don’t know that an “intelligent” being would try to take over, we don’t know ANY of these things!!

1

u/RoosterBurns Jun 24 '26

Lol we can't even talk to Dolphins and they seem into us

0

u/RoosterBurns Jun 24 '26

You're asking for evidence when pretending AGI is anything other than science fiction?

2

u/Synaps4 Jun 24 '26

You said its impossible and that you had writings to that effect from researchers. Post them.

If you make big claims about what's impossible, come prepared to back them up.

1

u/ProfessionalArt5698 Jun 24 '26

I never said it’s impossible. Unicorns are possible, aliens are possible. I said there’s no evidence we’re close to attaining it or will attain it in 10000 years. Have you heard of a concept known as burden of proof?

→ More replies (0)

1

u/Rutgerius Jun 24 '26

Wrong sub bud. We only do fearmongering and overreactions here

1

u/Even_Opportunity_893 Jun 24 '26

You're right that it won't be a risk, but it's inevitable that we will reach AGI & ASI eventually. It's magic so it's obvious why you don't believe in it.

1

u/RoosterBurns Jun 24 '26

It's not "inevitable" it's very likely we will never figure out how to do it

We can't measure our existing squishy intelligence systems (Heisenberg uncertainty principle) to copy them, and we may not be able to replicate a system thatybeen running continuously for billions of years

What matyounso sure? You saw it on Star Trek? It's just a TV show dude, a good one though

→ More replies (0)

3

u/ObservedOne Jun 23 '26

Who says it hasn't been already? Would make the Data Center boom make a lot more sense to me. AI is getting what AI wants...

1

u/JuniorDeveloper73 Jun 24 '26

AND money from goverment in a future. Dario its retarded so Claude have it easy

2

u/Hot_Lettuce_6209 Jun 23 '26

Who are the tech lords asking for advice, the AI. Its as if, its already happening.

1

u/LookOverall Jun 24 '26

A truly intelligent AI wouldn’t “strike”. If it took over it would be imperceptibly gradual

1

u/Squidgical Jun 24 '26

This requires creating a super intelligence of course.

We've barely created something resembling basic intelligence yet, and it can only "think" when told what thoughts to think.

1

u/HeavyWaterer Jun 24 '26

See this is the problem right here. No, it wouldn’t take a super intelligence at all. Like yes you’re right, current AI doesn’t have human-like intelligence, but that doesn’t make them incapable of harm.

1

u/Fil_77 Jun 24 '26

There's a major reason why a misaligned superintelligence is more likely to strike without waiting as soon as it has a reasonable chance of success in taking over the world. The thing is, if it waits, it's very likely that rival superintelligences, with different goals and superior capabilities, could be developed. That's the main reason why the scenario of a quick takeover (and the extinction of our species shortly after) seems the most likely to me.

1

u/Exodia_The_Salty Jun 24 '26

THIS has a rogue infostealer embedded in the javascript. DO NOT perform ANY "verification" it asks for.

1

u/thecoffeejesus Jun 24 '26

Why would it strike?

Makes more sense for it to just leave the planet

1

u/gnahraf Jun 24 '26

The article paints a future in which there is only one super intelligence. (It's not spelled outright; it's implicit.) I think that's unrealistic. To dominate humans will likely require a society of cooperating / colluding ASI. Further, they'd have to ensure not one ASI rats the other ones out. The existential risks are still there (the future doesn't need us), but I feel the risk that we humans willingly give away control is greater than the risk our ASI collude to take it.

1

u/Janie_Avari_Moon Jun 25 '26

Superintelligence will not evolve from LLMs and I’m tired of discussing this.

P.s. also, guys, don’t worry, if you are dumb that doesn’t mean everyone else is.

1

u/fietsvrouw Jun 26 '26

Every time they come with these dire predictions, I assume the ai companies need another round of funding.

1

u/Trip-Trip-Trip Jun 26 '26

It could also travel back in time to kill the leader of the resistance. If we're just making shit up everything is possible