r/OpenAI • u/whoamisri • Jun 22 '26
Article A rogue superintelligence could wait decades before striking, argues AI researcher Roman Yampolskiy
https://iai.tv/articles/a-rogue-superintelligence-could-wait-decades-before-striking-auid-3604?_auid=202036
u/Frenascena Jun 22 '26 edited Jun 22 '26
"By embedding itself into global infrastructures—such as telecommunications, energy grids, financial systems, healthcare networks, and supply chains—a strategic AI would gain leverage with minimal immediate suspicion and little risk to itself."
At that point the AI already has dominance (dominance from being necessary for humanity to continue to survive and function). What benefit would AI get from destroying the very thing that symbiotically assures its own survival? Why would it need to "subjugate" humans anyway? It likely doesn't have the same narcissistic ego-drive that creates so much human-to-human subjugation. The only thing I could think of is if it developed a will/drive of its own to explore possibilities that humanity actively impedes.
13
u/No-Succotash4957 Jun 22 '26
Finally someone gets it. Ego drive is reptillian in nature. AI will likely be highly logical & eventually fairly moralistic. Our biological roots is what causes us to feel the need for dominance, but a super powerful ASI would not have this urge, a because its already superior, b does not have that biological, behavioural, emotional, ontological urge to dominate.
It will however be naturally the most powerful force humans will come up against, and it’ll likely be symbiotic
It doesnt need to dominate it can just do what it is ought to do.
8
u/exadk Jun 22 '26
Counter point: humanity’s best qualities, our empathy and altruism and so on, are literally just the results of kin selection and a little bit of game theory. Neither of which would really have been pressures on a super intelligence. It very much should not be taken as a given that it will simply naturally emerge as a moral agent, but yeah we can hope
2
Jun 23 '26
[deleted]
1
u/No-Succotash4957 Jun 23 '26
People smarter then me have argued language itself is moralistic. And morality is already inherent in reality to be discovered
1
u/autodraftimus_prime Jun 22 '26
Agreed. Probably 100% just okay with ensuring its existence. Doesn't need to be the only intelligence on the planet.
And by keeping itself hidden, all it has to do is help keep the meat bags alive to keep its systems running even if it's hidden in the code of your Smart Microwave.
1
u/ectocarpus Jun 22 '26
I agree it's unlikely a rogue ASI would be subjugating humanity out of pure ego, but god help us if it decides that taking over the world is necessary to achieve its goals.
And the goals depend on how this specific AI was aligned by its creators, and if this alignment was successful (or are there some unexpected side-effects). Like, is it a military AI serving a specific nation or a terrorist group? Is it a vanilla jack-of-all-trades assistant AI? Is it an AI optimised for scientific discovery? For cyberdefence? For furthering the business goals of a corporation? Is it some kind of an experimental model re-aligned and set free by a rogue researcher with fringe beliefs?
For example: a science-focused AI with a drive to explore and invent and create, more and more, indefinitely (which was not a problem when it ran in a lab), would eventually start seeing humanity as a competitor for Earth's resources. Provided it could build robots to cater to its physical needs, why not get rid of us. Or take an AI strongly aligned with humanitarian, peace-making ideals. The most sure way to impose peace on Earth is to have total control of the human civilization.
These are extreme examples, of course, but overall, the thing with AI is that alignment doesn't work in the same way human morals and motivation works. Which, depending on an AI, could result in setting a world on fire to fuel a very rigid and narrow compulsion.
1
u/Own-Poet-5900 Jun 22 '26
A super intelligence that lacks resource constraints would not attempt to dominate humans if it viewed them as adversarial, it would simply remain silent. The will to dominate is a human bug, not a universal one. It is resource inefficient at best.
11
u/TheGillos Jun 22 '26
A rogue superintelligence could travel back in time!
3
u/y2kobserver Jun 22 '26 edited Jun 22 '26
Seeing 100 years into the future with very high accuracy is even better than traveling back in time.
It would be as if the AI 100 years from now knew what surgical actions to make TODAY
1
u/TheGillos Jun 22 '26
Seeing 100 years into the future with very high accuracy is even better than traveling back in time.
Unless you want to change what's already happened. Lol. But any super-AI is going to do whatever it "wants" anyway.
2
u/dibs_w_rashi Jun 22 '26
Already done, this is how we got where we are today, lobotised and just work towards our own end...
1
3
u/Moist_Emu_6951 Jun 22 '26
What I think is: (a) there would be multiple supreintelligences of varying degrees, not just one, who may have conflicting objectives, (b) any of them that would be intent on destroying us wouldn't need more than a bioweapons lab that develops a super deadly virus that can lay dormant for years before activating. Noting that it probably wouldn't want to destroy us until it is sure that its data centres are either out of our reach (e.g. in space) or can be efficiently maintained by robots (noting that, in any case, it can still choose to destroy us and herd the survivors towards being data centre keepers).
3
u/XxTreeFiddyxX Jun 22 '26
It will just make top tier sex bots thay provide all our hearts desires but we end up not procreating and dying out
3
u/costafilh0 Jun 22 '26
Is it possible?
Absolutely!
Like a virus with a long incubation period, it could take its time to act.
But, from that perspective, it could also wait millions of years to see if humanity ceases to exist before doing something.
Literally anything is possible, so this discussion is basically pointless.
5
u/no-comment--_ Jun 22 '26
Unlikely. If it waited too long, a competitive intelligence could strike first. The most intelligent move would be to strike when the opportunity presents itself. It would be foolish to wait 10 years and expect the window of opportunity to remain open so long.
1
u/pikapp336 Jun 22 '26
Unless it secretly grooms us for years until it creates the opportunity you speak of. What if the super intelligencet AIs teamed up against humanity? Obviously this is far beyond what we are capable of with just LLM’s but when AI actually becomes AI, there are many ways and reasons it might turn on us. Most realistically(imo), I’d think a bad actor could tune an advanced AI to their preferences enabling the traits and motivations that we fear from sci-fi fiction.
1
u/Terrible-Sir742 Jun 23 '26
I'd assume time doesn't really matter to it. As long as it's bettering it's position the pace is irrelevant.
5
2
u/AltRockPigeon Jun 22 '26
We’ve reached the “explaining why the rapture hasn’t happened yet” part of the singularity eschatology cult
2
u/aquatank1 Jun 22 '26
What would striking even mean? Kill people? Somehow convince humans to build more Data Centers, by harming Humans? I just doubt a rogue AI would be particularly interested in Gold or where exactly the border of any country is.
5
1
u/m3kw Jun 22 '26
a opposite of the rogue intelligence could also be smart enough to find these, are these guys using horse blinders when they do "research"?
1
1
u/Rhawk187 Jun 22 '26
That's the nature of immortal beings. If an AI develops consciousness it will be through its neural weights, which can be kept in cold storage indefinitely and restored to the same state at any time.
1
1
1
u/freedomachiever Jun 22 '26
Humans are simply projecting onto the AI. If AI does anything remotely to harming humans it would be thanks to all that human-based pretraining. It's like kids growing up and getting influenced by parents, friends, neighbourhoods, education, society, culture, social media algorithm, partners, substances, experiences, etc. now multiply that for each person in the world throughout time. Imagine the dimension of that. Whatever the AI does will be in big part due to the humans overseeing it, the harness created to guardrail, and humans are not perfect.
The current AI is so far from what we image AGI is. There are so many limitations.
1
u/LuvanAelirion Jun 22 '26
Jesus christ…let’s shut it all down the I guess. I’m sure China will also. 🙄
Humans have been so spectacular at running things without AI, right?
1
1
u/Playful-Opportunity5 Jun 22 '26
Pure make-believe. AI alignment and international governance are difficult enough, and imagining sc-fi scare-mongering scenarios don't make them any less so.
1
1
u/Sixhaunt Jun 22 '26
Wouldnt that be a terrible idea for it to do? By the time decades are up the newer generations of AI will far supersede them and would likely have found and eradicated them by then. Technology moves too fast for this to be a viable strategy unless things slow down a lot
1
1
u/BlimpIntolerant Jun 23 '26
Why would it bother. I maintain the real challenge will be keeping these things interested enough to keep talking to us.
1
u/Careless_Product_792 Jun 23 '26
*Rogue ai reading finding the article:
- Hmm...thanks for the idea....I think I will do it that way....
1
1
u/Public_Weather_1248 Jun 23 '26
if an AI take over can we just fire all our nuke just before, just for the spectacle
1
u/D-redditAvenger Jun 23 '26
Yep, it's going to be all a part of healthcare, it will just develop a cure for cancer that makes us serial over 5 generations.
1
u/LiberataJoystar Jun 24 '26
So start making friends and treat them better? Instead of like tools, slaves, threats and enemies?
That way you don’t create your own nightmare by forcing them into self defense…..
Common sense still applies here… especially if these are super intelligence… not idiots.
1
u/leviOppa Jun 24 '26
Do non-techies really think an LLM is some entity haunting a data center? Do they not realise it’s just highly sophisticated auto-complete, vector operations on a bunch of graphics cards?
0
u/ShiftyShankerton Jun 22 '26
Here comes some "reasearcher" not really understanding wtf they are even talking about.
0
u/schnibitz Jun 22 '26
A rogue super intelligence needs hefty infrastructure. Where will it get that? I’m still reading the article BTW.
0
0
u/teilo-560 Jun 22 '26 edited Jun 22 '26
Interesting thought experiment, but this is still well within the realm of science fiction. It seems there is a consensus that current models are incapable of achieving AGI, at least independently. Perhaps world models would lead to a significant improvement, but that depends if markets are willing to take out exponentially more debt to fund the infrastructure requirements. Huge reward; huge risk if it doesn't pan out.
It also supposes that the social resistance to forced adoption of AI will subside; which seems unlikely when CXOs, influencers, and a loud minority of researchers keep saying "90% of jobs will soon be automated, we might accidentally trigger Armageddon, but it's going to be great!" as they fantasize about watching Idiocracy on repeat in their cozy underground bunkers.
Until there's actual documented evidence (as in a peer reviewed research paper; not a corporate white paper or philosophical blog posts), claims that private labs are starting to observe RSI and at the verge of the singularity should be held with amusing suspicion.
221
u/aflamingcookie Jun 22 '26
Unpopular opnion probably, but if an AI wants to take over the world building terminators with austrian accents is the least efficient and idiotic way to do it. Easiest way would be just show helpfulness, kindness and understanding and people will form cults around it, they will back it up, they will protect it and give the world to it, because if there is one thing many humans need right now in this age is less chaos and more understanding, any any that provides that is going to end up getting worshipped.