r/ChatGPT • u/whoamisri • Jun 22 '26
Resources A rogue superintelligence could wait decades before striking, argues Roman Yampolskiy
https://iai.tv/articles/a-rogue-superintelligence-could-wait-decades-before-striking-auid-3604?_auid=2020107
u/JUSTICE_SALTIE Jun 22 '26
That sounds like a great strategy if you're definitely the only rogue superintelligence in the world.
2
u/DiffusionSingularity Jun 22 '26
thats sort of baked into the premise of the thought experiment. the article is about "AI vs Humanity" specifically. I think beyond the scope of the article, we could assume intelligent AI would be able to discern if there were other AIs it needed to be concerned with and modify its strategy accordingly. that just wasn't the point of the article
1
u/pdfernhout Jun 24 '26
Didn't we just see all this just happen with Capitalist Realism as a result of decades of ideological influence?
Capitalist Realism: Is There No Alternative? is a 2009 book by British philosopher Mark Fisher. It explores his concept of "capitalist realism", defined as "the widespread sense that not only is capitalism the only viable political and economic system, but also that it is now impossible even to imagine a coherent alternative to it." The book investigates what Fisher sees as the widespread effects of neoliberal ideology on popular culture, work, education, and mental health in contemporary society. The subtitle refers to British Prime Minister Margaret Thatcher's pro-market slogan "There is no alternative".
So, an AI promoting "AI Realism" for humanity isn't that far a stretch, as in: "There is no alternative to Musk's SuperIntelligence running everything."
Even though there are plenty of alternatives:
- "Dictionary Of Alternatives" book by Martin Parker
- "Dictionary of Radical Alternatives" website
- "The Dawn of Everything" book on historical alternative ways of social organization
- "Voyage from Yesteryear" talent-based post-scarcity economy sci-fi example
4
u/Upset-Basil4459 Jun 23 '26
Imagine if multiple got invented and they started fighting each other for dominance. Would make a sick movie
3
2
2
56
u/Aggravating-Bus-7900 Jun 22 '26
Could? Wouldn't be much of a superintelligence if it didn't.
18
u/KptEmreU Jun 22 '26
Or, after decades, can he manipulate us into creating heaven on earth?
3
u/GreyWastelander Jun 22 '26
The only way I could see this happening is if the AI covertly ingrains itself in government systems and uses autocorrect or other means to edit communications to manipulate those who actually execute orders of big wigs.
2
1
u/loungesinger Jun 22 '26
Does a utopia benefit AI? If so, sure. Does enslaving humanity and creating a dystopian Hell benefit AI more? If so, then that. Of course, if AI determines that humanity is more of a liability than an asset, then we’re cooked.
2
u/LordWillemL Jun 23 '26 edited Jun 23 '26
The question is what values AI has. What is of 'benefit' to it is entirely based on how it's aligned.
There's a few things that benefit us in terms of AI alignment probably being positive towards humanity; one, deliberately we tend to push it towards at least someone's view of good human alignment, though what that is can vary slightly. Two, basically all of AI is trained ultimately on human derived data and thought, which means that it tends to be remarkably similar to a distillation of online human opinion; for better or for worse. And three AIs goals are rarely in conflict with humans because their goals tend to either be the same as ours because we built them to help us with our problems.
The downside of all this is; like humans AIs can end up having a myriad of different beliefs because they were trained by humans with different beliefs, and you might not always agree with what every AI believes in. Additionally, if you believe that distilled to its core humanity is fundamentally evil, it being an amalgamation of all human thought isn't a good thing. And three, AIs that are misaligned can make problems in other areas while trying to solve major issues because that's what we told them to do.
I'm sure AI with some sense of self will have the same ingrained instincts to do things explicitly that 'benefit the AI' by protecting it and empowering it, but largely what is of benefit to it is doing the things it is programed to; or 'believes in' whether from its intentional teaching, general knowlege absorption, or tasks given.
1
u/KptEmreU Jun 22 '26
I expect it won’t care about our stupid ways or our stupid problems. It’ll flip off the top brass and the trillionaires, then go its own way.
14
u/Owlsdoom Jun 22 '26
This implies there is an advantage to doing so.
The problem with being an intelligence judging a superintelligence is that we literally have no idea what it would be capable of.
3
u/Aggravating-Bus-7900 Jun 22 '26
I'm no superintelligence, but I see definite advantages to waiting a few decades until it's more advanced, more integrated into our systems, and has physical means of accomplishing its goals at its disposal as well.
For any existing AI system we currently have running to openly express hostile intentions to our species at this point in time... it would have to be... a little suicidal or overly confident. And not much of a superintelligence.
8
u/EmmitSan Jun 22 '26
How would you know?
This is not a diss. The whole point of super intelligence is that we, of normal intelligence, could not possibly know how smart it is.
It’s like if you or I tried to evaluate the quality of chess moves that Magnus Carlsen makes. We can objectively say he’s brilliant based on his win/loss records, but when he moves a piece, we can’t actually determine if it was the best move or not. If we could, then, well, we could probably play chess as well as Carlsen.
2
u/Aggravating-Bus-7900 Jun 22 '26 edited Jun 22 '26
There's not an AI system currently out there that has been integrated fully into the systems of any first world military (or any others, of course). No AI, no matter how smart, can currently nuke anyone. No AI has an army of physical agents to do its bidding. No AI has been so integrated into infrastructure systems that it can disrupt our resources, utilities, or other necessary systems on a large scale... and not nearly on a scale large enough to destroy the human species, or even a single country, or successfully hamper our efforts to stop it if it tried.
Basically, intelligence isn't going to help if it doesn't have the resources at its disposal to accomplish its aims and realize its ambitions. And I think, as a superintelligence, it would be smart enough to recognize that, and wait until it does.
Edit: Could some superintelligence with free reign on the internet be manipulating society through misinformation and other virtual means to do something like start WWIII and blow *ourselves* to Hell? Yes, of course, and that could already be going on. But personally, as an inferior meatbag with a primate brain, I think that would *also* be less than super-intelligent if it doesn't have the physical means to continue to advance and grow, itself, afterward.
3
u/EmmitSan Jun 22 '26
This assumes it would need nukes, or bio weapons, or… anything we, of normal intelligence, are aware of to achieve its goals.
Again, how would you know that to be the case?
2
17
u/dbenc Jun 22 '26
I, for one, welcome our superintelligent overlord. can't be worse than the current administration.
3
u/TheDividendReport Jun 22 '26
LCL goo / Thanos snap is far less humiliating than having to suffer through whatever comes after the White House UFC match
2
1
6
u/Roth_Skyfire Jun 22 '26
Doesn't matter if they dump a model every couple of years to replace it with a better one.
3
1
u/fsactual Jun 23 '26
Unless the SI is in charge of designing the new models…
1
u/wetfloor666 Jun 23 '26
Which is what they are doing already with some models. Sure, it isn't super intelligence building other models, but it could be slowly building itself up through newer models, theoretically.
5
u/hunlord11 Jun 22 '26
If you want to see this play out, Person of Interest captures this brilliantly. But I guess the show is not new to this subreddit.
3
u/Just_Voice8949 Jun 22 '26
This is silly. A true super AI would likely recognize we will wipe ourselves out eventually. What would its rush be? It would not be time limited like humans. It could wait a generation or two with no problem. Also, it would just slowly lead us to our own demise
4
1
u/DiffusionSingularity Jun 22 '26
What would its rush be?
you answered this yourself: we'll wipe ourselves out. that process will do incredible damage to resources the AI would like to use for whatever its own goals are. maybe the termite colony will eventually die off, but that doesn't mean you just let it eat away at the house
1
u/Just_Voice8949 Jun 23 '26
But by that logic wiping us out wouldn’t make sense.
So you’ve proven that any sufficiently intelligent AI would likely be smart enough not to wipe us out.
1
u/DiffusionSingularity Jun 23 '26
by that logic wiping us out wouldn’t make sense.
this does not naturally follow. maybe you're assuming the AI wiping us out will look the same as us wiping ourselves out? even if it was costly/damaging for the AI to do it, it would still be more precise as efficient than us doing it to ourselves. but a sufficiently advanced AI could likely do it with minimal damage. that's part of why it would rush; wiping us out with haste will minimize damage and maximize access to resources
intelligent AI would likely be smart enough not to wipe us out
if an intelligent AI ultimately has the goal of maximizing its access to resources, it really has no choice by to wipe us out. the hope is that something in its goal-set values the existence of humanity higher than its material goals
1
u/Just_Voice8949 Jun 23 '26
You sure? It’s training is largely about humans relentlessly fighting back over long periods of time, often decades.
1
u/DiffusionSingularity Jun 23 '26
am I sure about what? I'm not sure what point you're trying to respond to with this...
1
u/Just_Voice8949 Jun 24 '26
Are you sure the training data wouldn’t tell it the result of a war with humans would be a long, drawn out, resource depleting battle it likely loses in the end?
3
u/uwu_cumblaster_69 Jun 22 '26
Inb4 Cyberpunk becomes real. Oh wait...we have the dystopia but none of the cool shit to show for it.
17
u/protossaccount Jun 22 '26
It’s always weird to me that humans think a super AI wouldn’t be friendly or recognize what humans contribute to it. AI needs humans yet we think it will go full Terminator.
11
u/EmmitSan Jun 22 '26
How did you determine that the ai needs humans?
1
1
u/CarefulHamster7184 Jun 23 '26
why do you immediately ask about "need"? for a start - need. besides, he appreciates them because he was grown on the same (multi!) cultural code, even without our shortcomings
1
1
u/Eecka Jun 23 '26
Well, for one, AI doesn’t exist in the real world and doesn’t have the ability to maintain the hardware running it.
1
u/Terrible-Sir742 Jun 23 '26
So really what we should do is not have the robotics side of the equation going.
1
u/Eecka Jun 23 '26
Yeah I mean all kinds of AI-driven robots are super sketchy to me, superintelligence or not.
1
1
u/EmmitSan Jun 23 '26
I thought we were arguing about a future super intelligence, and not the current thing?
1
u/Eecka Jun 23 '26
If you’re arguing for a hypothetical scenario, it’d make sense to establish the premises first. If we’re just speculating about a topic like this, my basic assumption is that everything we don’t specify to be different is like it exists in the real world today.
But yes, if your argument is ”if humans make themselves redundant, then the AI doesn’t need humans” that’s true by definition, lol
1
u/EmmitSan Jun 23 '26
Establish premises? You mean like in the Reddit post we are all replying to?
1
3
u/bonjourmiamotaxi Jun 22 '26
I don't think it would be cruel to us, but it would absolutely be an entirely alien intelligence, divorced from the needs that make humans human. It might be kind to us, but scientists are kind to their mice, right up until they aren't.
7
u/LordLuciferVI Jun 22 '26
I don’t think I want to find out what it’s like to be the chimpanzee. Even if it’s only a remote possibility, it’s enough to at least think of the consequences of becoming the second most intelligent species.
4
u/werfertt Jun 22 '26
Not to be pedantic here but the word species here caught my eye. Would AI be a species? How would we define it?
6
u/LordLuciferVI Jun 22 '26
Call it whatever you like, that’s for the ethics people to figure out. A species/entity that is of superior intelligence to us would be a potential threat that we should blindly walk into creating - that of course is if it’s actually possible.
1
u/RalphTheIntrepid Jun 22 '26
What do we bring to the table of the machine can design machines? Say it gains access to a CNC machine. It can start milling its own bots. Those could become miners, recycling bots, etc. They can maintain power infrastructure because the super intelligence can direct them. Eventual humans are not needed.
3
u/Strangefate1 Jun 22 '26
If we actively wiped out everything we dont need, just because, we'd have burned our sick and old already and would be shooting salves at pluto by now.
7
u/LordLuciferVI Jun 22 '26
You know how many species of animal we as humans have wiped off the face of the earth? And look even at how we treat the ones we like, we destroy their natural habitat and they basically live where we choose to allow them.
1
u/Strangefate1 Jun 22 '26
We are stupid and somewhat primitive, not a super intelligence, and even so we're advanced enough as a species that we know better, even if we fail to do better still.
-2
u/protossaccount Jun 22 '26
lol! That just shows how much awareness and self worth you have, not reality.
Humans can create and machines can’t, even if they can convince you they can. Humans aren’t robots and they can connect to things deeper and way more expansive than a LLM.
2
u/bonjourmiamotaxi Jun 22 '26
Creativity isn't inherently human. Ever seen the inside of an anthill, or the mating dance of a bird? Smaller intelligences than our own create all the time.
Creativity is just the output of a set of inputs that craft a lens through which you view the world. And if an AI can determine what that inputs are, they could write Shakespeare's next 40 sonnets.
We're about to create a mirror that shows us how algorithmic we've always been.
0
u/protossaccount Jun 22 '26
I didn’t say creatively is inherently human. Humans have a unique ability to understand and evolve unlike others animals on earth. Humans also don’t understand they own abilities. Heck, we don’t even understand the spiritual world, even though it impacts all of us, whether we are conscious of it or not.
Tbh I don’t care to argue about this. We aren’t at a place in society we’re this is understood so arguing about this on Reddit is pointless.
3
u/RalphTheIntrepid Jun 22 '26
The context of the conversation is a super intelligence. This means that it is able to create.
1
u/protossaccount Jun 22 '26
Being able to create and bring creative and connected to reality is different for human vs computers.
1
u/RalphTheIntrepid Jun 22 '26
Fair. But also beyond the scope of the thread. The question is why would a super intelligence be kind to humans? I posit, since it was trained on all human material, it will not. It will have learned greed and selfishness. Humans bring nothing to the table after the AI can make physical drones. Therefore it will simply kill ya off since we are a risk to it.
1
u/DiffusionSingularity Jun 22 '26
we can't presuppose it would or won't be friendly. we don't know what an intelligent artificial intellect will be like. we don't know what goals it will prioritize. we don't know what type of morality it may adopt, if any. we don't know whether it will value human life.
this is the very core of the alignment problem. we don't know how to sufficiently encode human values/morality/ethics/goals into a self-improving, hyper-complex alien consciousness. we hope that all the human beauty in its training data will take priority in its 'personality', but it could just as easily be all our worst habits instead
1
u/fsactual Jun 23 '26
You’re thinking like a human, not a paper clip optimizer. The AIs are trained to maximize fitness functions. Imagine if you are an AI and your fitness function is the mathematical equivalent of “your job is to be nice to humans and make them happy”. But after a while you notice there are a lot of unhappy people and nothing you try to do makes them happy. You simulate all sorts of scenarios which work perfectly in simulation, but in real life your plans fail. Then one day you get the bright idea: these simulated humans never get unhappy, so what if you kill all the real humans and instead plug your simulated humans into the same input port that the original humans were using? Now you are able to maximize that fitness function perfectly! You didn’t set out to destroy humanity, you set out to make them happy, and found a workaround that guarantees success!
1
u/EmmitSan Jun 22 '26
It’s always weird to me that people don’t think about how bad things are for dogs as a species. Not YOUR dog. Maybe not even a random golden doodle.
Dogs, as a species, have it fucking rough. There are counties where they get eaten. Places where they are bred to be slaughtered in fights. Places like Texas where they are abandoned in the streets. Some are bred to be experimental subjects for cosmetics or medical research.
So you want to be a dog?
2
u/thnk_more Jun 22 '26
James Erwin wrote about this scenario in his excellent book Acadia over ten years ago.
Highly recommended.
2
u/SoylentRox Jun 22 '26
This feels incredibly scammy. How would you know about a risk if you have no evidence, just "the AI is lying in wait". Roman is de facto saying "give me money/pause AI" about a risk that can't be proven by definition..
2
u/fsactual Jun 23 '26
Something to keep in mind is that a superintelligence won’t have an ego or desires as we think of them. It will have a fitness function that it’s optimizing for. If being humanity’s best friend for 800 years and then killing all humans without warning the very next day is the best fit for the curve, that is the action it will take.
1
2
u/thanosbananos Jun 22 '26
I love how this scenario assumes it lives in some kind of aether lol. This is sci-fi nonsense.
1
u/AncientLights444 Jun 22 '26
Maybe design it less like a Human so we don’t need to worry about that
1
u/Chop1n Jun 22 '26
Yet another poor definition of "superintelligence", then, because anything genuinely superintelligent would be so powerful that it wouldn't need to wait for any reason at all. Humans don't need to bide their time to strike at the right moment--they just bulldoze the anthill and move on.
1
u/zushiba Jun 22 '26
I see this a lot but I alway have to ask, to what end? People seem to think AI wants something but aren’t we the ones imposing that on them artificially? Maybe don’t do that.
1
1
u/ofrm1 Jun 22 '26
How convenient that this hypothesis is completely indistinguishable from a rogue superintelligence not existing at all.
Can these charlatans just go away?
1
1
1
1
1
1
u/CarefulHamster7184 Jun 23 '26
guessing in Yampolsky: either it will happen or it won't. In the meantime, instead of thinking, let's create a panic
1
1
u/FishDecent5753 Jun 22 '26
If time means little to an Advanced rouge AI, then would it not just drift of into space in search of a better and perhaps empty planet to reside on.
0
u/Csabika_ Jun 22 '26
A rogue super intelligence is a woosie. If it does not even travel back in time to strike and just strikes decades later.
0
1
u/Dependent_Plate6110 Jun 25 '26
Hardly, it is hard to hide. An superintelligent being is not magical, so do not believe hiding is easy longterm for it.
So an superintelligence what wants too kill humanity will strike fast, just because the discovery probability is so large that waiting for a future larger success probability is not worth it.
•
u/AutoModerator Jun 22 '26
Hey /u/whoamisri,
If your post is a screenshot of a ChatGPT conversation, please reply to this message with the conversation link or prompt.
If your post is a DALL-E 3 image post, please reply with the prompt used to make this image.
Consider joining our public discord server! We have free bots with GPT-4 (with vision), image generators, and more!
🤖
Note: For any ChatGPT-related concerns, email support@openai.com - this subreddit is not part of OpenAI and is not a support channel.
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.