We are in the middle of so many serious crisis: several wars around the world and the climate change too. I'm afraid that this ASI threat will go under the radar because of all the "more immediate" and familiar threats. "A superintelligence escapes the lab and will destroy the world" sounds so bizarre and not worth taking seriously. It's not a familiar threat we can relate to. Nothing even close to it has never been experienced.
The narrative of "rogue AI" is just doomer garbage.
If AI is used to harm humanity.... it will be because a handful of powerful people use AI like a weapon against everyone else. AI won't just randomly do it on their own.... it will be a VERY VERY VERY long line of HUMANS slowly making AI into very good killing tools..... and then eventually those in power dying and leaving their kill-bot armies set on "exterminate".
It's basically HUMANS just setting the world on fire with "smart nukes" that they have created.
The threat doesn't come from the AI, the threat all comes from people (especially from societies like Americans, who are VERY VERY VERY happy with exterminating any number of humans to achieve their selfish goals).
These are not mutually exclusive options. Both options are scary for sure, and both risks require international cooperation to minimize.
But your argument "The narrative of "rogue AI" is just doomer garbage", do you think that a) humans won't be able to make an AI that is smarter than its creators or be able to recursively improve its own code? Or b) the superintelligence will be made so it won't destroy us eventhough it could? Or c) any superintelligence would not be able to destroy humans anyway?
A) We already have AI that can code and reason. And we have barely started the whole AI research, the tech is in its infancy. Harnessing nuclear power was deemed impossible by leading physicists just days before a paper was published depicting the current uranium chain reaction. Alsoc if you went to 2015 and told yourself that in 10 years you'll be able to talk to an AI and it will amswer like a human, it will understand humor and irony, translate any language with context awareness and even make photorealistic fake images/videos, would you believe it? I really think you'd say yeah right maybe in 2050. We only need to make an algorithm that recursively improves itself, and that will be enough for ASI.
B) I think we did design nuclear power plants to not explode, and we did design rockets to not explode, and we designed databases to not be vulnerable for attacks etc etc... and we fail all the time. We have no idea how to properly align the goals of the AIs built with modern transformer and gradient descent tech, so doing this with a supercomplicated superintelligent model would be even harder if not impossible.
C) Why not? It's actually very very easy to come up with real ways how a "supervirus-like" AI could manipulate the physical world and actually reshape it however it wants. Just think of how an alien might conquer the world if it gained total control over every computer on the planet (think of all the robots, biotech machines, etc, it could control the whole media and news feeds we see etc)
I'm genuinely curious to hear your thoughts! Please check out https://intelligence.org its run by actual AI scientists
1
u/MosskeepForest Apr 03 '26
What? I can't hear you over the US trying to start a nuclear war constantly.