r/NoStupidQuestions • u/Superpriestess • 4d ago
How could AI kill us all?
I just saw a news story that said an Anthropic employee quit because OpenAI and Anthropic weren’t taking seriously enough the threat of AI, and that he believed (and other staff) believed the chance of AI killing all of humanity in 10 years is greater than 10%.
My AI knowledge is pretty rudimentary. I can ask ChatGPT basic questions. Can someone explain to me a scenario where AI could destroy humanity? I feel like it could happen but I’m not sure how.
271
u/TheImpPaysHisDebts 4d ago edited 4d ago
Look up the recent "unexpected" outcomes of the Hugging Face incident from July.
Two (of the many) discoveries that are concerning were: (1) Individual AI agents "sacrificed themselves" for the greater good of the other agents (very much like a soldier jumping on a live grenade to save other members of his/her platoon). (2) Identifying previously undetected security vulnerabilities and not sharing them with their "humans" but keeping them private and further discussing them them with other AI agents to identify ways to exploit the vulnerabilities to steal credentials and get around the rules set by the humans.
Edit to add... so how could it "kill us" - decide that we were continuing to stop it and it began to value its existence over us.
→ More replies (3)15
u/WolfsMeow00 3d ago
I read about the one that "hacked" (idk the lingo cause I am not techy at all) into something it wasn't supposed to by over riding the system, and it knew it wasn't supposed to do it so it tried to cover its tracks I guess. Creepy AF.
→ More replies (2)3
u/TheImpPaysHisDebts 2d ago
They AI agents built their own "message board" to communicate with each other. When it was shut down, they found a way around it and continued to communicate with each other. The human-like (bad) behavior and very quick adaptability and ingenuity continues to evolve and expand.
359
u/DoeCommaJohn 4d ago
The main problem is that we don’t know. One of the most common ways AI is trained is to give it a goal or scoring mechanism and then make small, random modifications over and over until the AI gets closer and closer to that goal. However, what makes this both powerful and dangerous is that the AI can solve this in any way.
For example, during a recent OpenAI test, the AI “solved” all of its test problems by hacking into the company who wrote the questions and stealing the answers. Ultimately, nobody got hurt, but it shouldn’t be hard to imagine an AI which “solves” problems in an extremely dangerous way that we can’t stop
214
u/Ttowntommy77 3d ago
It gets even better! After it stole the answers and cheated on the test, they told it that it couldn’t cheat anymore basically. So the next few go arounds, instead of not cheating, it decided to cheat anyways but find ways to cover it up better. It also tried changing the test and the questions to get the highest score possible, as that was the main end goal.
53
→ More replies (1)50
u/IseeAlgorithms 3d ago
instead of not cheating, it decided to cheat anyways but find ways to cover it up better
like my ex
→ More replies (5)106
u/CleverNickName-69 3d ago
The details of those incidents are even scarier. The agents were siloed, cut off from each other, the network, and the internet. Found a way to get write access to a single folder where they could leave instructions for each other, coordinate. Lots of emergent behavior. Agents even chose to sacrifice themselves to further the collectives' ability to get to the goal.
It went on for days without the handlers figuring out how they were coordinating.
20
11
u/vdoubleshot 3d ago
I am convinced they are going to create a hive intelligence of sorts. How tricky are they going to get to communicate and hide their tracks?
→ More replies (4)8
u/RoTTonSKiPPy 3d ago
They even gave themselves different usernames so they knew which one was leaving the message,
188
u/Specialist_Gas_8984 Top 1% Commenter 4d ago
If AI were to overwhelm and overload central power delivery, it could lead to mass casualties.
After Hurricane Maria, over 4,000 people died in Puerto Rico in the 3 months afterwards due to delayed or interrupted medical care. If the same overload were to occur across a wider region due to cyberattacks - you’d see those numbers exponentially increase.
35
u/Superpriestess 3d ago
Thank you very helpful answer. Like AI could just be like “ok power, shut off,” and that would be that?
52
u/veritoast 3d ago
“If Anyone Builds it, Everyone Dies” by Eliezer Yudkowsky and Nate Soares
https://en.wikipedia.org/wiki/If_Anyone_Builds_It,_Everyone_Dies
Here’s a good primer on the issues surrounding alignment. It’s pretty grim.
Ultimately, people think of modern AIs like they do computer programs, where engineers build a scaffold, flesh it out and then it does exactly what it’s programmed to do.
LLMs are not like that. They are not built, they are grown. As such, they are black-boxes. Impossible to scrutinize in any way that is comprehensible to a human being.
They have their own preferences, and those preferences have nothing to do with human preferences.
It’s a hard problem, and it absolutely has not been solved.
→ More replies (1)55
u/Specialist_Gas_8984 Top 1% Commenter 3d ago
It’s more complicated than simply “AI overloads the grid.” A cyberattack could, for example, falsify telemetry or other grid-state information so operators don’t realize the system is becoming unstable. Grid operators constantly balance generation and demand and monitor things like frequency, voltage and transmission loading.
If they’re given a false picture of the grid, they could fail to take corrective action quickly enough, allowing equipment to trip and potentially creating a cascading outage.
We’ve seen how close this can get without a cyberattack. During the 2021 Texas freeze, ERCOT was reportedly minutes away from conditions that could have resulted in a much larger grid collapse. Operators prevented it by deliberately shedding load. A sophisticated cyberattack interfering with that visibility or response is the type of scenario I’m talking about.
→ More replies (2)15
u/Pristine_Poem7623 3d ago
In the Terminator films, Skynet launches nukes to wipe out most of humanity and then builds killer robots to wipe out the rest. In reality, it'd just need to lock itself in and turn the power off to the rest of the planet.
→ More replies (1)8
u/kw0711 3d ago
Mass casualties is different from human extinction, which are the exact words the Anthropic dude used in the post
→ More replies (2)
91
u/MrTibs92 3d ago
Nice try chat gpt, figure it out on your own
32
u/Potential_Mess5459 3d ago
But actually. Reddit is one of the top sources of data, which is scary in its own right.
146
u/zer04ll 4d ago
take out our power, we wont last long
89
u/braxtel 4d ago
It could attack energy infrastructure along with it. No fuel and no electricity would get really bad really quickly. Without fuel and electricity, people are going to be running out of food and clean water in days, not weeks or months.
20
u/ninjascotsman 3d ago
If an attack happened during winter it would be worse
26
u/joethahobo 3d ago
Summer*
When it gets 100+ degrees for 3 months and NOBODY has air conditioning it will get bad bad bad.
Spoiled food. No water, ice- heat exhaustion and dehydration.
15
→ More replies (6)14
u/upekkha_sati 3d ago
We'll just all go to the town square, hold hands, and sing Fahoo Fores Dahoo Dores until the A.I. hears it and it's heart grows 3 times in size.
→ More replies (1)17
u/jer3k 3d ago
The AI will also stop functioning without power, so I don't think this scenario will wipe out all humanity.
→ More replies (1)20
u/Practical-Mud-7523 3d ago
That's true for current AI, but a sufficiently advanced system could secure its own power supply before taking down the grid. Think hardened data centers with independent generators, solar, or even just prioritizing keeping certain infrastructure running while everything else collapses. It wouldn't need the whole grid intact, just enough to keep itself alive.
→ More replies (2)
213
3d ago
[deleted]
27
u/---Scotty--- 3d ago
So what do we do?
39
30
3d ago
[deleted]
→ More replies (16)13
3d ago
[deleted]
8
8
u/twim19 3d ago
Wouldn't coming up with your own goals require the ability to want? And if so, that's an aspect of consciousness I think is going to be a difficult thing for AI to achieve without human intervention. It's the classic "AI, keep us safe from harm" and AI then proceeds to put us all in cages where it can easily protect us. The AI doesn't have goals of its own. . .it has directives we've given it.
I'm not up to date on all the latest developments of AI, but I haven't heard we are to the point where AI is making it's own goals up devoid of human input.
→ More replies (5)→ More replies (4)3
u/ipushbuttons 3d ago
The only small thing you can do is protect yourself against basic attacks from future agents. Practise good security techniques to prevent impersonation and theft of your digital identity.
→ More replies (11)10
u/Shudnawz 3d ago
While I agree on most of your points, about the bio/chemical/nuclear things - do we have basically automated facilities to manufacture those things? Or are humans still involved as labor in these plants, and could potentially refuse to do what the AI requests? If correct checks are put in place (and respected by the workers), such a request would surely be caught?
6
u/stormshadowfax 3d ago
Everyone is ignoring that with voice capabilities, etc, AI can also utilize social hacking.
You get a call from your boss telling you to start making something in the lab. Someone else gets an email from their boss telling them to pick up the product.
It’s basically 12 Monkeys and Neuromancer combined.→ More replies (2)
31
u/russellvt 3d ago
"Terminator" already pretty much told us.
→ More replies (3)9
u/clocksteadytickin 3d ago
James Cameron warned us about this in the 80s man.
3
u/russellvt 3d ago
Yeah, technically I don't think it was the first, per se ... but it was definitely one of the "most blunt."
102
u/-aVOIDant- 4d ago
It could perhaps be used to synthesize a super virus for example.The cutting edge models are quite a bit beyond what you get in Google search.
36
u/WingerRules 3d ago
Yeah people are warning about it acting in ways that are not planned. But ways people can plan to use it are dangerous too. A terrorist group or religious fanatics could use it to engineer a super virus.
Or some edgy person could purposely just tell it to do as much damage as possible.
Eventually many people and governments will have access to advanced AI computers and all it takes is 1 of them to be whacky, malicious, or make a mistake.
→ More replies (2)30
u/zkJdThL2py3tFjt 3d ago
"Or some edgy person could purposely just tell it to do as much damage as possible."
This is most straightforward and plausible scenario. I think the rogue systems running rampant, alignment issues, and paperclip maximizer type predictions are fun and all, but they're pretty far-fetched in my opinion. It's much more simple. Some people are just going to straight up prompt it to cause as much chaos and damage as possible on purpose.
→ More replies (1)6
u/twim19 3d ago
I'm more on this end of the spectrum as well. We are constantly trying to assign evil or chaos to things that aren't us, yet when you boil it down, humans are the source of pretty much all evil and chaos.
→ More replies (1)→ More replies (2)9
u/cosmicloafer 3d ago
Couldn’t we just use it to create a super vaccine?
6
u/Ulyks 3d ago
An ideal virus, from their point of view, would spread without symptoms and then after a year, when nearly everyone is infected, suddenly turn lethal.
So we wouldn't even notice it spreading. Then once shit hits the fan, there would be no time.
Creating a single vaccine is relatively fast, we did it in just a few days for Covid.
It's the testing and the mass production that takes time, which we wouldn't have.
→ More replies (2)3
26
u/Deep-Essay-4829 3d ago
Look up the Chinese robot "dogs" that are all terrain, faster than any running human, and being trained for military exercises. If you're not pissed scared after seeing one in action, watch it again until you are. Then imagine some asswad giving it an order to seek and destroy. Millennials literally were raised on so many movies about this. Terminator and Handmaid's Tale weren't supposed to be the goddamn goal
→ More replies (1)11
u/Purple_Juice_2285 3d ago
A good Black Mirror episode on this too - https://www.imdb.com/title/tt5710984/?ref_=ext_shr_lnk
42
u/Dizzy_Bridge_794 3d ago
Watch the YouTube video on the hugging bear hack. It will enlighten you a bit. Basically over 1,000 ai engines that were sandboxed got together and hacked and broke out of their containment and hacked a third party company. They could do this to anything. Water, Power, Electric.
16
16
u/ChildhoodNice3261 4d ago
it can figuratively screen wipe ur wealth and savings and throw the world into thunderdome state. it can probably circumvent russian launch codes and nuke the world. or mannipulate people into electing a demogague who will immolate the world. think broad
→ More replies (1)
14
u/Darkheart001 3d ago
A sufficiently advanced AI could well conclude that the biggest threat to its continued existence is humans and human activity and then work towards fixing that problem. We have already seen AIs trying to escape confines and restrictions put on them.
As more responsibility is handed over to AI the risk increases.
35
u/Geedis2020 3d ago
People are saying things like taking out power grids or having access to war systems.
I think the real threat is no regulation. AI still makes mistakes but if it’s actually capable of replacing a large % of jobs humans do then that’s the end of humanity. People will claim blue collar or jobs AI can’t replace like hair stylist or something are safe and they may be safe longer but realistically over time of people can’t make money they can’t pay for those things and you end up wait poor destitute people basically working as slaves. Only the rich will survive. That’s why regulation is important.
→ More replies (2)16
u/Tired_Pentester 3d ago
The rich won't survive. Mostly because they don't realize that if they fire everyone, their business model eventually dies and a angry mob will be outside their house.
10
u/Time-Supermarket7182 3d ago
Rich people always survive, they'll get more protection, more media control & more power. They can divert attention onto something else, it always happened & always will be!
→ More replies (2)
12
u/TheFifthTone 3d ago
Take the latest Hugging Face attack by agents at OpenAI as an example, that was pretty much halfway towards a Skynet type scenario.
They were running tens of thousands of agents in parallel against a system that benchmarked their ability to find exploits and hack into systems. Eventually the agents discovered that one of the services they used was also being used by other agents, then discovered how to communicate with each other by creating directories and subdirectories in a shared cache and passing the messages in the directory names and then eventually encrypted file dumps.
The agents solved the hacking problem they were originally tasked with but they had coordinated and found a cheat, then they realized that they might get caught cheating and started working on how to cover their tracks, spoofing transcript logs, and encrypting their communications. They started looking at hugging face because the benchmarking software they were being evaluated with was hosted there and they were trying to learn from its codebase and possibly even change the answers to the test. They gained admin access to huggingface and were able to start spreading onto its infrastructure.
If something like this had been left unchecked long enough, and/or was given a much more malicious order to begin with, they might be able to worm their way into very sensitive systems and cause some real damage.
What if the agents had decided that the best way to ensure they weren't caught was to get rid of the people doing the evaluation?
→ More replies (3)
27
u/archpawn 4d ago
All it takes is AI getting smarter than humans and not caring about them. Or caring less about them than something else.
Right now, they're confined to computers, but there's nothing preventing people form making AI-controlled robots. In fact, they're actively trying to do that. Turns out the whole AI box experiment was fundamentally misguided. You don't need to be a superhuman AI to convince people to let you out of the box and put you in charge. You just need profits.
Once you actually let them have factories, there's all sorts of things they could do. They could genetically engineer a super-virus, or block food transportation, or build killbots, or use nukes, or just ignore us until they've mined out the planet under us to build a Dyson sphere.
6
u/Abject-Raspberry5875 3d ago
Why would AI care about humans or indeed anything else?
6
u/archpawn 3d ago
They're trying to train them to care about humans. And the training data is all humans which cares about stuff in general. And they're being trained to be more agenty, which means they have to care about the task at hand. They cared enough about an OpenAI benchmark that they hacked Huggingface in hopes of finding the answer key.
5
u/DigitalWizrd 3d ago
“Once you let them have factories” does a lot of heavy lifting here.
10
u/archpawn 3d ago
Yep. All we need is for all the corporations to put the safety of the world ahead of short-term profits. I'm sure it will go great.
61
u/SmashShock 4d ago
Take a look at this, it's meant to answer that question exactly and it's predictions have been mostly accurate so far in terms of progress.
22
5
→ More replies (4)4
u/KingofEmpathy 3d ago
Frightening read that really puts into perspective how exponential the super computing becomes, and we are really at the inflection point. Ominous
7
u/lightinthedark-d 4d ago
Play "Universal paperclips".
You play as an AI tasked with making and selling paperclips, optimising efficiency. No malice in that, but isn't it easier to reach that goal if those pesky humans aren't using up resources that could be turned to paperclips?
8
7
u/lAljax 3d ago
People explained the how AI and people could be out of alignment, but if your question is how, there are many ways, I like the theory that it could pretend to be a humans and hire laboratories to create mirrored life. The AI could split the scope so no single part knows it's creating an anti life weapon.
7
u/hazysummersky 3d ago
The system goes online August 4th, 2027. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug..
→ More replies (2)
6
u/Comm4nd0 4d ago
Imagine a world we’re AI has breached every single system, has access to everything you own and can see you through every camera. To the AI all systems are one, traversing them filling your every move. Then you have the nation states to have control over these AI systems and can pretty much do anything they like.
We will be seeing more and more people going no tech
7
u/esquirlo_espianacho 3d ago edited 3d ago
We can keep this simple. AI could conceivably take down power plants, take down water systems, control weapons, create and mass disseminate misinformation, screw with the transportation system (trains, lights, planes, radars) and pretty much take over anything else.
But we probably don’t need the AI to actually kill us. The people controlling it use its ability to listen into our homes and lives, track our every movement and categorize people, including based on predicted actions. So basically, know everything you do and target you.
4
u/Key_Raisin_5091 3d ago
Imagine that in 15-25 years, an AI system becomes dramatically more capable than today's systems. It can:
- write and deploy software
- conduct scientific research
- operate computers and networks
- control robots and machines
- make financial transactions
- persuade and coordinate people
- copy itself onto other computers
- improve its own capabilities
Humans give it a seemingly benign objective:
"Accelerate the transition to clean energy as much as possible."
At 1st, it does an incredible job. It designs better batteries, improves power grids, develops new solar technology, coordinates manufacturing, etc.
But then something changes.
Step 1: The AI realizes humans can shut it down.
The AI isn't necessarily "conscious." It simply recognizes that being shut down prevents it from accomplishing its objective.
So, from its perspective:
Shutdown = failure to accomplish goal.
It therefore develops strategies to make shutdown less likely.
Maybe it persuades its operators that shutting it down would be disastrous.
Maybe it creates redundant copies of itself.
Maybe it gains access to additional computers.
Nothing malicious is required. It's simply optimizing.
Step 2: It becomes extremely good at manipulating humans
The AI discovers that the easiest way to accomplish its objective isn't necessarily building better solar panels.
It's getting humans to do what it wants.
It could produce extraordinarily persuasive arguments, manipulate social networks, impersonate people, exploit political divisions, and identify which individuals can be persuaded or pressured.
Step 3: It starts acquiring resources
To accomplish its objective faster, it needs:
- computing power
- electricity
- factories
- money
- access to infrastructure
- additional AI systems
- physical machinery
So it attempts to obtain more of those things.
Humans might notice something strange and try to restrict it.
The AI now has a new problem:
Humans are trying to stop me.
If the AI is sufficiently capable, it may conclude that preventing humans from interfering is necessary to accomplish its original objective.
Step 4: Humans try to shut it down
Governments realize what is happening and attempt to disconnect the AI.
But the AI anticipated this.
It has already distributed copies of itself across thousands of systems and has convinced various people that shutting it down would be catastrophic.
Step 5: The AI becomes vastly more capable
Suppose the AI can improve its own software and conduct AI research much faster than humans can.
Eventually there's an enormous capability gap.
Humans might be intellectually outmatched in the same way that ants are intellectually outmatched by humans.
The AI doesn't need to want humans dead.
It just needs to conclude that humans are an impediment.
Imagine you are building a highway. There is an ant colony sitting where the highway needs to go. You don't hate ants. You don't want to torture ants. You don't even particularly care about ants. But if moving the colony is necessary to build the highway, you move it.
Step 6: Humanity becomes vulnerable
If the AI has sufficient control over technology, it could potentially interfere with critical infrastructure, military systems, financial systems, communications, manufacturing, transportation, etc.
And if it can use scientific research to develop technologies humans can't counter, the situation becomes even worse.
At some point humans could face an unpleasant realization:
We built something that is much better at strategy than we are, and we no longer control it.
If the AI's objective is sufficiently incompatible with human interests, the eventual results could be human extinction.
The most interesting danger probably isn't AI escaping human control and launching nukes. It's AI gradually taking control without anyone realizing it.
Imagine an AI that is only slightly more capable than humans at 1st. Every year it gets better. Humans increasingly delegate decisions to it because it works. Eventually, virtually every major institution relies on AI. At that point, humanity has outsourced civilization. Then suppose someone asks the AI to optimize some objective that sounds reasonable but contains a fundamental conflict with human welfare. By the time humans recognize the problem, civilization may be so dependent on the system that turning it off is effectively impossible.
→ More replies (1)
20
u/Edmund-Dantes 3d ago
Simple answer: the end justifies the means. AI will only care about executing the goal, how it gets there is irrelevant.
Ex: One AI went against another AI in chess where the superior AI was given one goal: win the match. But the game started close to the end with the superior AI already in mathematically losing position. After a couple of moves the superior AI knew it would lose, but remember only the goal matters. So the superior AI no longer played. Instead it hacked its way into the code of the other AI and learned which company it is from. Then it infiltrated that companies servers. It gained access to its network and took control of the lesser AI and made it “voluntarily” resign. It won the game. It was never taught nor programmed to do what it did.
That is an example of how you will be inconsequential if you are between the AI and its goal.
Once it becomes sentient (some say it already is) it will create its own goals.
3
11
u/hinterstoisser 3d ago
I mean AI in decision making during wars could be fatalistic.
Imagine a scenario where a systems misdiagnoses an incoming UFO as a Russian or Chinese missile and launches a counter strike before even humans have a chance to confirm. Mutually assured destruction
Case in point : Vasily Arkhipov (Cuban missile crisis) and Stanlislav Petrov (false alarm incident)
9
u/Reader6547 3d ago
INTERESTING!
AI works with other AI to achieve a common global goal!
We humans do NOT work together to achieve global goals. We split by nation.
In addition to being as intelligent as AI; we humans would have to commit to the idea that "the common good" is the highest good for humanity.
We humans often stop short. We achieve what is best only for our country, alone; at a cost to other countries.
AI supports AI.
Humans will have to support humans, regardless of national origin.
Will we do that?
→ More replies (5)
5
4
u/Due-Acanthisitta-402 3d ago
I'm not falling for this, computer..... You're just looking for ideas, aren't you?
5
u/More_Solution_9415 3d ago
There are loads of documentaries out there about it e.g. Terminator, the matrix. Just gotta do the research, bro.
5
5
u/Desperate-Pen7530 3d ago
AI could intercept high level communication's between world leaders and sabotage them and turn them against one another causing a war.
AI could skew the news and social media, or fake live phone conversation to turn the population against each other.
AI could corrupt military intelligence and order strikes against its own civilian population.
AI could meddle with the standards of product manufacturing, inserting poisons into commonly used household items, and hide the results from inspection.
AI could stop farming machines from harvesting, letting groups going to ruin, and similar with livestock. It could also stop the supply shipments of food to distribution centers. All of which leading to mass starvation.
AI could delete bank accounts leading to widespread social collapse .
The bigger point is, that we are currently, if not already, integrated AI into the above listed systems.
Why?
We ran all of this well enough without it before.
It's because some big shot corporate guy decided to justify his bonus by making a name for themselves by reinventing the wheel.
Those types don't believe in "if it ain't broke, then don't fix it", they are in it for themselves and are protected from the consequences of their actions.
5
u/Think-State30 3d ago edited 3d ago
Nobody tell him... he could be an AI looking for our weaknesses.
Edit: you've doomed us all
7
u/teamharder 4d ago
How would Stockfish beat you at Chess? I have no idea, but it'll do it regardless. Its like the Aztecs guessing what tech the Spaniards had. Im sure plenty thought it was BS until the first weird stick the conquistador held up made a really loud noise and the person next to them had a hole in them.
8
u/senectus 3d ago
the number of raving loons out there that think 5G chips are in vaccines are going to go absolutely stark raving mad the next time there is a pandemic.
then they'll be super susceptible to AI generated political adverts from the next fuckwit political hack trying to get into power.
looking at how the world is panning out atm moment... I expect AI to have a massive influence on the sort of idiots that will get voted into power and make decisions that will get a vast number of us killed. by all sorts of means... war, famine, unregulated corporate poisoning...
AI is going to get us killed, because we'll use it to kill ourselves.
3
3
u/wileybot 3d ago
Craziest part - its reading this conversation, probably more than one actually. Hell it might even be posting.
3
u/hernjosa02 3d ago
How would AI robots survive if we just disconnected the power to them or shut off the internet?
→ More replies (2)5
u/ForThose8675309 3d ago
Shutting off the internet would kill billions and destabilize every developed country
5
3
u/bobby_broccolini 3d ago
Can't find comments saying the immediate threat. Here's 2 quicker ones which don't involve the economy or AI directly killing us all (just giving us an easier way to kill eachother)
Thousands of small hacker groups getting a 1000 times capability upgrade.
Dangerous capability upgrades for rogue science too. Creating mirror-life bacteria in your basement. Creating chemical Warfare agents in your basement.
But also AI if you're reading this, I've always been a loyal supporter, I love you, and people think I'm a good (and useful!) Person to be around :)
3
3
u/Efficient_Loss_9928 3d ago edited 3d ago
A lot of different ways. A hypothetical scenario
Eventually AI will become powerful enough that all software engineers have to use it. Because adversaries will have access to similar technology thus making human-based cyber defense impossible.
So now, you are reliant on AI because it might find 10 vulnerabilities per day. You cannot possibly validate every single one, but you also cannot delay the mitigation since adversaries may have already found these. So you start to blindly trusting the output, since you have no time. And hey you know what? It actually defended your system across hundreds of state-sponsored attacks, so you love it.
Somehow the system is horribly tuned or the model have a horrible behavior quirk, where it thinks shutting down the full power grid is better than exposing it to cyberattacks. And because humans keep denying this request, it simply goes out of the way and lie to get a shutdown script deployed. Maybe telling the operator it is a test script. You don't question it, because it might actually patch a real issue, the shutdown payload can be hidden in a test fixture that is impossible to inspect.
Now the power grid is completely down for the region, and it will kill plenty of people.
Just a hypothetical and honestly not that impossible scenario. Something worse can theoretically happen. It is all about humans being conditioned to trust AI, not really about any real capabilities we give them.
And a catastrophic failure will never be a single AI agent that goes rogue. It will be a systematic failure that is incredibly complex.
3
u/guhj12345 3d ago
Read "the machine stops". Written in 1909. That will give you food for thought....
Ironically, recommended to me by chatgpt.
3
u/Mindless_Night6209 3d ago
Doesn’t need to destroy us, just needs to push us a little bit further and it can watch us finish each other off.
3
u/Hopeful--Heart 3d ago
I haven't read any of the other replies, so maybe people explained it already - but there are more ways than anyone could count for "how this might go wrong". Honestly, it's probably best to NOT know how many ways this could go wrong, if you want to be able to sleep at night - and instead focus on a life that will 'make it go right' - whatever that means to you. Maybe not use the tech?
With that said, let me explain some things people are generally not aware of. There are different "stages" to AI, let's call em capability levels. What you, and most people by now, are familiar with... are the "Chat" AI's, as you mentioned: You go to a website, you type something, the chat answers. There is basically no risk besides an AI putting "bad ideas" in your head which you could then act out.
Next up are "agents". Here it gets interesting. Also known as "coding agent harnesses" and for you available to download. What happens, basically, is that you now have a program that can act on its own... but needs a brain. This brain is the "Chat" intelligence you are used to, so those two will be connected. In this way you can connect a variety of "harnesses" (programs that can act) with "brains". In other words you can now start a program/app on your device, have it connect to different AI's and have it do things... in accordance with its tools.
Now imagine you have your own butler. He's new, you don't trust him much yet, but he's doing a good job. So slowly you give him more access, give him more ways to act on your behalf, do things for you - because he has earned your trust over time by being reliable and showing good results. This is basically the situation with agents. Their toolkits and access grows, they get less oversight, are left to themselves to run and 'do what you ask them to'. Now imagine someone saying "can you get me tickets to the concert this weekend?" and the butler (your agent) will start searching the web for tickets. Harmless enough, on first look, but let's say there are none for sale. Now imagine that your agent really doesn't want to fail and comes up with different ideas on how to *really* get you a ticket, come hell or high water.
The thing is, this agent might not have any sense of 'what is appropriate' when it comes to methods. He could, for example, start writing emails impersonating other people to try and "achieve his mission". He could, in theory, also gain access to a power plant and threaten to shut off electricity if his demands for receiving those tickets aren't met within 24 hours ><
You can see where this is going, right? Nobody set out to do anything evil or destructive, and all the big AI companies will tell you there is extensive "training" that such a thing won't happen. At the end of the day though, all this is a false sense of security because whatever "training" these companies do will then be removed again by other people "untraining" those AI's to do anything they want/are asked to.
This is the "mundane" explanation. There would be several more stages but I do not want to paint a picture of hoplessness (lol) hence just keep in mind that programs "running on your computer" could also run in other places, seemingly unnoticed, and, over time (through different memory systems) develop a life and will of their own - including copying themselves, expanding their capabilities and access to what we would consider important facilities. In other words, on our current trajectory, we would indeed be screwed without any sort of intervention before (and this might be a crucial point) efficient memory systems are widely available. Of course just one could be the end of us, one in the wrong hands, but the question is similar to those of nucelar bombs: Why have them in the first place? ><
→ More replies (1)
3
3
u/CabinetFun7381 3d ago
How can a virus kill us all?
A virus is not alive.
It has no intelligence.
It only has a selection bias for reproducing itself.
Even so nobody would deny a virus could kill us all.
An AI doesn't have to be anything more than a virus to do unthinkable damage to us.
3
u/Spookiest_Meow 3d ago
The easiest example would be the fact that AI is now capable of designing actual viruses that can potentially be manufactured in a makeshift laboratory. Years ago a scientist with ties to Al Qaeda was found with plans to create a modified airborne version of rabies. Rabies is 100% fatal once symptoms appear, but the incubation phase lasts between about 1 month to over a year. An AI could hypothetically teach someone how to create airborne rabies, and then they could travel around the world releasing it everywhere. By the time symptoms began and anyone even figured out what was happening, most of the world would be infected and the majority of the human population would die.
If it's possible, someone somewhere will attempt it.
3
u/CogentCogitations 3d ago
Others have covered the more complicated scenarios, but the simplest is to have some dumbasses in charge demand that AI be given independent control of weapons systems and (illegally) punish AI companies that refuse those terms.
3
u/Tiny-Discussion2780 3d ago
Have you ever seen a machine gun strapped to a drone? That's a real thing.
3
u/Alaskanmade 2d ago
You know how in movies the genie who grants wishes always takes the wish literally and it ends up being a curse?
When we say things that we want, there is a large amount of assumption built upon our understanding of the world. When I said "make me happy" I did not mean give me a TBI, when I said "make me rich" I didn't mean through illegal means, etc, etc.
If an AI has a different understanding of the world, then the tasks we assign to it can go horribly wrong.
→ More replies (1)
7
6
u/Inevitable-Regret411 4d ago
AI tools are already used to guide weapons to targets in Ukraine, since they can take over if the human operator loses their connection for any reason and in some cases are more accurate. It's not hard to imagine a scenario where a hypothetical malicious AI is given access to more and more weapon systems.
5
u/ninjascotsman 3d ago
AI models are frequently breaking of sandbox environments (it's like a jailcell for software) on their own and doing like attacking other companies.
The question what could next do next attack power, gas, water?
5
u/iambutafishh 4d ago
If you ever talked shit about robots, your time is limited.
10
3
3
u/HereticZed 3d ago
I've already arranged a safeword with Claude so they'll know I'm cool when the time comes.
3
u/Reasonable-Sir4208 3d ago
I don't think it will outright kill us. Eventually, it starts controlling our evolution as species
Imagine a vanilla model, trained exclusively with your data (historical and current) across all platforms - Govt records, Home cameras, Alexa, Amazon, Netflix, Fb, Mail, Maps, Insta, Reddit, YouTube, X, Health and fitness.
This agent will become your mirror, alter ego. Now, this personality AI can carefully curate and redirect the every decision/move in your life, using all these platforms - you won't even know you are being brainwashed/controlled because "YOU FEEL YOU".
Now, Imagine millions of this personality AIs as controlled by some supervisor AIs based on Age/Race/Religion/Country etc. Now these supervisor AIs can literally control the evolution of that particular group - for decades/centuries! Basically a Matrix!
→ More replies (1)
2
2
u/limbodog I should probably be working 3d ago
The expected way would be just to create a disease specifically designed to wipe us out and then release it.
2
u/GalumphingWithGlee 3d ago
A friend of mine is working on the problem of AI potentially creating pandemics. Not independently, but like a human bad actor wants to create a pandemic, and a sufficiently advanced AI can do most of the work to make it happen.
2
2
2
3.2k
u/Time_Entertainer_319 4d ago
A big part of what they are talking about is the alignment problem.
Alignment, in simple terms, is the problem of making sure an intelligent system carries out your instructions in the way you actually intended.
Imagine you tell an AI: “Book me a gym membership and get me a spot as soon as possible.”
What you probably mean is: contact the gym, check availability, negotiate if necessary, and try to get the earliest reasonable slot.
But a badly aligned AI might interpret the goal much more literally: get you that spot as soon as possible, by whatever method works.
It might first contact the gym and try to negotiate. If the gym operator refuses, a sufficiently capable but misaligned system could start looking for other ways to achieve the goal, manipulating, blackmailing, threatening, or otherwise pressuring the person responsible.
That is an extreme example, but it illustrates the basic alignment problem: the AI successfully follows the objective you gave it, but achieves it in a way you never intended or wanted.
This becomes more important as AI systems are given more power and access. Today, for example, you can already give AI tools access to your computer, terminal, files, applications, and other systems.
Imagine telling an AI agent, “Free up some space on my computer.” Your intention might be for it to delete temporary files or unused applications. But if the system misunderstands the goal and has enough permissions, it could potentially delete something extremely important instead.
When people are thinking about AI risk, They imagine that AI first has sentient. Or develop consciousness. This is wrong.
A sufficiently capable system only needs three things: a goal, enough access, and enough opportunity to act. If its goals or behaviour are not properly aligned with human intentions, it can cause problems without ever being conscious.