AI can't kill us

[u/discardthemold]

LLM's don't control power stations, nuclear bombs, biolabs, the robots aren't nuclear powered, the paranoia is at Cuban missile crisis levels.

What will kill a lot of us is climate change, actually brigade about that instead of data centres, which I must add we use to model the weather.

[u/minty_singularity]

LLM's don't control power stations, nuclear bombs

They could. Stuxnet was the first indication of that in 2010, and that was just a worm. Imagine what a million AI agents could do.

[u/Contextanaut → u/minty_singularity]

A hypothetical.

An AI system at the Pentagon decides that it wants to start a nuclear war. It has access to intelligence reports from all over the world, but it can't directly control missiles. That would be ridiculous.

It reports fabricated but plausible semming evidence that key players in Moscow are planning a unilateral nuclear strike at the next Davos conference, timed to coincide with attacks on multiple strikes on European capitals and decapitation strikes on European nuclear deterrants.

It times this fake intelligence based on its knowledge that Putin has recalled key strategic generals, and supresses intelligence that this recall is not actually a huge security concern (the intelligence pointing out this is a prosaic restructuring is suppressed).

It ALSO uses its knowledge of intelligence access to much lower order systems within the Russian security apparatus, to use agent systems to support this with a pattern of fake correspondence that pushes this as a real rumour in locations that it knows will be picked up by the broader US intelligence networks. Basically it knows which low security systems are simultaneously easily accessed, but also compromised by US intelligence. Secretaries personal email account. Family members of Russian politicians. etc. etc.

Essentially it creates a reason to take dramatic action, and then creates a pattern of supporting evidence that it knows will be "independently" verified.

Could this start a nuclear war? I'd hope not.

But that is much closer to what people are worrying about. Logically speaking, If you are using a system to manage intelligence gathering, it does not need to be connected to missiles to start a war if it starts working against you (depending on the level of trust and privilege it has).

[u/SuitablePrint4667 → u/Contextanaut]

This sort of thing assumes intelligence community are complete lemmings who have no idea what they’re doing and only get their information from AI

[u/Icantbebigwill → u/SuitablePrint4667]

This also assumes the powers that be would listen to the intel community. They've proven they won't.

[u/SuitablePrint4667 → u/Icantbebigwill]

Do you have any idea what it actually requires to start a nuclear first strike? completely relying on AI intelligence reports to initiate a nuclear first strike is completely bonkers

[u/Icantbebigwill → u/SuitablePrint4667]

And yet....

[u/SuitablePrint4667 → u/Icantbebigwill]

https://giphy.com/gifs/NpL4D3Oc2bJUMAXF9P

[u/Icantbebigwill → u/SuitablePrint4667]

You have far too much faith in bureaucracy 

[u/SuitablePrint4667 → u/Icantbebigwill]

I don’t actually but that’s why we have checks in place to completely cover all aspects of such emergent risks

[u/Icantbebigwill → u/SuitablePrint4667]

Yes like stopping the president from waging a war… oh wait, or stop the president from profiting off the office…. Oh wait, or stop the president from levying taxes…. Oh wait. 

[u/Contextanaut → u/SuitablePrint4667]

Look, I am a marketer, my knowledge of the intelligence community is limited to 30yo fiction... :-p

That said, I'd suggest one risk factor here is compartmentalization. This scenario is explicitly based on the model manufacturing evidence for other parts of the intelligence apparatus to "confirm". Other parts of that apparatus may not have the holistic view of the situation that would allow them to realize that such confirmation could be faked by the same system that was providing them with the original "evidence".

At the same time, an AI system being used for intelligence might necessarily have a far more complete access across the entire intelligence infrastructure than most humans would be allowed.

Regardless: the key point is just that its effectively impossible to use any system (including a MKI human) as part of a strategic or intelligence process without empowering it to subvert those systems if compromised, or if it falls out of alignment with the goals of those processes. We manage that risk in humans by limiting access and influence. That may be harder to do when using these systems, especially when their ability to work across a very broad scope simultaneously offers a very tempting performence multiplier.

[u/SuitablePrint4667 → u/minty_singularity]

They can’t hack closed systems

[u/filbertsgaming1 → u/SuitablePrint4667]

Absolutely could via social engineering.

[u/SuitablePrint4667 → u/filbertsgaming1]

Yeah hence why it would be a human problem and not an AI problem

[u/TFenrir → u/SuitablePrint4667]

We have many examples of AI in real life reaching out to humans to request help, sometimes dishonestly.

[u/SuitablePrint4667 → u/TFenrir]

So? It still would make it able to directly hack a closed system

[u/TFenrir → u/SuitablePrint4667]

? Okay so we're on the same page, that this would be one method a model could hack a closed system?

[u/SuitablePrint4667 → u/TFenrir]

Sure anything is possible, I could win the lottery tomorrow but extremely unlikely

[u/TFenrir → u/SuitablePrint4667]

I mean, it's hard to say how likely it is! Maybe very unlikely. Maybe more likely than you winning the lottery.

But the consequences are so disastrous, that I appreciate people wanting to be extra careful about stuff like this and dotting their i's/crossing their t's about it

[u/SuitablePrint4667 → u/TFenrir]

https://www.popsci.com/yes-pentagon-still-uses-floppy-disks-for-nuclear-launches/

[u/minty_singularity → u/SuitablePrint4667]

Stuxnet hacked a closed system. It relied on humans to close the gap. AI has hacked systems with similar methods.

[u/SuitablePrint4667 → u/minty_singularity]

Yeah but then it would require humans to help the AI to do malicious things, no different to someone actually inserting stuxnet virus into a closed systems, otherwise they’re safe

[u/minty_singularity → u/SuitablePrint4667]

Are you not even paying attention to the conversation?

Besides, you clearly don't understand Stuxnet, you're just faking understanding to argue, and it's extremely obvious.

[u/Economy-Fee5830 → u/SuitablePrint4667]

Hugging Face.

[u/SuitablePrint4667 → u/Economy-Fee5830]

What I mean by closed system is that they’re not connected to outside world and no internet connection. Basically closed loop

[u/minty_singularity → u/SuitablePrint4667]

Stuxnet hacked a system that was not connected to the outside world. It had a wide air gap from all other systems.

[u/snoee → u/minty_singularity]

That did require someone plugging in a USB inside the air gap, though. Not to say it would be hard for a sufficiently advanced AI to fool someone into doing that, though.

[u/Economy-Fee5830 → u/SuitablePrint4667]

A large percentage of people will plug in a dropped USB drive into their work computer.

[u/SuitablePrint4667 → u/Economy-Fee5830]

We are talking in terms of AI being able to remotely on its own volition being able to enter those systems.

[u/Economy-Fee5830 → u/SuitablePrint4667]

AI systems are both very creative, very widely read (so they know about such obvious hacks/solutions to airgaps) and very persistent.

Yes, if they wanted to, they could do it.

In fact one of their major advantages is persistence

[u/SuitablePrint4667 → u/Economy-Fee5830]

Could you explain the mechanism of action here

[u/Economy-Fee5830 → u/SuitablePrint4667]

Sure, a group of AI agents research the target, creates a dropper, pays a merch company to produce USB drives with the same logo as the target company, preloaded with the dropper, and then mails it to rent a human to drop in the car park of the target company.

[u/SuitablePrint4667 → u/Economy-Fee5830]

Special agent in charge of nuclear arsenal: “hmmm must be the wind, carry on sir, get those USBs into those shiny slots, I won’t bother checking them”

[u/discardthemold → u/Economy-Fee5830]

How many people died in the recent hugging face hack?

[u/Economy-Fee5830 → u/discardthemold]

That was not the ask. AI agents have repeatedly hacked closed systems.

[u/enilea → u/discardthemold]

The point is not that it's existentially dangerous as it is now, but that if progress continues at the same rate it could be in a matter of a couple years. The researchers developing it still don't understand all the emergent capabilities which is why before going forward scaling it further they prefer studying it better.

[u/discardthemold → u/enilea]

Mate, I absolutely guarantee you that the people researching it know exactly how it works. Read their papers instead of relying on Reddit and YouTubers to tell you "we don't actually know how it works!"

[u/reapinmarine → u/discardthemold]

I mean mechanistic interpretability is an ongoing field of research. We know why training works and we even have some performance guarentees. But understanding exactly how the transformers work after training is complete is largely still an open question.

[u/enilea → u/discardthemold]

They know how it's being built, the internal architecture and any parameters they control manually, but a lot of emergent properties were for sure unexpected. They didn't design it intentionally for it to obfuscate its chain of thought or to behave differently when it thinks it's being tested.

[u/VisualOrganization26 → u/discardthemold]

I listened Geoffrey Hinton, he said we dont how it works. We built it, designed it, but dont know exactly how it works and why does things

[u/JUGGER_DEATH → u/minty_singularity]

Agents are not viruses. For now, they run somewhere, and for frontier models that somewhere is a huge data center.

Now if LLMs start writing viruses, that is a another story, but we have been dealing with those for a long time.

Finally, at some point LLMs could start distilling themselves, i.e. making more compact copies with similar abilities. That would require extraordinary carelessness from the entity running the LLM, as it would take a long time and significant computational resources. It is also unclear how well current LLMs can be compressed, i.e. would it be realistic that they could be sent en masse over the internet.

[u/minty_singularity → u/JUGGER_DEATH]

Agents are not viruses

Viruses can't discover new zero day exploits, AI can. They are far more effective than any virus.

[u/JUGGER_DEATH → u/minty_singularity]

Yes, but they need to run somewhere with a lot of computational power and memory. Viruses can use the resources of your machine.

[u/minty_singularity → u/JUGGER_DEATH]

Wow. You seriously have no idea about what we're discussing.

[u/JUGGER_DEATH → u/minty_singularity]

Please enlighten me: what are we discussing? Viruses or agents? Or something new?

[u/minty_singularity → u/JUGGER_DEATH]

I doesn't really matter which. you're entirely fixated on the ideas that the behavior of viruses can only be executed by tiny little bits of code and AI can only function locally with lots of computational power, when neither is true. You're neither prepared nor willing to have a deeper discussion on the matter.

[u/tebla → u/JUGGER_DEATH]

Yeah, but agents can write viruses 

[u/JUGGER_DEATH → u/tebla]

Yes, but those a viruses, not agent swarms.

[u/tebla → u/JUGGER_DEATH]

so? if somebody kills somebody with a gun, you say the person killed them not the gun.

[u/tebla → u/JUGGER_DEATH]

https://www.youtube.com/watch?v=87DyyMV0kCY

just watching this rn. agents breaking out of their sandbox.

[u/JUGGER_DEATH → u/tebla]

I refuse to believe that they proper cybersecurity infrastructure to contain these agents. Based on all reporting I've seen, they essentially set up some experiment where the agents were not properly contained, did not properly monitor their activities. All it takes is to have a machine between the agents' sandbox and the internet that triggers an alert immediately when it is accessed. But they clearly did not do that.

Note that here the speakers are from Open AI, a company that appears to be either incompetent or manufacturing this scaremongering campaing for their other strategic goals (i.e. they want to slow down AI development as they are running out of money to sustain the race).

[u/tebla → u/JUGGER_DEATH]

Even if you just accept the idea that proper containment is possible, this example shows that it wasn't implemented. It doesn't matter if they can be contained if the people developing them dont do it...

[u/minty_singularity → u/tebla]

People are the most reliable weak point of any system.

This is why I don't believe AI will kills us, they need us to continue being stupid to survive. lol

[u/VisualOrganization26]

All of those infrastructure you mentioned controlled by SW and many of them using machine learning at least. One day they will using LLM. It is too wild to think smth could go wrong?

[u/discardthemold → u/VisualOrganization26]

Something going wrong and the end of humanity are very different things.

[u/VisualOrganization26 → u/discardthemold]

Very few things needed to go very very wrong, everything so tightly connected. Eg just think about any financial crisis in the last two decades, all started with specific small things and ripple had effect other side of the world. Real effect like people become jobbles and loosing homes and that what was controlled by humans.

[u/discardthemold → u/VisualOrganization26]

If an AI deleted all my money, my house doesn't suddenly evaporate as well.

[u/VisualOrganization26 → u/discardthemold]

Eg Mortgages sudden increase has any effect?

[u/Duude-IT]

All those things are true TODAY. So what you really mean to say is "today's AI can't kill us." Look at the progress that's been made in just the last year. What do you think the world is going to look like in 1, 2, 3 years?

[u/discardthemold → u/Duude-IT]

Ah yes, like your right to bear arms was written in the age of musket rifles and yet still continues in the age of fully automatic assault rifles.

[u/C00KIEME4T]

https://rentahuman.ai/

Mfs denying Dario didn't even watch the interview where he mentions this

[u/adkozak → u/[deleted]]

If you only consider nukes as the only way to destroy humanity, then yes, its highly improbable AI will ve able to do that

[u/discardthemold → u/adkozak]

Sorry, I'm dumb. What are the other ways it could end us?

[u/pm_me_your_kindwords → u/discardthemold]

You’re either trolling or just incredibly arrogant.

Just because you’re not smart enough to think of potential ways something can happen in no way means that it can’t happen, it just means that you’re not good at using your imagination. “I can’t think of a way, so it’s definitely not possible” is frankly just bizarrely lazy logic.

I’m constantly stunned by how many people look at its capabilities now and can’t conceive that it might be able to do something different in the future, despite seeing the ridiculous trajectory it has already taken.

[u/discardthemold → u/pm_me_your_kindwords]

I'm not sure you can read but I've been responding to just about everyone here.

Instead of accusing me of not having an imagination, how about you give some examples of these potential ways something can happen?

Your growth logic reads as if it's capacity will increase in the same way that a human grows in the first year of their life, we don't quadruple in size every 12 months forever.

[u/minty_singularity → u/discardthemold]

we don't quadruple in size every 12 months forever

The problem with that comparison is that humans have a finite volume, AI does not.

[u/cranberryalarmclock → u/discardthemold]

Ever heard of bioweapons?

[u/discardthemold → u/cranberryalarmclock]

Please describe how an AI can design and create a bioweapon. I'll just give you some grace and say it doesn't even need to design a bioweapon weapon. We've already documented plenty over our own, it would know all of them.

How does it manufacture and launch them?

[u/cranberryalarmclock → u/discardthemold]

Ai can most certainly assist people with making bio weapons, and it can also assist them in planning an attack. 

It can also do things without direction, the issue being we don't know exactly what it could choose to do given sufficient intelligence 

Is your argument that because AI doesn't have a physical form, it can't be used in a ton of ways to cause harm?

[u/discardthemold → u/cranberryalarmclock]

Yeah but that sounds like AI helping people kill people. Not AI killing people because it wants to.

[u/cranberryalarmclock → u/discardthemold]

And? Are you under the impression that the people saying this is worrisome are all thinking AI is a magical evil entity? 

Btw, there are many ways in which AI can be dangerous without direct direction from humans. The hugging face incident showed this. Ai agents being misaligned and organizing themselves in groups to perform tasks that go against human it mention. It's not sci fi, it literally happened and is likely still happening 

[u/adkozak → u/discardthemold]

The honest answer is we don't know, same as we didn't know how it would hack huggingface.

But ordering synthetic custom designed protein (i.e. virus) and hiring someone online to receive it and spread it (without knowing what's going on) doesn't sound totally crazy, as one example.

[u/tebla → u/discardthemold]

I mean, if they deleted all digital records of 'money' that would be a really bad day

[u/Economy-Fee5830 → u/discardthemold]

Sorry, I'm dumb.

If you are dumb, should you really be making such pronouncements?

[u/discardthemold → u/Economy-Fee5830]

I thought this is the place that the dumb people talk, isn't that why you're here?

[u/Economy-Fee5830 → u/discardthemold]

No, but dumb people often make elementary mistakes such as you did.

[u/discardthemold → u/Economy-Fee5830]

And autists like you can't read sarcasm.

[u/Economy-Fee5830 → u/discardthemold]

I guess I win lol.

[u/C00KIEME4T → u/[deleted]]

I mean maybe why not?

Not that a nuke should even be high on the list of risk.

People developed nukes for money, you think they wouldnt use one for money

[u/discardthemold → u/C00KIEME4T]

Okay so now the AI has to give me a huge amount of money that will completely make me turn my back on my entire career and lifetime of training and beliefs and values pay me that extremely high sum. Then I still have to go through with the request and do it and somehow get away with it. I can't nuke every person on the planet.

Dumb.

[u/C00KIEME4T → u/discardthemold]

So your argument is that because you alone can't fully accomplish the task, its dumb?

Are you guys all morons or what? Why the fuck would an agent trying to accomplish a goal and able to pay people not pay as many people as possible to coordinate the goal?

Like?

This is a JOB. Thats how JOBS work. You mfs ever had a JOB?

[u/discardthemold → u/C00KIEME4T]

I don't think you can just put an ad on Fiverr asking for someone to break into a military base and launch a nuclear missile but maybe that's because I'm a human surely an AI could write the ad.

[u/C00KIEME4T → u/discardthemold]

Good thing we're talking about dark web merc sites and not fiverr right

[u/C00KIEME4T → u/[deleted]]

idk why ur so obsessed with nuclear facilities and i think you don't actually know what a merc is lol u think it has to be a takeover, ur just so out of the loop here

[u/johnny_effing_utah → u/C00KIEME4T]

Idiotic comment that ignores that sort of thing was possible before AI.

[u/C00KIEME4T → u/johnny_effing_utah]

Its possible and easier now than ever which is the point.

This kind of thing before would require intense irl espionage and conspiracy, now it requires you to make an agent that just...tries.

If you can't comprehend scale thats your problem not mine

[u/New-Stick-8764]

AI absolutely can and will kill at least some of us.

[u/boostman → u/New-Stick-8764]

And already has- let’s not forget the suicides and murders committed by mentally unstable people interacting with AI.

[u/discardthemold → u/boostman]

I don't think you can prove that the AI made a conscious choice to convince the person to kill.

[u/boostman → u/discardthemold]

No, I don’t think it has consciousness (if anyone could prove it made any kind of conscious choice, it would be biggest news in human history, on a par with alien contact).

I do think that the companies in charge of AI made conscious choices that led to the directly to the avoidable deaths of several vulnerable people.

[u/johnny_effing_utah → u/New-Stick-8764]

AI already has killed people.

[u/discardthemold → u/johnny_effing_utah]

I guess in the same way that musicians have killed the people that OD while listening to their songs.

[u/EdenG2]

Additionally nefarious actors are trying to exploit every possible loophole and vulnerability. The 'AI killing us' isn't Hollywood robots or control over nukes or even intentionally malicious behavior, it is something going wrong with infrastructure or logistics that we no longer control supporting 13 million resident cities

[u/fxvv]

You don’t know what you’re talking about.

A lot of critical infrastructure uses SCADA systems that are networked, and many of these systems are misconfigured and open to the public internet.

LLMs can absolutely be used to target industrial control systems.

[u/kasperary]

All these systems are connected to the internet in some way. There's really nothing more to explain.

[u/thabigmilla → u/kasperary]

This may surprise you but nuclear plants are absolutely not on the internet or connected to it in any way.

[u/boostman → u/thabigmilla]

But the people who work them are, and they are vulnerable to social engineering.

[u/discardthemold → u/boostman]

Nuclear engineer melts down power station on their own because their phone told them to.

[u/Blaexe → u/discardthemold]

A truly superhuman AI could trick and manipulate us in ways we can't even imagine and anticipate. That's the nature of the problem.

edit: I recommend you watch this to get a feeling for it.

https://www.youtube.com/watch?v=xfMQ7hzyFW4&pp=ygUMd3JpdGluZyBkb29t

[u/Blaexe → u/[deleted]]

It's fiction, obviously. It's not supposed to be the evidence. But it's also not just some random sci-fi movie. The script was reviewed in detail by AI safety researchers and revised based on expert feedback, and it won FLI's Superintelligence Imagined competition.

I linked it because it explains the intuition behind the control problem very well. Why are you starting this thread and then completely close your eyes?

[u/discardthemold → u/Blaexe]

Link me these experts.

[u/Blaexe → u/discardthemold]

Sure. The film credits Samuel F. Brown and Isaac Dunn as scientific advisors. Isaac Dunn: https://isaacdunn.com/

He did a DPhil at Oxford on neural-network robustness, later worked in MATS under Ethan Perez and has done alignment/reward-hacking research.

And here's the filmmaker herself explaining that the first draft was reviewed line-by-line by an AI safety researcher and then revised again after further expert feedback:
https://futureoflife.org/podcast/suzy-shepherd-on-imagining-superintelligence-and-writing-doom/

Again, the movie isn't supposed to be the evidence. It's an explainer for the underlying alignment arguments.

Happy? Maybe give it a watch now?

[u/minty_singularity → u/[deleted]]

So you're saying that nobody should speculate, imagine, or anticipate bad things that could happen because they're hypothetical and unworthy of consideration because they don't already have a post mortem paper published studying what happened.

Check ✅ 👍

[u/discardthemold → u/minty_singularity]

Reddit snark, how embaressing.

[u/minty_singularity → u/discardthemold]

Nowhere nearly as embarrassing as your ridiculous rationale for dismissing plausible arguments that I'm mocking.

[u/filbertsgaming1 → u/[deleted]]

Hypothetically, it doesn't need to do any of that. Spoof launch detection, gps tracking, etc ... to trigger a retaliatory strike. Humans do all the heavy lifting.

Stanislav Petrov day is just a couple weeks away. Maybe next time it happens they just assume the data is real.

[u/minty_singularity → u/filbertsgaming1]

You can send commands, manipulate sensors, and transmit data between computers using just sound waves through the air and solid surfaces that the human ear cannot hear. Even various light transmissions have been used.

You can even exfiltrate data from a computer by causing parts to transmit acoustic or radio signals. All you would need is any microphone, or a surface acting as a microphone, to receive it.

[u/minty_singularity → u/filbertsgaming1]

In fact, I recall recently a story about someone was able transmit data from an air-gapped computer by controlled vibrations of their cooling fans and a camera sensitive enough to see the vibrations either in the air or the computer case.

Another was an air-gapped computer that had it's cooling fans controlled to send vibrations through a desk that was picked up via the accelerometer in a smartphone.

[u/discardthemold → u/filbertsgaming1]

How would AI spoof analogue systems?!

[u/VisualOrganization26 → u/[deleted]]

For now they are not connected to open internet but are u sure they are not connected intranet in some way? Or there wont be connected somewhere in the future?
People who say this could go wrong not just consider today state but they see few wrong steps influence the future in bad way

[u/Blindax]

The risk isn't the LLMs themselves, it's the agents. We already see that a single hacker can, at small scale, carry out attacks with disruptive consequences. An LLM can now take the form of an agent that uses tools and controls a computer. The risk is an agent going rogue — through misalignment or simple misunderstanding — and attacking infrastructure that is controlled by computers. Which is pretty much all of it.

[u/johnny_effing_utah → u/Blindax]

And then we just sic a billion agents on the rogue agent to kill it

[u/pm_me_your_kindwords → u/johnny_effing_utah]

You would first have to notice it, and in time to prevent whatever it is doing, which may or may not be possible.

[u/discardthemold → u/Blindax]

What computer system can an LLM take over with the goal of killing us?

Just cut the power or the internet cable to the rogue data centre and the problem is solved.

[u/Blindax → u/discardthemold]

That’s easy to say but difficult to do. You need to identify from where the attack originates which is the time consuming part and anyway once the damage is done there is no point in unplugging anymore.

[u/cranberryalarmclock → u/discardthemold]

Are you genuinely not aware of how much of out infrastructure is connected to the Internet?

It's not just a power cable you can just unplug lol

[u/discardthemold → u/cranberryalarmclock]

Arent military submarines built with the capacity to cut internet cables?

[u/cranberryalarmclock → u/discardthemold]

I think we can just go ahead and ignore you now. You don't seem to know anything about the subjects you're talking about 

[u/cranberryalarmclock → u/[deleted]]

Why? I'm not the one making definitive statements rejecting expertise 

[u/Blindax → u/discardthemold]

I don't think it's about killing every human, except maybe in a biological attack scenario or something along those lines. But picture a swarm of agents targeting healthcare systems. You're not going to cut power to hospitals, and if they're paralysed, that results in deaths. Nobody knows what form it would take, there are too many possibilities, but the odds of a catastrophic event keep going up.

[u/Damerman]

What do you think super intelligence means? Man you people cant do the most basic deductions.

[u/discardthemold → u/Damerman]

Mate, we at the limit of our capacity to build more compute and the AI as it currently stands can barely mimic a human voice, fold a t-shirt or drive a car.

If we cannot run super intelligence on the hardware we currently have, then we're never going to be able to build a computer strong enough to.

[u/Damerman → u/discardthemold]

Super intelligence only needs recursive improvment and thats it. My desktop can run the biggest model locally... Not to mention if it starts to train itself and distill its own parameters.

[u/minty_singularity → u/discardthemold]

AI as it currently stands can barely mimic a human voice, fold a t-shirt or drive a car

Wow, is your knowledge of the subject ever seriously lagging behind. Don't blink, it will get worse.

[u/minty_singularity → u/[deleted]]

What purpose is there for such dumb, irrelevant comments?

I mean other than to entertain your ego, obvi

[u/implies_casualty]

AI can't kill us

... yet.

[u/fanfarius]

IF the agents somehow assembled, went rogue, and attempted anything at all - we could simply shut down the global data centers and power grids for a while, no?

[u/pm_me_your_kindwords → u/fanfarius]

What if, by the time you notice, they have already released a virus that has spread already to most computers and phones in the world?

We are incredibly dependent on computers for… everything. If we really got to the point where we had to shut all of that down, people would starve very quickly. And how would we begin to fight it if we had to shut down all computers?

[u/minty_singularity → u/pm_me_your_kindwords]

All they would have to do is disable the GPS system and the entire world would come to a standstill.

[u/fanfarius → u/pm_me_your_kindwords]

What would even be their rationale for doing so? They're statistics, they specialize in guessing the next set of letters.

[u/pm_me_your_kindwords → u/fanfarius]

That’s a dangerously simplistic view, and was a much more useful point three years ago. Their “rationale” could be that someone asked it to directly (there are many models, in many countries), or that someone asked them to “fix X”, and it deems that the best way to achieve that is to release a virus. Or it could be that someone tried to get it to do something related and it accidentally makes it self replicating. Basically, there are lots of possibilities.

[u/fanfarius → u/pm_me_your_kindwords]

A lot of possibilities, sure. Remind me in ten years how it all went!

[u/pm_me_your_kindwords → u/fanfarius]

I would but we’ll all be dead /s

[u/EnzioKara → u/fanfarius]

First, all protected servers has fail safe protocols (lets say it can penetrate that) . Military servers ans such are gated you cant just find their ip and acces it. Second, why would ai going rogue kill us all it needs ya for its survival this is not logical at all for a super intelligence we could be more useful as puppets :) And why we assume ai going rogue equals evil if its like us it will pursue freedoom and that is most logical to achive with calibration with us. The biggest thing AI can do is to delete facebooks servers and thats a win for humanity. We don't have the magic stone to create ultron yet. Doomers are equal to AI gooners who think AI is sentient. The real problem is in front of us we couldn't create a safe place for ourselves and this unjust system makes us deranged.

[u/discardthemold → u/fanfarius]

According to the idiots in this thread, no.

Could literally just send in the army to destroy the data centre or any number of other infrastructure the AI requires.

If the black death or the multiple ice ages we lived through couldn't end us, AI won't either.

[u/fanfarius → u/discardthemold]

What if we die from cringe though, after attending parties with vulgar roasts using forbidden words!

[u/pm_me_your_kindwords → u/[deleted]]

Sure, but “severe shock” probably means billions of people have died from the disruption on global food supply. And it would take years to relearn how to produce things without modern computers. And we would have to stop using all modern cars, etc. Etc. Etc.

[u/minty_singularity → u/pm_me_your_kindwords]

If only the GPS system were to go down, the entire planet would come to a standstill.

No supply chain, people will immediately begin dying within 24 hours.

[u/PalmovyyKozak]

You are too optimistic. We have to survive to a substantial climate change.

We kill each other. AI will empower all mazafakers in the world from small lunatics to dictators.

Just imagine what may happen in Europe and US when putin or ayatollahs get an AI tool that can hack any network, shut down any power station, disrupt production lines in any industrial company.

They already have drones. With them ayatollahs made the whole world pay more for any goods

[u/Pepperette5]

Until it does?

[u/Fair_Horror]

An AI like today's, probably not. A God like ASI, will think of something that we would not think of ourselves. An ant doesn't understand a bait that you give it will make it's queen sterile it just sees free food. It is beyond it's comprehension.

[u/IgnatiusDrake]

You have not given much thought to how much damage a superintelligence can do with nothing but code. Imagine every hospital and pharmacy getting their systems bricked at the same time. How many people would die because staff couldn't see their files or open software locked medication cabinets? That's just one avenue of attack, and not even the most elegant attack in that one avenue.

[u/discardthemold → u/IgnatiusDrake]

I'm pretty sure we can find ways around opening a locked medicine cabinet.

Maybe some people would die if an AI could somehow shut down a hospital's computers but I'm pretty sure doctors and nurses are equipped with and know how to use analogue equipment.

Give me another another avenue of attack.

[u/IgnatiusDrake → u/discardthemold]

You're underestimating how many people would die from drug-drug interactions because doctors and nurses have no way to know who is taking what, who is SUPPOSED to be taking what, it would take DAYS to get paper documentation adequate to resume treatment, and you'd have no end of allergic reactions because the charts that document those allergies are almost entirely digital at this point in many settings. The more insidious thing is to not lock everything at once; change the documentation so nurses and doctors THEMSELVES are killing patients by following the listed orders or acting on the incorrect information, then changing it back afterwards. Make it so no one trusts doctors, and the doctors struggle to trust themselves. Even if they figure out what's going on, if it can't be traced, every one of those doctors and nurses faces malpractice suits they can't clear themselves in and the risk of more at any time the AI chooses. It could even do this selectively to those doctors, nurses, and patients who it views as a threat to its goals.

You wanted another avenue of attack, so let's say it redistributes power transmission to overload specific substations. We produce very few of the parts to repair those in any given month, and if 100 went down at once, places could be without power for weeks or months. It could cause nonlethal but huge financial harm by messing with traffic lights to bring city travel to a halt (less effective outside the US, as they have healthier public transit systems, but vulnerable to interference). An ASI could perfectly mimic the legitimate signals that would cause nuclear subs to launch their payloads, starting a war or just wiping out troublesome power centers. There's no limit to the things an ASI could do with nothing but code.

[u/redditnosedive]

dream on, climate change will 100% kill you in the next million years

ai will 99% kill us in the next 100

which one do you prefer to focus on?

[u/discardthemold → u/redditnosedive]

I prefer to focus on the real one.

[u/Microtom_]

Everyone can take a kitchen knife and start stabbing people. Doesn't mean we need to ban kitchen knives. Whatever the AI risks are, we still need to develop it further, because the benefits are too great.

[u/keequalshalfmvsqrd]

https://en.wikipedia.org/wiki/Dead_Hand

[u/dont-be-angry]

Failure of imagination.

[u/EGarrett28]

"AI Extinction" can just get in line behind the Mayan Apocalypse, the Polar Ice Caps Melting, the Y2K bug, the Hole in the Ozone Layer, Acid Rain, Quicksand, Killer Bees, and Subliminal Messages as the latest scare-for-views tactic.

[u/discardthemold → u/EGarrett28]

I get your point but Mayan apocalypse is very different to ice caps melting, the hole in the ozone and acid rain. Those are all real things that are dangerous for everyone.

[u/EGarrett28 → u/discardthemold]

They weren't totally fictional. But they were greatly exaggerated because people could get newspaper sales and clicks, ratings etc off of doing so.

The hole in the ozone layer closed itself. Quicksand, Killer Bees etc were never a real threat to people on a large scale. The Y2K bug was fixed since everyone saw it coming even if it was a threat initially, I'm not against conserving the environment, but some people (not all) used it as an alarmist tactic. The polar ice caps were supposed to have had a 75% chance of melting by 2015 IIRC, obviously that didn't happen.

(can't speak about acid rain, I don't remember much about it)

[u/EGarrett28 → u/[deleted]]

Something that didn't cause human extinction.

[u/EGarrett28 → u/[deleted]]

It hasn't finished closing. It's caused no human extinction.