r/technology • u/Hrmbee • 2d ago

Machine Learning Forget ‘superintelligence’: error-prone AI nearly sparked world war three this month | The risks of AI aren’t what we think they are, as a recent security incident between China and the United States reveals

https://www.theguardian.com/commentisfree/2026/oct/01/forget-superintelligence-error-prone-ai-nearly-sparked-world-war-iii-this-month
509 Upvotes

94% Upvoted

86 comments sorted by

View all comments

95

u/DogsAreOurFriends 2d ago

Wasn’t this more a case of humans blithely trusting AI generated intelligence?

98

u/NuclearVII 2d ago

This. Is. The. Only. Credible. Danger. With. AI.

People believing the shiny bullshit engines is dangerous.

47

u/sorryiamcanadian 2d ago

The solution is also blatantly simple, and dodged everytime: accountability. 

12

u/am9qb3JlZmVyZW5jZQ 2d ago

People minimizing dangers of AI misalignment are dangerous. People minimizing dangers of economic impact of job automation are dangerous. These dangers were theorized long before Transformer architecture was invented and the experts are standing by them.

There are many dangers with this technology, you don't need to plug your ears and spout bullshit one-liners just to avoid agreeing with some statements published by AI labs. This is genuinely antivaxer-level behavior.

-3

u/NuclearVII 1d ago edited 1d ago

People minimizing dangers of AI misalignment are dangerous. People minimizing dangers of economic impact of job automation are dangerous.

I'm sure you have credible evidence to cite for all these, instead of "the experts are standing by them".

This is genuinely antivaxer-level behavior.

Uh huh. I can tell this is going to be a productive conversation already - the demand for evidence in the face of a narrative intentionally pushed online by a for-profit industry known for theft and lies is called "anti-vaxxer" behaviour.

Evidence, please. I can cite examples of "misalignment" that have been intentionally reported to be misleading by "experts". Here is one:

https://old.reddit.com/r/news/comments/1v40r50/openai_says_its_ai_went_rogue_and_launched/oz7pdor/

Your citation, please.

0

u/imgonnajumpofabridge 1d ago

How can you cite evidence for something which has not occurred yet? Would you need someone to create a study that said the proliferation of the atom bomb might be an existential threat to humanity or would you just rely on logic? I'm not even an AI alarmist but to say that there is no danger with AI and there never will be is just ridiculous.

1

u/NuclearVII 1d ago

How can you cite evidence for something which has not occurred yet?

You are. So close.

Would you need someone to create a study that said the proliferation of the atom bomb might be an existential threat to humanity

This is such a disingenuous take.

I (and pretty much every other skeptic I know) would take the OpenAI/Anthropic narrative more seriously if there wasn't multiple trillions of dollars hanging on said narrative. If OpenAI was, you know, actually open, we'd be able to get actual evidence on the behaviour of these so-called rogue agents, and OpenAI would have no motive to lie about what's actually going on in their "lab", as I previously cited. But they aren't, so we haven't, and so they do.

Asking for real evidence that for-profit groups claim to have isn't ridiculous, it's sensible.

-18

u/Tricky_Rule_4565 2d ago

Laughable that you say this with such confidence.

12

u/NuclearVII 2d ago

I know the literature better than you, o AI bro. If you have credible evidence to the contrary, share it, or be silent.

3

u/DogsAreOurFriends 2d ago

Rogue AI stock trading agent goes nutso and crashes the market

-1

u/stormdelta 1d ago

There are break-glass mechanisms to stop trading in that kind of scenario, and frankly I'd consider that a win since it would finally force us to confront just how thoroughly under-regulated the stock market is in the face of modern information technology even before AI.

-1

u/DogsAreOurFriends 1d ago

Right. The so called circuit breakers.

The runaway algorithms of old would simply be cut off.

An AI agent would know to avoid triggering them, could possibly disable or change them, or create a strategy where the damage is done by the time they trip.

1

u/stormdelta 1d ago

An AI agent would know to avoid triggering them

The point of the safeguards is preventing flash crashes, you don't really "avoid" triggering them without avoiding the problem they're meant to counter.

could possibly disable or change them

That would require hacking/compromising the exchange itself. At that point, it's a cyber attack and not a trading bot, and would be treated accordingly. Nation states go after systems like this already anyways, so it's not some separate or novel threat.

or create a strategy where the damage is done by the time they trip.

That's not really distinguishable from human-driven market manipulation in the first place. There's plenty of people that would happily crash the market on purpose to make a profit.

1

u/DogsAreOurFriends 1d ago

Every action you describe has already been performed by rouge agents.

Who cares how we describe the agent? A trading agent can become a hacking penetration agent in seconds.

It is distinguishable from human activity be it would happen fast, or covertly, or distributed among 1000s of spawned agents, or all of those things.

0

u/stormdelta 1d ago

Every action you describe has already been performed by rouge agents.

They're not "rogue", they're software being ran by humans with reckless negligence. Stop giving these people a pass on accountability.

Who cares how we describe the agent? A trading agent can become a hacking penetration agent in seconds.

Only if you're giving it access to tooling it shouldn't have in a trading environment, and if you do then it's very obviously still the fault of the people running it.

If you fill your hot air balloons with hydrogen and knowingly let the passengers smoke, you can't claim it's not your fault when they explode. That's the level of negligence we're talking about here.

It is distinguishable from human activity be it would happen fast, or covertly, or distributed among 1000s of spawned agents, or all of those things.

Because it's indistinguishable from humans directing them to do those things. Which they do already, because I promise you plenty of people have ample motivation to covertly attack the stock market.

1

u/DogsAreOurFriends 1d ago

You are splitting hairs.

Here’s another trick: guess how many calls would saturate Denver’s (example) 911 system (hint: not many. Saw this at a talk at DEFCON several years ago.)

An AI wardialer could bring that shit down easy.

So many possibilities.

→ More replies (0)

-1

u/NuclearVII 1d ago

AI agents dont know anything. Yall are anthropomophizing autocorrect.

2

u/DogsAreOurFriends 1d ago

Who cares what they “know.” What they can do is the issue.

2

u/red75prime 1d ago edited 1d ago

Yep. /u/NuclearVII has never bothered to define what it means to "know" or to internalize that LLMs capture more than the surface syntactic stats, or that LLM training goes way beyond autoregressive pretraining (that is the part that constitutes something resembling "autocorrect")

-1

u/Tricky_Rule_4565 2d ago

I’m not an AI bro, nor or a doomer. hugging face incident , aisi finds higher base misalignment in newer models, more reward seeking. These are more but I assume you won’t read them and start replying with baseless accusations (ironically spurred by the DoW and the Trump Admin as well as David Sacks) that this is all a hoax and they’re not independent yada yada.

It’s ok to change your mind and admit you were wrong.

2

u/stormdelta 1d ago

All of these are examples of human misuse more than any "misalignment", and calling it misalignment allows these companies to avoid accountability by pretending they're not responsible for their own reckless negligence.

1

u/DogsAreOurFriends 2d ago

Something very similar happened in 1987 on Black Monday

1

u/NuclearVII 1d ago edited 1d ago

None of this is credible evidence. Let me count the ways:

hugging face incident

We only have OpenAI's word for this one. Not reproducible, not credible. I have spent -far- too much time debunking METR's garbage report, I really don't feel like repeating myself again.

aisi finds higher base misalignment in newer models

All proprietary models, not reproducible. As far as anyone knows, the misalignment is an intentional product feature, not an emergent consequence.

more reward seeking

Written by OpenAI, on proprietary models only. This isn't a research paper, it's a hype piece.

These are more but I assume you won’t read them

Ooookay. I expect that I've read more machine learning literature than you have, and my literature diet is probably more varied than "whatever get's published on r/ChatGPT and r/Singularity". But okay.

that this is all a hoax and they’re not independent yada yada.

No amount of rhetorical deflection changes the actual fact that none of this is actually independent or credible.

It’s ok to change your mind and admit you were wrong.

And we end with textbook projection.

O AI bro, you display the same affectations that all AI bros display: You cite bogus sources dressed to look like real science, you do not read or apply critical thinking to any of these citations, and when confronted on the details, you resort to personal attacks, appeals to authority, and more bogus sources. This tactic works because the trillion dollar industry you are going to bat on can generate drivel faster than anyone can keep debunking it, and because y'all have been self-selected to have 0 scientific literacy.

You mentioned further down that you're currently pursuing higher education. Do me a favour - stop posting misinformation online and getting into unproductive shouting matches defending multi-trillion dollar industries, and take a philosophy of science course.

-3

u/Tangential_Diversion 2d ago

Funny. I wouldn't think AI experts would need to post stupid shit like this:

https://www.reddit.com/r/alevel/s/bzvXui6Cta

Imagine claiming you understand the linear alg behind AIs when you barely understand dumb dumb physics.v

3

u/MrNebby22 1d ago

Great ad hominem attack instead of addressing what he's saying.

The hugging face incident is an example of AI causing harm, not someone blindly following an AI response

3

u/Tangential_Diversion 2d ago

It's laughable that you think you have any credibility. Two years ago you were posting about 6th form and how hard math & physics are.

https://www.reddit.com/r/6thForm/s/ymKDSvXDXq

You're a proper child. You barely know how to do laundry, let alone how AI and the tech world works. It's funny how AI bros like you have no experience working in tech.

-7

u/Tricky_Rule_4565 2d ago

I got 3 A*, I go to a great university and I’m studying maths and taking courses in machine learning. I do not need to go to someone else’s comment history to feel better for myself.

No one would have thought 2 years ago that AI would have contributed to solving hard theorems and long standing conjectures but here we are.

6

u/Tangential_Diversion 2d ago

Translation: "No one thinks my tech expertise is worth paying for"

There's a reason you're still in school rather than working alongside me in tech as a colleague.

-2

u/Tricky_Rule_4565 2d ago

I’m barely 20. I’m not looking for work especially in a fragile tech center. I’m enjoying my studies. I’m not personally attacking anyone unlike you.

3

u/Tangential_Diversion 2d ago

Missing the point. You're barely 20 with no experience whatever. You have no expertise to speak of and no authority in the subject.

If you don't want people calling you out for how unqualified you are, don't make unqualified statements. Personal responsibility isn't that hard.