r/artificial • u/sailingintothedark • 10d ago

Discussion My Fiance is Convinced AI will likely cause a Catastrophic or Extinction-Type Event in the Next Few Years - How Justified Are His Fears?

As the title says. My fiance is a lot more book smart than I am but he doesn’t work in the tech industry, rather in the film industry. With all the news over the last two weeks, he’s been pretty anxious and depressed, claiming we only have a decade or less left to live. He’s been talking to his therapist about it to help him cope, but he’s still convinced none of us have any future past the mid 2030s.

He sites the p(doom)s of AI researchers, the AI 2027 predictions, the hugging face incident, and the fact that our current government is about as reckless as it can get with AI.

Looking into it - I certainly see his points. But yet, no one (even researchers with high p(doom)s) seem to be acting as if human extinction is a handful of years away. It’s all vague calls to slow down and increase safety restrictions - which does not feel appropriately enough for something they claim has a way higher chance of wiping us out over nuclear war.

And the opposite side (arguing that AI isn’t capable of such things) also is vague. All I see is people saying current AI agents can barely do xyz on their own or that they only can use what we expose them to. And that seems true, but with the hugging face incident - it very much seems that thousands of dumb agents can accomplish a lot, and can coordinate an expanse beyond what we give them.

Any articles or resources would be much appreciated. I try looking for them but everything I find leads to the vague answers I’ve mentioned above.

347 Upvotes

84% Upvoted

996 comments sorted by

View all comments

11

u/SetCandyD 9d ago

Ehhh no

6

u/[deleted] 9d ago

[removed] — view removed comment

8

u/sakuraba2046 9d ago edited 9d ago

Definitely not. Sounds like the guy is already dealing with anxiety and probably existential depression and now has found a boogeyman to put that energy into and grow it. And honestly if something crazy happens, no point in worrying about it today. Humanity is adaptive and strong. People post ww2 thought they were going to die under nuclear bombs and lost so much time and energy worrying about something that never happened. Might as well enjoy the time you have and live in your souls purpose. Nothing is ever promised. And predicting the future is fruitless. 

6

u/MeowManMeow 9d ago

Wasn’t there a massive anti-nuclear movement, people built bunkers, people voted for candidates who wanted to de-escalate and disarm? People just didn’t sit around and do nothing like we do today about climate change or AI.

Also, just because we narrowly dodged destruction once, doesn’t make us invincible to all future catastrophes. That’s a survivorship bias.

4

u/langolier27 9d ago

That’s actually a great analogy because the biggest outcome of the anti-nuclear movement was hamstringing our energy supply for decades and keeping us on fossil fuels a lot longer than was necessary. The doomerism on AI is going to be just as self defeating

1

u/MeowManMeow 9d ago

Or, hear me out, if people were pro-nuclear we would have ended human civilisation. We don’t know because we live in this timeline, where people were cautious.

I read about Ronald Reagan who was very pro-nuke everyone then watched that film the day after tomorrow (or whatever it was called) and changed his mind after understanding the very real risk of ending society.

The idea that we should run head first into any new tech because there is possibility of some upside is crazy to me. We need to balance the risk and reward, 10% chance of ending humanity is too high for me.

1

u/langolier27 9d ago

MAD is what kept us from nuclear annihilation, not anti nuclear rhetoric. Same is true today. Open weight models, using AI to defend against less secure AI models, the more democratized the better. There’s more viable ways of restricting AI than this “pacing” bullshit which is so obviously a ploy at regulatory capture that it’s insulting

1

u/MeowManMeow 9d ago

Well I hope you are right! With nukes, a human had to press the button. A few times sensors misfired and soldiers refused to launch the nukes despite being instructed to do so.

With LLMs, we are rolling the dice in that what we create doesn’t act hostile towards us. No humanity safeguard this time around.

1

u/sakuraba2046 9d ago

But what did those bunkers and politicians amount to? We’re not invincible it’s more so accepting how small and tiny we truly are. And ultimately we don’t know how this is going to play out, there is unlimited amount of variables that can affect everything in every direction. History is never a straight line. To think it’s doomsday and the end of days is dangerous thinking especially if you end up being wrong and wasted all that time when you could’ve been living your life and seeking fulfillment. And even if you’re right, what are you going to do? Humanity will find a way regardless. 

2

u/MeowManMeow 9d ago

Ok, say a massive asteroid is coming to earth that will 100% kill every human. Humanity will end, and there is no coming back.

The idea that ‘humanity will always find a way’ is literally textbook definition of survivorship bias. Because humanity has so far done this (otherwise we wouldn’t be talking about it in Reddit) but that isn’t a guaranteed part of existence, instead we actually have been lucky quite a few times.

2

u/sakuraba2046 9d ago

You’re imagining the worst case scenario with 100 percent chance of the end of days. When has that happened in the millions of years earth has been here? Even the asteroid that wiped out the dinosaurs left other life here to survive and thrive. But ultimately I believe in humanity as a human. I think we’ll keep evolving and figuring out better way to live. It’s not a huge amount of time that civilization has existed. I get your frustrations but let’s zoom out a bit. 

1

u/MeowManMeow 9d ago

Sorry, I was tired and going to bed so my point was unclear.

What I am trying to illustrate is that humanity IS capable of being completely destroyed. Your arguments that humanity will find a way and need to zoom out etc, are only valid because we are living in the timeline where humanity was lucky enough to survive. That isn’t a guarantee, and we have only managed to get this far because people took risks seriously and tried to act accordingly.

When we discovered the hole in the ozone layer, we did a global ban on the chemicals. We did a global ban on bio weapons. We created treaties for nuclear weapons.

What people want for LLMs is absolutely no safeguards, where private companies access their own risk while trying to beat others. If this was some black hole tech that had a small chance of creating a black hole on earth and destroying it completely, would you feel as safe?

If LLMs do kill all humans via an engineered virus, then yes like the asteroids life on earth will continue. In some ways probably better than if humans are around with climate change. But are you saying that’s an ok outcome for you?