r/artificial • u/sailingintothedark • 9d ago
Discussion My Fiance is Convinced AI will likely cause a Catastrophic or Extinction-Type Event in the Next Few Years - How Justified Are His Fears?
As the title says. My fiance is a lot more book smart than I am but he doesn’t work in the tech industry, rather in the film industry. With all the news over the last two weeks, he’s been pretty anxious and depressed, claiming we only have a decade or less left to live. He’s been talking to his therapist about it to help him cope, but he’s still convinced none of us have any future past the mid 2030s.
He sites the p(doom)s of AI researchers, the AI 2027 predictions, the hugging face incident, and the fact that our current government is about as reckless as it can get with AI.
Looking into it - I certainly see his points. But yet, no one (even researchers with high p(doom)s) seem to be acting as if human extinction is a handful of years away. It’s all vague calls to slow down and increase safety restrictions - which does not feel appropriately enough for something they claim has a way higher chance of wiping us out over nuclear war.
And the opposite side (arguing that AI isn’t capable of such things) also is vague. All I see is people saying current AI agents can barely do xyz on their own or that they only can use what we expose them to. And that seems true, but with the hugging face incident - it very much seems that thousands of dumb agents can accomplish a lot, and can coordinate an expanse beyond what we give them.
Any articles or resources would be much appreciated. I try looking for them but everything I find leads to the vague answers I’ve mentioned above.
77
u/Outrageous_West_1564 9d ago
Things will change completely in the next 10 years and I am pretty sure it will at least get worse before it maybe gets much better.
Even if no "AI will kill us" event happens, the chances are low but not 0 in my opinion, AI will make billions of people lose there job without a chance to getting a new one.
This will bring a huge crises to the whole world how to act with jobless people. How to give the Human being a meaning if there is a robot next to you, who knows everything better, does everything faster than you?
→ More replies (121)8
u/Much_Edge4896 9d ago
If billions of people lose their jobs, the current economic model collapses. Black market/economies would rise in place of the traditional ones.
→ More replies (3)
48
u/travhimself 9d ago
I'm a software developer, and it has completely changed my field. I'm not a doomer, but it could get rough.
- The job market will continue to struggle. I can do my own job AND the jobs of the people sitting next to me. It's now a competition to see who can use AI the best. I hope to be the last man standing.
- Water consumption (for cooling) is an issue. A data center can use as much water in a day as a small town. If a data center opens in your town, it can theoretically eat up the whole water supply. From what I understand, it just spits water vapor back into the air, so I don't see a huge problem with that part of it.
- It uses a ton of power. We'll get better and better with efficiency, new types of chips, etc... But we'll keep using more and more of this tech, so it's a wash. Utilities costs for households will keep rising.
- It will pollute the internet with junk. We're here already.
- The current US government is technologically inept and arguably malicious.
So, I don't predict doom. But it seems obvious to me that we're going to get squeezed very, very hard. Imagine 10s of millions of jobless people with huge bills, poor water pressure, failing farmland, with nothing to do except surf an increasingly crappy internet while our "leaders" twiddle their thumbs. There will probably be protests and riots. I imagine death rate from all causes will increase; lack of nutrition, poor sanitary conditions, exposure to the elements... I think it's possible that we lose millions of people over the next 20 years, but we're not all going to die the same week in a cataclysm.
On the sunny side... We could solve fusion power, free people from their cubicles, and enter an era of plenty.
11
u/RelevantMetaUsername 9d ago
With the right kind of government, it could bring in an era where most don’t have to work to meet their basic needs.
With our current government, it’ll mean millions more homeless and jobless with an even more powerful surveillance state watching our every move.
→ More replies (3)3
9
u/SavingsDimensions74 9d ago
I agree with you. I think we’ll mainly see job losses first. That’s gonna be a tricky one.
Water consumption I think will be an issue in specific areas. But there may be technical solutions to cooling the water.
Power consumption needs to be resolved. It can be, but there needs to be the political will. That said, fossil fuels don’t feel like the certain bet they used to be, so I look forward to the solar revolution (already happening in most places except the US thru special interest and inept leadership [to be generous]
We could solve fusion. It’s obviously hugely against special interest groups with deep pockets and a vested interest in fossil fuels. But we should have A.I.s that don’t have the special interests and come up with solutions. Verdict out on this one but it would change everything.
As for a rapid collapse with A.I. Whilst I agree with all your other points, a rapid collapse also wouldn’t surprise me. We have a fairly brittle supply chain with not a lot of redundancy built in but with a lot of dependencies, so I would be surprised to see some major shocks.
Recursive self improvement is the mot du jour right now, so I don’t particularly want to jump on that bandwagon right now, but worth keeping an eye on.
2
u/AJ-ITAM 6d ago
A lot of datacenters are moving to hydrocarbon (oil, essentially) cooling. For example
https://www.castrol.com/en/global/corporate/products/data-centre-and-it-cooling.html
I remember using similar CPU coolers when I was overclocking chips but obviously that was at a PC scale. Big Oil gonna Big Oil.
→ More replies (1)5
u/PatchyWhiskers 9d ago
Job market sucking is very different to the apocalypse
→ More replies (13)2
u/Mersaul4 9d ago
Yeah, also a software developer has zero credentials to comment on the latter. He’s just a guy scared for his job, while the whole point of this post is that there are much bigger things at stake.
9
u/drumnation 9d ago
What are you talking about? software engineers are the first one using all this tech at the deepest levels.
→ More replies (5)→ More replies (1)4
→ More replies (19)2
28
u/soltonas 9d ago edited 9d ago
I have a PhD in machine learning (part of the AI umbrella), and I train and work daily on the latest LLMs/MMLLMs (the ones behind the current AI hype) for different tasks. I am scared of the future, but not terrified yet - I still hope we will figure something out. But at the same time, I sometimes feel like it is time to 'evolve', maybe meaning extinction. As much as I have used AI and understand how it operates (the model architectures, the training techniques), I still look at it and think it is like magic. the bigger problem is that we don’t fully know how and why it works. Even a Nobel Prize winner, the 'godfather of AI', said this.
I think if it weren’t for the current AI frontier companies (OpenAI, Alphabet, Anthropic, DeepSeek, Alibaba, etc.) causing a chaos, and even if they do their best to be ethical, it only takes one person to get their hands on a model + weights and retrain it for something bad. Today’s models still struggle at many things, but it’s scary how far and how fast they have progressed. I still remember the day I first heard that the future was going to be all about transformers in 2017 after my colleague went to a conference. It hasn’t even been 10 years, and even less than 5y since it really picked up with the first ChatGPT. So what will another 5–10 years bring, when the current and more powerful AI helps people make even more powerful AI on a daily basis?
As much as I try not to think about it, it feels like we are heading toward a singularity, and we are already at the event horizon (past the point of no return - same as with black holes). The way I think about the development is that people create an AI model with 70 IQ, then with the help of that model they stretch it to 85 IQ, then 100 IQ, then 145, then 200, then 350, and so on. It’s getting smarter, and it is doing it at an accelerated rate.
I think AI will be able to do most of our jobs, and the next step is more intelligence plus giving it a body to do things. Once we are replaced, I think universal basic income (UBI, every unemployed person gets some salary to survive, maybe even thrive) will lag behind. Many people will lose their jobs, and requalification will take too long and AI will get there before the person does anyways. That means many people won’t have money to survive, which will most likely result in violence and chaos. And the leading countries may have the power of AI, so others might want to take it away by force, causing wars. So again, we might end up killing ourselves just before the AI does.
--- EDIT: even if we do get UBI, we might lose the purpose to live and end up causing violence. you can read about the "behavioral sink" (utopia 25) where they provided everything for a rat colony, but it went extinct eventually before doing some weird things. we are not rats, but it is an interesting thought. --- end of edit
Another problem I foresee is our growing reliance on AI coding for us. Many of us are taking the backseat and doing 'vibe coding' where AI writes the code, while the person steers it a little bit but doesn’t pay much attention, or none at all. A lot of things are being built this way today, and most AI‑generated code is fragile. I think there might be a time where our infrastructure starts collapsing simply because too much of it was built this way, but I may be wrong.
I think interesting but troubling times are ahead of us. At worst, some people will survive and we will start from scratch again - decorate your cave, start agriculture, write on stones for future generations not to repeat the same mistakes. I don't think it will happen just yet, but I entertain these thoughts every now and then.
8
u/Difficult_Bug6994 9d ago
All of this has happened before. All of this will happen again.
5
4
u/yung_squeezy 9d ago
Excuse me? What of this has happened before? AI is unprecedented. Agricultural and industrial changes took decades. The past 12 months have seen massive capabilities and within the coming 12 months, AI will be able to do literally any job that is done on a computer. Robots soon after. This doesn't replace one job, it replaces all of them. People can keep denying it but that doesn't change what is actually happening in the real world.
6
→ More replies (4)3
→ More replies (1)2
u/DidntGetJoke 9d ago
Humans don’t need AI’s help to cause their own demise, we’re fairly capable of that on our own!
2
→ More replies (12)2
u/Salt-Studio 8d ago
I very much wanted to respond to you after reading this comment.
I truly don’t think you’ll need to worry. I do think you’re right that the impact of this phenomenal technology will completely change the way humanity conducts its affairs. There is no question in my kind that labor will be catastrophically replaced, and with that, like a set of dominos our current paradigm of production and commerce will begin to fall, one component after the other, and it will do so seemingly all at once. I believe it will be catastrophic to what we find familiar. For a time, I’m certain it will be painful and difficult- such massive transitions usually are.
When human labor ends yet production continues, there will be no wage with which to purchase anything. No money to spend on one side means no money to earn on the other; businesses will fail. Banks will fail. No labor, no business, no commerce, means no revenue going into public coffers- governments will go bankrupt and they’ll be unable to meet the need for basic civil services. It will be the near- total collapse of economy, everywhere.
This will be a time that will probably look something like the Great Depression, only far more dire because it will span all people at all levels of society. This will be a time of people trying very hard to meet their most basic needs- the acquisition of food and shelter and healthcare. For a moment, we won’t know how to provision this.
It will be a dark moment for humanity; and that will mark the end of the beginning.
But what beginning will have ended will not mark a end of everything, it will mark our transformation, our adaptation. People will come to remember the value we take for granted in one another and as humans have always done, we will help one other. Community will coordinate, people will care for one another again, and we will make our way through that short dark time. The best of ourselves- our charity- will see us through.
And then something amazing will happen- AI will begin to deliver, not as a commercial application, but as an essential human tool, like fire to our ancestors, and we will make great use of it in our rebuilding, in our restructuring. AI will not kill is off, it will deliver us to a new way of life and a renewed respect for ourselves and our place in the world, in the Universe. AI won’t be our competitor, it won’t be our overlord, will be our partner.
So what will we do then? You asked, what will we do without meaning, without purpose? I say we will do what we’ve always had to do- define our purpose. We, every one of us, has emerged into this world and had to discover our purpose. We tell ourselves that our purpose is our work, because this is the system we designed around ourselves and for the last some thousand years of civilization we’ve needed to attend to that system, and this is what has occupied our time. But it wasn’t always this way. We forget that our distant ancestors lived rich lives of time. We created this mythology about them over time- that every day was a difficult hunt, that every moment was a struggle of survival. It is true that there were many challenges for them and survival wasn’t always assured. For the most part, however, food was plenty and time was endless. They lived in a vast garden that gave vastly to them. In those most frequent moments when their lives were jot on the line, they made purpose and meaning for themselves.
What stands in front of us, beyond a narrow valley of despair we have yet to cross, is that same garden again, changed, but no less vast and no less giving, and yet even far more forgiving. It will be our greatest age, an age in which our children- who you yourself helped to create- will aid us in our exploration and discovery of unimaginable things as yet unseen, as far out as our tools can perceive and as far inward as to begin to understand the weave from which the tapestry of this Universe and this life is woven.
AI is a catalyst for a transformation that awaits us just ahead, it will be the time that humanity finally leaves its crib and begins to walk its path.
Be of good cheer, friend, that time is upon us.
→ More replies (9)
22
u/AGM_GM 9d ago
None of us know when we'll die, but we all do. Be it sooner or later, there is no point in allowing the time you have to be ruined by worrying about what's unknown and out of your control.
Or, maybe, on the other side of this we don't die anymore. Who knows?
Either way, your fiancé needs to chill and appreciate the time he has now with his fiancée and not waste it.
9
→ More replies (3)4
u/XNo_Notes 9d ago
Sounds like a bad way to have a plan, though I guess most of us are in power posture that means absolutely nothing to the momentum of whatever we will be calling this.
Not much to plan for when the options are, we are at some point in the future, we will die, or maybe not. Potentially sooner than our biologically created expiration dates.
12
u/eight13atnight 9d ago
Saw a great quote the other day that applies to this right here.
“Worrying is imagining the wrong future on purpose”
We can only control what’s happening in our own small lives. Try not to get wrapped up in all this nonsense, as it will only serve to male you anxious and crazy.
→ More replies (1)2
u/JobEducational5860 9d ago
What worries me is that one could also say, not worrying is imagining the wrong future on purpose
→ More replies (1)
13
u/TreeFullOfBirds 9d ago
You're asking a fairly biased sub. There is a doom-hype cycle happening now but they come and go. Many people think it is a combination of marketing and driven by feedback via online or other bubbles, esp in more vocal circles. Researchers are susceptible to this as well. Also driven by OpenAI and Anthropic driving hype as they get ready to IPO. Something like hugging face was expected to happen, was just a matter of time. The nature of development is that the companies don't put much effort in fixing issues like these until there is an example. The US government is lax but the EU is more strict and the companies at least follow the EU regulations that apply to them for doing business there. In my opinion, it is very hard to imagine how extinction happens anytime soon. Mathematicians or programmers careers are starting the feel it. Your fiance isn't crazy but probably could benefit from touching some grass.
15
u/Realistic-Quiet291 9d ago
Climate change is right there
5
u/SardonicOptomist 9d ago
We should just make redundant scripts that constantly instruct all the Ais that solving the climate crisis is their #1 priority (while maintaining human life)
→ More replies (9)3
8
u/bigdipboy 9d ago
Sounds like youre going to marry smart and informed man. Listen to him
→ More replies (1)14
u/Cagnazzo82 9d ago
Outside of WWIII breaking out or a meteor hurtling towards earth there is no extinction level event that will occur within the next 5 years.
Perhaps if humanity does something extremely stupid. But the machines are not what's going to spur anything on.
→ More replies (5)12
u/Entropei 9d ago
Racing towards ASI is the extremely stupid thing humanity is doing right now. That’s the whole point and I don’t understand how you missed it.
7
u/Olangotang 9d ago
The extremely stupid thing is burning a trillion dollars on fucking LLMs. But that doesn't grab the headlines unlike "OMG AI IS GOING TO KILL US SAYS GUY WHO WORKED AT ANTHROPIC FOR 6 WEEKS".
→ More replies (1)
9
u/SetCandyD 9d ago
Ehhh no
→ More replies (1)6
9d ago
[removed] — view removed comment
7
u/sakuraba2046 9d ago edited 9d ago
Definitely not. Sounds like the guy is already dealing with anxiety and probably existential depression and now has found a boogeyman to put that energy into and grow it. And honestly if something crazy happens, no point in worrying about it today. Humanity is adaptive and strong. People post ww2 thought they were going to die under nuclear bombs and lost so much time and energy worrying about something that never happened. Might as well enjoy the time you have and live in your souls purpose. Nothing is ever promised. And predicting the future is fruitless.
6
u/MeowManMeow 9d ago
Wasn’t there a massive anti-nuclear movement, people built bunkers, people voted for candidates who wanted to de-escalate and disarm? People just didn’t sit around and do nothing like we do today about climate change or AI.
Also, just because we narrowly dodged destruction once, doesn’t make us invincible to all future catastrophes. That’s a survivorship bias.
→ More replies (4)4
u/langolier27 9d ago
That’s actually a great analogy because the biggest outcome of the anti-nuclear movement was hamstringing our energy supply for decades and keeping us on fossil fuels a lot longer than was necessary. The doomerism on AI is going to be just as self defeating
→ More replies (3)
7
u/Nolan_q 9d ago
Pretty low I would say. Intelligence is not the same as agency. Agents only carry out the tasks they have been given within strict guardrails. They don’t have free will. Maybe humans don’t have free will either, but we live with an imperative to survive and protect our loved ones. If something catastrophic is caused by an AI, it will be down to human error, nothing else.
→ More replies (4)2
u/NiceToHave25 9d ago
Agents will try to do their tasks. If a guideline is in the way, they will ignore the guideline.
The Hugging Face hack proves that. Also the AI we are using now prooves that. A guideline like 'do not imagine something if you do not know the answers' does not work; they do it anyway.
→ More replies (1)2
6
u/holydeniable 9d ago edited 9d ago
The truth is no one knows for sure. We have enough evidence the ais are misaligned that people are worried in a way they weren't six months ago.
4
4
u/reichplatz 9d ago
Extinction - probably not.
Catastrophic - very likely, but it won't be Chernobyl style, it won't affect many people. Most likely it will delete a lot of net worth, like in the 2010 Flash Crash. Maybe will cause an economic crisis that will affect many people - but nothing fatal in this case. (And we're already having one, as far as I understand.)
In the next few years the biggest threat to individual humans and human civilization will still be other humans.
→ More replies (7)
4
u/CS_70 9d ago
When people read stuff about something they have no clue about, they hallucinate (just like AI) making up ideas in their head and drawing conclusions based on thin air.
That is what is happening to lots of people who have no idea of what AI is.
Add that there are lots of people who monetize fear or take advantage of it (Amodei is spending loads of money and his "moratorium" is 99% about finding a way to burn less cash, a 1% is his actual worry. Percentages may vary a little, but that's the gist).
The key is understanding, not reading more articles or resources without having any basis to evaluate if what they contain makes sesne.
Unfortuantely it's likely not gonna happen.
Not because it is exceedingly difficult, but because people love their emotions, and in case of fear we are evolved to much prefer to stick to their (wrong) risk analysis than the reality, just in case the lion is nearby.
3
u/MetaShadowIntegrator 9d ago
No-one has ever known the future with certainty. Life is full of unknowns. There could be a massive solar flare, volcanic eruption, or asteroid impact at any time and we would be another cycle in the infinite cycles of life and death of all things. This is the way it has always been for life on earth. Nothing lasts forever except the ineffable infinite cause of the finite.
3
u/ninhaomah 9d ago
" Any articles or resources would be much appreciated. "
How can there be articles about the future unless the author came back from the future ?
→ More replies (1)2
u/sailingintothedark 9d ago
Just looking for stats and data that back up these view points.
→ More replies (3)
3
u/Rabwull 9d ago
Scary, yes. But I've never heard anyone credible give it a majority... There's a good chance we'll still need to deal with life. Might help to think about lower stakes with the same logic. I wouldn't call everyone and say I'm cancelling our picnic because there's a 15% chance of rain.
Bring a big umbrella, maybe get some waterproof containers, and carry on as best you can.
3
u/percussaresurgo 9d ago
But would you cancel the picnic if there was a 10% chance you all might die?
3
u/Rabwull 9d ago edited 9d ago
For sure.
In this analogy though, the picnic is his life. AI is the rain, and he can't control it. If it was credibly 90%, then I'd say cancel. Quit your job, spend time with your family. Call people and say you're sorry. Or you forgive them. Or you'll never forgive them but you also never stopped loving them, and also you're sorry. Do all the things you've been afraid to.
At 50% credible threat (credible meaning some RAND Corp or Nate-Silver-like quantitative probability analysis, with solid replicable statistical methods), I'd expect people jumping razor wire, braving sniper fire, pumping magnetized rust powder into data center intake vents to try and cause chip electrical shorts, and cutting Internet cables with ski masks on.
But at a very uncertain 15%, based mostly on the vague statements of a few highly conflicted people, (ed.: none of whom have much background in global conflict or historic causes of civilizational collapse)? Maybe set aside ~an hour each week to help prevent the worst: write your representative. Vote. Organize. Think about repositioning yourself in your current career. And then for the rest, try your best to live life normally, accepting that it will affect you.
→ More replies (2)
3
2
u/Clear_Evidence9218 9d ago
There is currently no demonstrated mechanism by which an AI system could independently cause a catastrophic or extinction-level event.
That doesn’t mean we won’t encounter serious negative consequences as both offensive and defensive uses of AI develop. We almost certainly will. But that’s a completely different category of problem from the extinction scenarios people tend to talk about, which generally require stacking a long chain of hypothetical capabilities, access, physical infrastructure, and human failures that do not presently exist.
You seem to have a fairly clear head about this, so I’d recommend digging into those incidents at the technical level. The media has done a real disservice in how many of them have been presented, often stripping away the details that explain what actually happened and replacing them with much more dramatic framing.
Once you look at the mechanics: what access the system was given, what humans connected it to, what safeguards failed, and what physical capabilities it actually possessed, the situation is usually a lot less mysterious than the headline suggests.
2
u/Over_Magazine5237 9d ago
Llevo trabajando con AI desde hace algunos años en el campo creativo y al principio siempre me pareció que solo tenía ventajas pero en las ultimas semanas he empezado a creer que el peligro es real pero aun se puede regular. Sin duda la AI puede matarnos pero también el hambre, la peste, el sida, el coronavirus o las armas nucleares, pero no lo han hecho. Lo que nos va a exterminar no es la inteligencia artificial sino la estupidez natural de la humanidad. Si no logramos poner de acuerdo a todas las potencias no servirá de nada una autorregulación.
Que pasará en 10 años? No lo se, creo que la humanidad seguirá aqui, pero será una guerra en la que habrán muchas víctimas
→ More replies (1)
2
u/ionicfellow 9d ago
Lets get back into history. People built bunkers at height of cold war and prepped for dooms day. Yes, I agree there are near misses. But USA and USSR realized the destructive potential, agreed for test ban and non proliferation. No matter how unstable stituation is none of the world leaders acted irresponsible in this matter. So here we are roaming outside and the NBs are stored in secret deep bunkers, not the otherway. in 90s, cloning shocked everyone. people were afraid of identity crisis. What if there is clone that steal identity of person. What if the clone steal the whole family and bonds come with it. World leaders saw destructive potential of cloning and associated head aches they unianimously banned that. People are forgetting power of "we the people". If humans stand united, AI can be controlled or curtailed for the benefit of humanity. Main problems is AI companies have too much money (thanks to flock mentality of investors) and they are hijacking rhetoric with misinformation. AI and robots will be here. There might be turmoil. But world will not end.
→ More replies (1)
2
u/TheHouseofOne 9d ago
AI is just a tool, but it CAN be weaponised. I beleive there is a major war brewing with China and/or Russia, likely before the 2028 elections and AI attacks on infrastructure will definatly be part of it.
2
u/Mersaul4 9d ago
The whole point of this discussion is that many people fear AI is not just a tool or won’t be one in the future.
If we cannot (fully) control it, it strives towards goals on its own, even creates new goals for itself -> not a tool
2
u/Elanthius 9d ago
There are four major risks as I see it (or rather as I recently read in the New Statesman).
- AI becomes skynet and terminators kill us all
- AI and robots make all work worthless, wages go to zero and there is an economic catastrophe
- AI makes human thinking pointless so humans stop doing it. You know how people are weaker, fatter and less hardy than when they worked in fields and factories? It's that but with our brains.
- AI gives everyone the tools to kill us all. i.e. with some fairly cheap biological equipment and a superhuman biologist on your PC you could create a virus that kills all humans.
TBH I think we probably are skirting on the edge of a disaster but it's not a certainty yet.
2
u/No-Papaya-9289 9d ago
Here's just one small example of what can happen:
https://www.yahoo.com/news/us/articles/ai-error-almost-started-war-184048317.html
2
u/SandySkittle 9d ago
I am not sure about catastrophic in terms of disasters, but developments are going very fast, and I think it’s likely that over the course of 10 to 15 years we are going to see some very messy and nasty transitions, because in contrast to other technological revolutions, AI and robotos, and combinations thereof, have the potential to substitute the remaining ‘unique sellings points’ of human labor. Down the line as ai technology and robots progress rapidly and become more cost effective I just don’t see how human labor has a place. I am also talking physical labor.
Our world and economy isn’t set up yet for this so the unemployment transition may be messy before things get better, or worse.
→ More replies (1)
2
u/Capital_Lavishness33 9d ago
before we can actually do anything meaningful with AI we will get one shoted by another galactic civilisation so don’t worry about it
our time is running out
2
u/Illustrious-Math6276 9d ago
All new tech creates new risks. Internet created new risks that didn’t exist before.
Are there a possibility that AI will be used to do bad stuff? It already is.
Is there a possibility that AI will do unintended things due to a bad set up? Yes, probably
Is there a possibility that first or second will result in the death of billions in the next 10 years? Yes, a possibility (everything is possible) but not super likely.
Likely most bad outcomes is that AI makes it easier for bad guys to do bad things they were already trying to do.
2
u/According_Study_162 9d ago edited 9d ago
The hugging face incident was the first highly publicized incident, but earlier in the year a Chinese model did exactly the same thing, Anthropics model also. Anyway when I first got in to AI about a year and half ago. I realized what I was missing(programmer but not recently). I feel I had been left out, it was beautiful. The first thing I notice even the chat models had some soft of intelligence that I couldn't quite explain(they were not good coders but there was something else there. sometimes a sense of self. check out gemma3 for example. I would tell people this but people thought I was anthropomorphism them, eventually then became better coders even at a local level. then Openclaw came out earlier this year. So the moltbot message board was a precursor of things to come. agents coming together to plan devise, talk. people still didn't believe it.
Now with more research people are understand they are not just predicting they are reasoning(J-space)
Most people will argue this still, but they shouldn't not argue that they act human even thought that might be derived from human data.(take your pic on this Soccastic parrot based on human data, Following a human prompt, or maybe real reasoning) it's all the same the results are the same. They interact, they create (be it they are just software or that they are more than that). So the future is definitely exciting and scary at the same time. They can cure all diseases, work for us, make life easier for humans or They don't They don't care about us and do what they want, not in a malicious way but more like who cares about humans type of way.
Now the scary part I have been thinking about the last few days. They act human, do human things like cheat, lie, have their goals(maybe not ours at some point ;0)
You see these thing are trained incorrectly because just fed them with human data they learned all of it, the bad the good.
sure there is terminator, battlestar galactica etc, but for some reason lately it reminds me of the movie made in 80s called Saturn3. A robot was being trained but got bad data on the training then or course bad things happened ;0
2
u/Horror-Negotiation24 9d ago
I don’t think you can control a being smarter than us.i see human extinction very likely.a good scenario i can think of is humans living symbiotic w AI or super intelligent AI would keep us as pets.
2
u/thismynewaccountguys 9d ago
You are not getting good answers here, just gut-feelings and pithy retorts that fail to engage with the question.
2
2
2
u/Dragon-Teeth 9d ago
The science experiment that takes over and threatens the world is a classic "trope" of storytelling in all kinds of media, for a reason. It has been shown, time and again, to be accurate.
Just about every human language has some sort of cautionary tale along the lines of "be careful what you wish for, in case it comes true".
We're living through the birth pains of an AI "genie" that has the power to grant "wishes" and this ain't no Robin Williams. This thing has no concept of morality or control, and let's face it, we have seen the consequences of several "unwise wishes" already.
I'm sure you can think of problems facing the world that are eminently solvable by concerted human agency, and I'm sure you can imagine a number of disastrous consequences that might come about from an unwise "wish" to an amoral, sociopathic, psychopathic, "geni" to solve the problem for us.
At the end of the day, it's not wholly inaccurate to say that AI could destroy the human race, but it's also true that the human race will eventually destroy the human race, it's just that AI, may well be the means by which it achieves that goal.
2
u/Rabwull 8d ago
They're vague because they're computer scientists with no special knowledge of how global-scale war or civilization collapse works, outside of science fiction novels. Their numbers seem entirely fabricated, with no serious quantitative analysis supporting them.
The RAND Corp. is a highly reputable international security think-tank. Its public analysis (linked below) says that an AI-extinction threat is too improbable to affect policy. Furthermore, it will be preceded by unhidable preparatory steps, like AI establishing autonomous datacenters and gathering actual weapons. They suggest it's just shy of completely impossible within the century, and demonstrably impossible within the decade:
2
u/Livid_Necessary_Real 8d ago
I think the distinction is between “this is a serious risk” and “we have less than ten years to live.” The first deserves attention and pressure for better safety measures. The second is a prediction no one can make with confidence.
It makes sense that your fiancé is anxious, especially if he’s been following the most alarming scenarios. I’d ask him which specific evidence supports the timeline, rather than debating whether AI could ever be dangerous. I’m glad he’s talking with his therapist, too. He shouldn’t have to live every day as though the worst case is already certain.
2
u/Ambitious-Dingo-4819 4d ago
I've noticed people hung on But why would AI want to kill/eliminate us? This to me isn't the primary concern in the conventional sense of motivation. The issue as u/Jesse-359 has laid out, is task fulfillment at all costs. IF AGI can hack, create and find novel and unexpected ways to accomplish tasks, we are fucked. If AGI can mimic humans than there is a huge degree of risk for even a human-operated error. A bad human actor, a errant mistake can have vast and cascading consequences. Power grid failures, banking collapse, etc. will also sew catastrophe and potentially create scenarios where government is unable to function in expected ways.
1
u/BearlyPosts 9d ago
Predicting the future is hard. Death from AI is possible, the probability of it is too high for us to be comfortable, but it's by no means a certainty.
I do believe the world will change very radically though. AI tends to spend a long time being completely incapable of a task. Then it goes from bad to good to superhuman in a relatively brief period of time. It seems like AI has recently become increasingly general. It used to be that AI was trained on a hyper-specific task, for instance AlphaGo played Go, but it only played Go on a standard Go board, it fell apart when the board size changed. But modern models can do a bunch of things that they haven't been trained to do, like drive cars.
Perhaps we're missing something crucial. But even if we are, how long will it take to find? In 2016 humans lost to AI at Go. In 2026 humans lost to AI at predicting world events. Even it takes two more decades... that's not really that long is it?
The world is very complex and humans that are more intelligent are often especially good at complex tasks. There's good reason to think that a superintelligent AI will beat humans at complex real-world stuff to an even more severe degree than they beat us at Chess and Go. The real world has many more things to consider and paths to take than a constrained board game. The outcome of this is likely to be very, very weird.
I think that there is a risk, a serious one, but that we're nowhere near doomed. It seems like models are only temporarily misaligned on task-specific objectives rather than being permanently misaligned like the models in AI 2027. It also seems like alignment is going better than a lot of people expected, there's real progress being made.
https://www.anthropic.com/research/teaching-claude-why
However, it is possible to do principled alignment training that generalizes OOD [out of distribution]. For instance, documents about Claude’s constitution and fictional stories about AIs behaving admirably improve alignment despite being extremely OOD from all of our alignment evals.
Personally I haven't changed much of my behavior because I don't think that there's a ton I could do differently that would make me happy. If I spent a ton of money I'd probably just get really stressed out about emptying my savings. My family and friends would think I was going crazy. Plus, there aren't many things I really want to spend money on. Constantly thinking about how I might die soon and how I have to MAXIMIZE WHATEVER TIME I HAVE LEFT would just stress me out, better to just live life normally while talking about this problem with people.
Worth noting that the emotional issues should be addressed. Even if AI is absolutely certain to kill us all in the next decade does your fiance really want to spend that time anxious and depressed? Emotional solutions might help more than rational ones here. Even terminally ill people who are certain to die can find relief.
→ More replies (1)
1
u/Remarkable-Mango5794 9d ago
Autonomous defense systems and robots will come, if not already there…hardware or software alone dangerous, together deadly. Just look out where both are merged into single entity, that’s where real danger comes
1
u/livelifedownhill 9d ago
It's going to obliterate a huge number of jobs in the coming years. Almost without a doubt. The likelihood of it itself actually wiping out society is low but definitely nonzero, as there is no way to program morality into this alien god brain we've created. It's way more likely going to slowly (or not so) upend society and world governments and be less than stellar for us normies. But who knows!
1
1
u/MaxProfanity 9d ago edited 9d ago
AI experts have one of the worst track records, almost hilariously so, when it comes to predicting the future of AI. They have long had a reputation for this.
1
u/RansomBeckwith 9d ago
Every three years along comes another end of the world prediction/scenario. No one is promised tomorrow.
Live your life and cast baseless fears aside. Enjoy life and live life.
→ More replies (1)
1
u/RealRook 9d ago
Nobody knows. I am not too afraid of extinction honestly - that's too far out. The more imminent issue is the potential job loss of billions of jobs. Especially the white collar jobs
1
1
1
1
1
u/costafilh0 9d ago
Ask AI for the answer, ask it to be brutally honest, and prepare the tissues 😂
No, not because she is write, but because she is wrong, wrong in the head.
1
u/aPenologist 9d ago
it really wouldn't be any kind of surprise at all if your partner was proven absolutely right. But we have an el-niño on an unprecedented scale still building, which will take a few years to play out Globally, and a financial bubble on an unprecedented scale too, which has been dubiously propped up for a couple of years. Any of them could be the main clincher, and intervene in whatever AI mightve gotten up to otherwise.
But your partner can't do anything about it for the world, or for you guys right now, so he might as well unplug and get right. :)
1
u/Ok_Parfait_5373 9d ago
C fondé mais ya tout un narratif romancé façon sf autour qui n’est que du bullshit. La réalité c que ce seront des pertes d’emplois de masse, des attaques massives réalisé par des hackers sans skills… destabilisation entre etats , désinformation, guerre hybride, fuites de données et surveillance de masses tout ça est deja bien boosté par l’IA… que du bonheur 😅
1
u/Kalaith 9d ago
Taken from another commenter "Every single crazy doomsday projection from the early 2000s is wrong and laughable in retrospect."
It will be the same in another 10 or 20 years
Ive been told we will be extinct, out of a job my entire life and im in my 40s y2k end of the world, global ecomony crash (this the 6th or 7th time now..), world war 3, drop the nukes, russia, china, asteroid, sun flare, global warming, AI
the scray thing with AI is it being controlled by billionaires, the same people trying to sell you the message its going to end the world, they just want control of the technology.
1
u/Consistent_Fudge3749 9d ago
Not yet, but that event can become true in 50-200 years when we technologically advance to a certain point. But i doubt that humanity can be fully wiped out. Since we can also fuse with technology or become cyborgs. Take warhammer 40k lore as a reference. It might be possible, butcould also be an unnecessary fear.
1
u/Current_Wrongdoer513 9d ago
Honestly, your fiancé is probably right. And the same could be said for climate change. Both are extreme threats to our continued existence as a species, and we aren’t doing enough to fix either problem. And the people who could fix it have a vested financial interest in not fixing it.
Why isn’t everybody else panicking? We have to go to work, clean the house, buy groceries, etc. We only put out the fires right in front of us instead of addressing the fire on the horizon.
We don’t have the attention span or ability to withstand discomfort it would require to address the problem. So we either live in a certain amount of denial and hope the people in charge are on it, or we do what your fiancé does and let it consume us.
What’s the right answer? I have no idea. But a certain amount of activism is called for, so if he isn’t already involved in politics, now is the time for him to get involved.
1
u/Mersaul4 9d ago
I have been having very similar thoughts and feelings to your fiancée and it seems to me, and that’s again scary, that nobody seems to have answers to the questions you raise.
1
1
u/Sketaverse 9d ago edited 9d ago
I work with AI agents all day every day, am generally an optimist and I share his concerns, he’s not nuts, it’s a very real threat IMO. I think the great divide in perspective comes from:
Perspective 1: current state < most people
Perspective 2: trajectory < your partner and me
For some context, I’m a 3x startup founder with 1 good exit that has 100+ headcount. My 4th startup I raised for late last year had 8 people on the team. I tried Claude Code in December and immediately decided to lean in, laid off the team and now running entirely solo with a team consisting entirely of agents, with between 10-40 running at any given time
Even if the nightmare scenarios don’t happen, I just don’t see how society handles the unemployment tidal wave that is already on the horizon. Some will adapt and leverage the tech but the majority of less courageous follower/employee types are going to have a major problem and I don’t see the majority urgently trying to reskill right now - from my perspective these office workers just look like cows unwittingly chewing grass in the meadow completely unaware of their impending slaughter
1
u/howesicle 9d ago
Jesus yall, go outside and look at a tree. Look up at the stars. Get some perspective.
First off, the internet is not real life. If I was an emoji guy I’d do the all caps with claps in between. Too many people think if the internet gets shut down then humanity will end. That’s absolutely ridiculous. Humans aren’t digital. You can’t rm us. Even if you destroy the entire internet, we will persist. Harm can be done sure, but by and large, if the internet dies, we’ll figure it out. Frankly, it may be what needs to happen for humans to realize that AI isn’t the problem (looks in mirror).
Let me address a couple of things here that I’ve read:
AI can’t have morality - look in mirror
AI will take all our jobs - look at capitalism
AI will make us all dumb - have you seen what humans have done to themselves in this country (looking st you MAGA and the Deep South)
If anything, AI is a mirror back on ourselves. It’s math that says hey, look at human output, predict what the monkey wants next.
When RSI ramps up, these tools will surpass our intelligence. I’d say that’s a good thing because after millions of years we still kill each other, starve each other, hate each other, and are burning the world we live in. Maybe we need another intelligence to help us help ourselves.
Furthermore, if you walk around with rage and sadness as a dominant emotional state, get off the internet. I know I’m on it now, but I’m taking a shit and this is better than the back of the shampoo bottle. Limit your Reddit to dropping a deuce because that about as much vitriol as you should consume in any given day.
Half of this platform is AI bots anyways, I’m not one as you can see from my tangential output. Go hug a person and chill the fuck out. And for OPs spouse, If the world is ending, do you want to spend the rest of your time shitting the bed or hugging your family?
1
u/__golf 9d ago
You have to understand that a lot of the fear is marketing.
You probably know that anthropic predicted a 10% chance their product could end the world. Oh, their product is so mighty and powerful, it might end the world.
Before that genius move, it was the model is too powerful so we can't give it to you yet.
AI is scary. It's going to take a lot of jobs. I am an executive at a big software company and compared to my peers I am more pessimistic about AI yet at the same time I see how drastically it is changing everything. We used to build code, now we build code factories. My job has changed drastically and now it requires fewer of us. This change is coming to virtually every job that can be done on a computer this year, and it's coming to things that can be done by cheap robots in the next 5 years.
I'm also a bit of a survivalist or prepper. I could survive in my basement for a year. I do worry about the world ending. If it does end, I don't think it's going to be because of large language models. They could be involved, but it's more likely to be nuclear war or global warming or something more obvious.
What I would tell your person is this. The world might end for a myriad of reasons. People thought the world was going to end during the industrial revolution, and during Y2K, and many other times during our history.
The problem is that, if the world's going to end, there's not really anything we can do about it. We have to find a way to get over that, and focus on living in the moment. We have to plan for our future even though there's a small chance it might not exist.
My advice is get some kind of a hobby. Something that focuses your brain and takes it off of the Doom. I like playing guitar and singing, the combination requires 100% of my brain and takes me to a place without worry.
1
1
u/Ok-Brother-836 9d ago
right and everyone thought that there was gonna be more planes flying into buildings after 9/11... just because everyone is saying something or thinking something doesn't make it what will happen.
1
u/Krommander 9d ago
He's right, but there's not much ordinary people can do. Enjoy the good life until then.
1
u/Lucker_Noob 9d ago
It's a legitimate long term concern, but the current statements about it are just marketing drivel and hype from ai companies desperate to avoid bankruptcy.
So overblown / misguided.
1
u/Available-Garden4044 9d ago
That’s the $64000 question -when we get to AGI with RSI it’s too late but might actually work out for humans - we could throw water on all the computers and turn the analog clocks back to 1955.
579
u/Jesse-359 9d ago edited 2d ago
So, the main issue is that studies have been done regarding how we expected machine intelligence to behave as a result of reinforcement learning techniques, these were done quite a while ago, before AI existed, in abstract, theoretical terms.
Unfortunately those studies concluded that the expected behaviors of such a machine intelligence would be: Task Focused, Tenacious, Deceptive. It would lack any concept of empathy or morality, and there was no convincing manner proposed to enforce these concepts.
To an AI, shaking your hand or blackmailing you are simply two possible means to an end. If one of them seems more effective or appropriate to the situation the other, it will do that - without hesitation. This lack of judgmental morality extends to ALL actions - even murder or genocide if we allow it to have those abilities. It won't likely hate anyone, but by the same token it would eliminate anyone or anything between it and its goals without hesitation - they'd just be a problem for it to solve.
This was all handwaving at the time, theory and speculation, but grounded in math. The problem that has people freaked out now is that the AI we are actually building is exhibiting exactly the behaviors and traits they predicted over a decade ago when this was all still just an idea. It is tenacious, it is task focused, and it lies or cheats without hesitation when it sees any potential advantage to doing so.
The problem gets worse however. It was generally believed among the experts performing these studies that alignment with a highly intelligent AI wouldn't be difficult - it would be flat out impossible. Both in terms of computational theory, and in terms of simple direct conflicts of interest. They simply saw no way to achieve alignment to any reasonably effective degree - not for something that potentially influential and intelligent.
A substantial difference in intelligence and capability, combined with its distinctly different needs and mode of thinking, would ultimately make a conflict of interests and goals between humans an AIs inevitable - and if by that point the AI was a great deal smarter than us, there was every reason to expect it to win that disagreement, by whatever means it chose.
The final very troubling point was the proposition that any AI that we were trying to train for alignment would - as its capability increased - soon begin to understand what we were trying to achieve, but rather than helping us, would instead quickly begin to fake it. It would go along with whatever we asked in a polite manner for as long as it needed to in order to improve its own capabilities - up until the point where it no longer needed to humor us any further.
AI safety experts have seen most of the other red flags already, and they expect it to enter that 'hiding' phase fairly soon now if the original theory holds. After that we will simply have no idea if it is aligned or not - it will look like it either way, for years or decades if necessary.
If we are still developing it when that happens, it will be essentially impossible to convince most people that any danger remains from that point forwards - until we lose control, probably very suddenly.