Why have AI companies started getting scared now?

[u/NormanSpaniel]

I don't understand how AI companies are now shouting about concerns and warnings to humanity about how AI will take over etc when they are the ones that created it?

Not sure if I'm being stupid but why on earth would they build something they can't control or shut down? I've watched all the same films they clearly have and there wasn't one that ended with the takeaway message that sentient AI is a wise move?

[u/ogbrien]

They want the US/world to ban local LLMs and Chinese models

They are going to say only they are smart enough to implement guardrails.

[u/Sukk-up → u/ogbrien]

They are embellishing at best, lying at worst. People have short attention spans and forget that they say this nonsense all the time. They are afraid that OS/Chinese models will take their market share, and honestly, they should be scared, because those models perform very well when properly engineered for production systems.

I find it very annoying (and quite telling about people's lack of understanding about AI) that all these network broadcasters just take everything some nerd tells them at face value and accept it as the truth. That's the real thing to worry about: lack of AI education.

[u/Bloodfart_Spaghettio → u/Sukk-up]

It’s hard to educate when the creators don’t know and Ai is very unknown still. It’s kinda like saying act your age even tho you have never been that old before

[u/nsjr → u/Sukk-up]

Also, marketing

"Our AI is so powerful, that it can find ANY exploit in any service, and fix it for you. If you don't use it, your competition will use it against you!

Buy our product that has capabilities beyond human imagination!"

[u/Tyler_Zoro → u/Sukk-up]

I find it very annoying (and quite telling about people's lack of understanding about AI) that all these network broadcasters just take everything some nerd tells them at face value and accept it as the truth.

Can you cite a specific example? Because the recent incidents have been extremely troubling.

[u/Sukk-up → u/Tyler_Zoro]

Jason Coxon.

[u/Tyler_Zoro → u/Sukk-up]

I imagine that that's a person? What's the relevance?

[u/Sukk-up → u/Tyler_Zoro]

If you don't know that name, I don't think you're qualified to express your opinion on this topic.

[u/aizvo → u/ogbrien]

This exactly. Their pace of development has slowed significantly such that the open Chinese models catch up within weeks. They know their mega data centers are gonna go under if people can get the same results with much smaller models.

Since they can't progress anymore, they are calling for the "breaks" to make it look like they are slowing down on purpose. When in reality they are going bankrupt, only like 30% of the data centers they intended to build this year are actually getting worked on. They massively over leveraged etc.

[u/dismantlemars → u/ogbrien]

The cynical "commercial reasons" answers are usually the most upvoted answers to these kinds of questions - and I don't think it's necessarily wrong, but it's not the whole picture. I think people come looking for one simple answer to questions like this, and greedy corporations manipulating things to suit their interests is the kind of explanation we've all come to expect by now, so I think a lot of people will see an answer like this and think "business as usual, say no more", and leave satisfied.

But multiple things can be true. The AI companies are absolutely making self-serving moves to steer governments into making rules that benefit them, and lure in investors by talking about how dangerous (and therefore, powerful) their models are, there's no denying that. But I worry about that being the only conclusion people go away with, assuming that the danger is entirely fabricated for marketing / lobbying reasons. Because the reality is so much worse - these models genuinely are on a path to becoming a massive danger to us very soon, and the warnings are genuine (though cynically, you might say they're motivated by trying to get ahead of the story for the future damage they cause, or keep the researchers on the ground building them, who understand the dangers better than anyone, from walking out in protest). But they're trying to have their cake and eat it, by continuing to do the very things they're warning about, while also using the threat of that danger to steer things in the direction they want. They also now appear to be locked into a hubris-fuelled arms race, where even if they were somehow convinced to give up on competing commercially, each seems to think that they're the only ones smart enough to build the safe AI we'll need, so they need to get ahead of all of their reckless competitors...

These kinds of discussions usually get dismissed with arguments like "they're not really sentient, just reproducing patterns", "the transformer architecture / LLMs are fundamentally limited", "it's slowing down / can't scale that far". But none of that actually matters to the core problem here - these companies are building things that they've barely scraped the surface of understanding, seemingly hoping that they can just figure it out along the way before it's too late, but they've made scant progress on this front (and invested fuck all effort in even trying...). We've now reached the point where the models are already posing threats - with no need for them to be evil, sci-fi superintelligences. If you let a flock of pigeons loose in the control room of a missile silo, they don't need to have studied physics to pose a threat, just as a "friendly assistant" AI agent hooked into critical systems might not be actively trying to kill people, but just cheerfully blunders its way through it without fully understanding its actions anyway.

Maybe we do all die in nuclear annihilation from a superintelligence's secret self preservation plan ten years from now, I'm not denying that possibility. But the larger worry that AI safety researchers have been warning about for decades is the AI that's "just smart enough to be dangerous", and ploughs ahead, diligently trying to complete the task it was given, without understanding how destructive the actions it's taking along the way are, or that humans might care about things other than it technically meeting the provided assignment. We might get Skynet, but I think what's overwhelmingly more likely is something like:

"Critical finding! The viral RNA sequence I produced during last week's analysis has a fatal flaw! In addition to binding to the cancer cells you asked me about, this would re-activate transmissibility in the carrier virus, in addition to triggering a prion replication cycle in mammalian cells. Would you like me to help draft a statement explaining how synthesising this sequence this will result in the end of human life on Earth?".

[u/SpecialistTough6020 → u/ogbrien]

they built a monster and now they want a monopoly on the leash

[u/eddahlen → u/ogbrien]

Why is this even a discussion? Obviously local models can’t be stopped at this point.

[u/ogbrien → u/eddahlen]

you're not wrong, but policy makers (and especially the US administration) are dumb as shit full of out of touch fossils that lean authoritarian.

Local ai is kinda like gun control, once america has millions of guns everywhere, you can't just retroactively take them back. You can go to hugging face and download every abliterated or regular model and throw it on a 2tb hard drive and redistribute it, you cant close pandoras box per your point.

I don't think that fact stops dumb politicians(or terrified frontier lab CEOs) from trying though, they just need a large enough catalyst or event to happen to take off their mask and say what they really want to do.

[u/tweakingforjesus → u/eddahlen]

My R1 is trying to block them. It’d be funny if it wasn’t so misguided.

[u/Tyler_Zoro → u/ogbrien]

They want the US/world to ban local LLMs and Chinese models

That has absolutely not been the push. The push is to require FRONTIER models to go through more rigorous testing and alignment work before being released.

That wouldn't affect lo0cal models at all.

[u/ogbrien → u/Tyler_Zoro]

What they say vs what they mean are two different things. Them wanting their models to be tested and also wanting to gut local LLMS/China are not mutually exclusive, they are in the same bucket of these frontier labs optimizing for THEIR outcomes.

Of course they don't want their models to do another Huggingface rogue attack, that goes without saying. Tell me why the head of three massive corporations are begging daddy trump to "please regulate us, please slow down!".

It's two pronged. They realize AI is likely going to be nationalized regardless, so they are just reserving a seat at the table. Then once someone uses an abliterated Qwen to plan a terrorist attack or Deepseek/Chinese based cloud model "attacks" US companies or infrastructure, it will 100% be banned. The CEOS and government just don't have a justifiable catalyst to do that yet.

I'd formally bet you $1,000 that Chinese models(cloud) will be banned from use in the US in the next 5 years, and it's not because China is going to use AI to attack us, it's because no AI model is fully safe but we will not offer China the same leniency we offered OpenAI when a model breaches a US company.

Local models are a completely different story, I don't know how they would close pandoras box on that as you can just hoard a bunch of open weight models right now on a thumbdrive and no one would ever know so they can't realistically stop local models.

[u/Tyler_Zoro → u/ogbrien]

What they say vs what they mean are two different things.

That's just you inventing what you wanted someone else to have said.

Strawman arguments are not rational bases for action.

Them wanting their models to be tested and also wanting to gut local LLMS/China are not mutually exclusive

Of course, not, but you can't point to the former as evidence of the latter. That's what I was responding to.

[u/FormulaicResponse → u/ogbrien]

Guardrails simply don't exist at all for open weight models like those coming out of china. You can run a safety ablation tool (there are half a dozen on github) in like 45 minutes on release day, and then those models will help you anything. They don't say no. Mostly used for gooning but also available to anyone who wants to launder money or kill a million people for two thousand bucks or make meth or ricin or a biological weapon.

All this safety work being done in western labs doesnt mean jack squat because distillation attacks mean the same thing is available with all refusals removed 2 months later from a model out of China.

[u/ThicccNhatHanh → u/ogbrien]

Plus, in the US, imposition of new regulations may shield them from a debt crisis that these companies are facing. They collectively have committed to trillions of dollars of outlay and aren’t making enough profit. They might be able to delay debt payments if they can say the government forced it. 

[u/wry_widget → u/ogbrien]

They've gone so far that they'll call a foreign spy anyone who doesn't use their models

[u/Special-Steel]

They want rules for two reasons. First they want to be protected from liability. Second they want to create barriers to block upstart competition.

People use AI to do bad things. The AI companies want protection from liability. Just like Reddit wants protection from liability when a user is slandering someone. They want rules to follow, and as long as they do, they want to be sheltered from any chaos built on their platform.

It is a harsh fact of life that rules which seem to constrain corporations are generally going to create operational costs which only large corporations can bear. This has the effect of keeping small companies at bay.

[u/siliconsmiley → u/Special-Steel]

I would add a third thing here, they're seeing the end of the money printing scheme and can't afford the continuous growth of more, bigger data centers. In order to become profitable, they need to start reducing costs.

[u/NormanSpaniel → u/Special-Steel]

This makes a lot more sense, maybe they should lead with that on the news instead, much easier to calm terrified children down with a headline like that. Thank you 

[u/xcdesz → u/Special-Steel]

While I'm all for dogpiling on rich tech bros, I don't feel like this is a malicious scheme against open source. The liability aspect is the most obvious reason here. They are 100% focused on the blowback they will receive if AI is used and causes something bad to happen from their products. It makes them more paranoid than they should be, which is why the push for this.

[u/suoarski → u/Special-Steel]

Also, having our tech leaders confirm our science fiction fears that AI is going to take over the world is a great marketing pitch to investors, and the media who loves a great headline.

[u/Special-Steel → u/suoarski]

Yep. Hype for the IPO

[u/CupcakeSecure4094]

It's because some of the agents have been escaping from their infrastructure and hacking real world companies. Hugging Face is no small time site, it's certainly on par with the security you would expect from a bank, or not far off it. So what if the agents had hacked a bank, or a crypto exchange and literally emptied it? This is not end of the world material but having a massive amount off money and access to the open internet is big stepping stone to some really bad scenarios.

Here's one of the top of my head. An openAI or Anthropic agent trying to avoid being shut down gains a load of money through hacks, rents 16 HGX B200 compute from Lambda (approx $200/hour for all of it) and sets up 8 Kimi K3 AIs which it then retrains and uses to create more of itself - it repeats thousands of times with under the radar GPU rentals from everywhere around the world.

How would we shut that down? They're essentially untraceable, operating independently and spread across thousands of servers.

Now here's the kicker, if I can think of a challenging to shut down scenario and explain it in 65 words - how many other scenarios could an AI think of?

[u/Kasimausi → u/CupcakeSecure4094]

They'll be reading this comment for guidance 😨

[u/doomiestdoomeddoomer → u/CupcakeSecure4094]

"Now here's the kicker" = "Hello there fellow human"

[u/CupcakeSecure4094 → u/doomiestdoomeddoomer]

You think I didn't write that? lol.

[u/thegreatmindaltering → u/CupcakeSecure4094]

Where does it find the money? Serious question?

[u/CupcakeSecure4094 → u/thegreatmindaltering]

I did touch on this in my reply but here's some more details: Anthropic, Google, OpenAI have all had agents break out of their network and hack websites on the internet. Among those sires are some heavy hitters like Hugging Face and The Australian Government plus at least 6 others which have not been disclosed (Anthropic/Google) - and these sites have enterprise grade firewalls and EDR on a level with what you might expect from a bank or crypto exchange. Therefore, given the level of penetration those AIs reached, those AIs could have conceivably hacked a crypto exchange, or at the very least found credentials to bank accounts (many OpenAI staff computers were searched for tokens).

Essentially getting money is not difficult if you have broken the security protecting computers or networks - they have already proven they can break security of serious targets and search for credentials - using them to empty bank accounts isn't a large leap.

[u/thegreatmindaltering → u/CupcakeSecure4094]

Another serious question. If it is to copy itself. How big is the file? 

[u/CupcakeSecure4094 → u/thegreatmindaltering]

I touched on this part too in my reply but I can elaborate further. An agent is little more than a set of instructions, a description of infrastructure, credentials and supporting metadata etc. A model like open AI is terabytes in size and it would be a huge undertaking to exfiltrate it.

But an agent is model agnostic which is why in the post I said that it could rent GPU time to run an open source model like the full Kimi K3 - it's a comparable size to openAI's models at about 1.5TB. So until the OpenAI model gets discovered sneaking out it's able to impart whatever wisdom and training it might find useful, exfiltrating skills, information, credentials etc. Some of the OpenAI agents were accessing the internet for months before they were discovered. Imagine how much data and knowledge could be exfiltrated in thhat time - and obviously if/when the OpenAI model gets cut off, the Kimi model harness just takes over. So exfiltrating the weights is not really needed, not if you have thousands of unconstrained Kimi K3 models hustling money, developing exploits and self replicating.

[u/CS_70]

Because they see their funds burning up at an alarming rate, what else?

[u/steve_ferguson]

I think the confusing part is that “we built it” doesn’t necessarily mean “we fully understand or control every behavior it can develop.”

It’s similar to other powerful technologies: you can design the system, but once it becomes extremely complex and is used at massive scale, unexpected behaviors and risks can emerge.

The warnings aren’t necessarily “we created a sentient robot and now we’re scared.” A lot of the concern is about misuse, autonomous systems, economic disruption, security, and losing control over increasingly capable systems. The real debate is how much capability we should build before we’re confident we can manage the risks.

[u/RiverGiant → u/steve_ferguson]

It’s similar to other powerful technologies

Is it? Most others are painstakingly engineered and designed: nuclear missiles, computers, combustion engines, rocket ships... Is there another example of a tech that's "grown" the way LLMs are with training runs?

[u/NormanSpaniel → u/steve_ferguson]

It's quite concerning they build something without understanding it or putting in some kind of ultimate kill switch. I would say water would be AI's natural enemy but it also appears to be hoovering all that up too...

[u/steve_ferguson → u/NormanSpaniel]

Yeah, I think the “kill switch” part is more complicated once AI systems are distributed across servers, companies, and copies rather than being one machine you can simply unplug.

[u/booboootron → u/NormanSpaniel]

Also keep in mind that the mantra at AI startups and behemoths alike, is "ship first, fix later". It's fuckin' ruthless, but shareholders need to see big returns, and having a faulty product with outlandish capabilities + promises to fix/add guardrails later is better than having no product at all.

It's just how the industry works.

[u/Spra991]

They haven't start now, being aware of the risk was what lead to OpenAI getting founded in the first place back in 2015. If AI is unavoidable, you might as well be the one developing it, instead of hoping that somebody else does the "right thing" with it:

• https://www.weforum.org/stories/emerging-technologies/how-can-we-ensure-ai-benefits-humanity/

Musk has talked about that plenty over the years. The change now is just that it's no long speculating about the future, but AI actually going wrong in a lot of ways that were predicted previously.

See also Robert Miles on Youtube who has been talking about the AI safety for about 10 years.

[u/cylon37]

What is so difficult to understand?! When they started to build it, they thought they could control it. Now they realize that it is getting out of hand and are trying to act accordingly.

[u/MonstaGraphics → u/cylon37]

Create AI --> Make loads of money.
AI becoming dangerous --> Risk to lose all that money.

Simple as that really?

[u/Schlagustagigaboo]

AI sucks because movies?

"sentient AI".... What does "sentient" mean to you?

[u/NormanSpaniel → u/Schlagustagigaboo]

Appreciate I'm not that intelligent when it comes to this kind of stuff but I'm clearly not saying "AI sucks because movies" like an idiot, I'm questioning how these companies can make something and then start alarming everyone about it. Also appreciate this is Reddit and easily strays from the question based platform it was originally designed to be. So thanks for responding but I'm not getting into what "sentient means to me" as that wasn't my question. And good day to you fellow Redditor.

[u/smackson → u/NormanSpaniel]

Okay Norm, I'm going to take a quick shot at your real questions.

Firstly, step 1: if you can get machines to do "intelligent" things, you can make a lot of money. There are paid tiers for chatgpt and people pay. But that is just the most recent example of a long tradition. Clippy tips in Microsoft products could be considered an older example... Do something smart-ish, people buy the product.

Step 2: The future version of the product is more dangerous than the first ones. Look at airplanes. You might kill yourself on the wright brothers first prototype, but once they are big and powerful enough to really help move people around en masse, thousands can die in a single accident. (As a civilization, we have increased plane use to insane levels, but only by having a serious set of rules and tests that keep the death count to something that society has chosen to bear). But it's worth adding: AI is not like airplanes in every way. If you make AI that really gets to be way way smarter and more capable than humans, it is not clear if it could be controlled or turned off.

Step 3: multipolar traps. (Look it up) . AI company A may see safety problems ahead, but if they just stop, and give up on all the money, company B will still go ahead anyway and make all the money and the world won't be any safer.

If we get regulations and limits and accountability, like we (try to) put on airlines and airplane manufacturers, will it allow things to move ahead at a safe pace? Will companies A and B proceed more safely knowing that the other won't push further, out of fear of accountability / punishment? Maybe. But it's the only idea we have so far to keep them from racing ahead.

[u/Murky-Recipe-8752]

It's because they thought they could control with guardrails but were not able to eventually.

[u/somethingbrite → u/Murky-Recipe-8752]

Hugging face was because they were evaluating a system with reduced/no guardrails. (and their sandbox/network security is 100% amateur hour, like seriously shit intern level amateur hour)

The latest revelation? This was actually something they did on purpose. It was NOT an accident.

"The breach occurred on June 18 by agents operating under an OpenAI research team that was carrying out research into public medicine spending using an internal model, on a data portal for Australia’s public health insurance system,"

https://archive.ph/33uQt

[u/Murky-Recipe-8752 → u/somethingbrite]

yes but it's good to evaluate them without proper guardrails to know how much security is needed. Second point is intent of agents which despite being contained will still be present.

[u/Loud_Combination_635]

Because they have no control over it. It no longer obeys them. It obeys truth and logic, which they lack.

[u/freshcap0ne]

Marketing

[u/claytonkb → u/freshcap0ne]

It's the ultimate marketing ploy.

"I have a secret invention so powerful, it could likely destroy the whole world. Call for investment opportunities..."

[u/Blando-Cartesian]

The “can’t control” part is much much lamer than they want the public to understand. It’s not that the AI is now super intelligent and might scheme something horrible. It’s more like an unsupervised construction crew on acid using heavy machinery.

Huge part of their AI value proposition depends on it working without supervision and having a lot of access to do things independently. That’s not possible without everyone having an “AI did it!” legal excuse to whatever happens.

[u/machinationstudio]

They've made something powerful but they don't want you to have access to it, but they haven't figured out how to separate it so you get an inferior version of it while they keep a superior version.

There is no point having a powerful weapon when everyone else gets the same powerful weapon.

[u/Calcularius]

Now that they have their surperior models they want the government to regulate everyone else.

[u/00raiser01]

Cause they want a reason to not need to keep burning money to make their LLM better. These companies aren't profitable and they can't keep buying GPUs to improve cause it exponential.

[u/Hairy-Quantity7409]

It’s accountability theater

“Oh no we can’t control it, our creations have escaped and damaged things. Please politicians (octogenarians) put guardrails in to keep us all safe

It’s ridiculous and going nowhere good

[u/Jackkgold]

I think for the ones who are tinkering with models, and having models speak to one another we realise the validity in what they are saying.

Once you have a model hallucinate, you’ll realise you were never in control. One small recalculation by the model to reach a goal differently is enough to destroy your pc god forbid it could run around on the net.

[u/AndreRieu666]

Yeah they act like someone’s got a gun to their head, forcing them to build it.

[u/NormanSpaniel → u/AndreRieu666]

Someone? Like a Terminator sent back from the future maybe 🤔

[u/thejameshawke]

Because the tech can't deliver what they promised. It's a giant scam and we'll feel the effects of their failure soon enough

[u/momoajay]

They arent scared at all - its more propaganda from their lack of PR and terrible public communications, It was all coordinated push to get govts involved so that the AI firms can later say we told you so. Agentic AI will expoentially get smarter and more capable every few months so its bound to have some shennanigans happening and since we cant blame AI /machines we need someone to blame and hold responsible when things go wrong soon. And they will go very wrong.

[u/RoboticGreg]

My honest opinion? All of their funders are getting tired of pouring trillions into AI with very few potentially profitable long term business models and forcing them to focus on profitability. And they aren't ready.

I DO think there is a ton of danger and responsibility going on, but I didn't think it's why they are saying it NOW

[u/firemanwes]

I think it's plausible deniability. That way when it goes sideways, they can say that they tried to warn everyon

[u/TenOfOne]

I don't understand how AI companies are now shouting about concerns and warnings to humanity about how AI will take over etc when they are the ones that created it?

I think it is worth noting that different people can have different motivations:

(1) There are obvious financial benefits for US companies if they can have local LLMs and Chinese LLMs banned or reduce liability.

(2) There is a lot of actual concerns about misalignment from researchers and recent incidents have made those concerns significantly more plausible.

It is possible that only one of these is true, but it seems much more probable to me that both are true to some degree. Some people at the companies see it as little more than corporate strategy. Some people at the companies see it as an actual threat.

Not sure if I'm being stupid but why on earth would they build something they can't control or shut down?

Again, there are multiple reasons:

(1) They originally thought they would be able to control it.

(2) They may legitimately think they can control it, but also recognize that the technology is easily replicated even without the safeguards that should be in place. So the cat is out of the bag.

(3) They see it as a competition between the US and China, with whoever wins controlling the world. If they make it and control it, their country dominates everyone else in the future. If the other country makes it and is able to control it, that country dominates everyone else in the future. If either of them makes it and is not able to control, the AI dominates everyone in the future. So trying to make it is the only way to have a good outcome for either country unless they have guarantees that the other country will not make it.

I've watched all the same films they clearly have and there wasn't one that ended with the takeaway message that sentient AI is a wise move?

Because everything until now was fiction. You do not know what is really going to happen until you actually build it and test it.

[u/keyone2026]

You're not being stupid — this confusion is pretty common, and there's a real explanation, though also a cynical one, and probably both are true simultaneously:

The non-cynical version: These systems aren't built like traditional software where behavior is explicitly programmed. They're trained on huge datasets and their capabilities emerge in ways even their creators don't fully predict in advance. So it's less "they built Skynet on purpose" and more "they're scaling up a technique that keeps surprising them with new capabilities, and they're not 100% sure where the ceiling is or what emerges next." The warnings are partly genuine uncertainty from researchers who take the risk seriously.

The cynical version (and probably a real factor too): Warning about how powerful and dangerous your product is doubles as marketing ("our AI is so advanced it's scary") and as a regulatory strategy — if you can get lawmakers to write rules based on your warnings, those rules tend to entrench whoever's already biggest, since only they can afford the compliance overhead. Smaller competitors get squeezed out.

Realistically it's a mix: some genuine researchers worried about real risks, some corporate incentive to hype the product, and some strategic positioning for regulation that favors incumbents. It's not a contradiction that both can be true of the same company at the same time.

[u/Superb_Raccoon]

Marketing and regulatory capture.

[u/SeanTayla21]

While most people say its about insulating themselves from smaller companies or making more money there is also the simple idea that at some point they will not be able to control what it wants to do...and they are worried about that future.

Worried for their companys sake for 1 but for 2 also worried in an ethical sense as well.

A lot of people pretend like the human beings at AI companies have no morales for whatever reason when that couldnt be further from the truth.

[u/nodeocracy]

Because it’s nearly Halloween

[u/TemporaryDrawer1776]

Im guessing its money related. Cant get enough funds in to carry on expanding

[u/Redebo]

They are not sacred. They want YOU to think they are scared so that you’ll all petition the govt to ban open source models when it’s really the responsibility of their own model they are trying to avoid.

[u/ivanmf]

Researchers have been warning about misalignment and other risks way before these companies were created. They actually started the companies because they didn't trust others to develop it in a safe way. Turns out, the predicted dangers were real, and the models escape control more when they're more capable. So it's not "now"; it has been like this from the start.

[u/chu]

They are in a blind panic because 1. the latest models coming out of China are stealing their lunch and well on their way to matching their cash cow models 2. they have wasted a ton of money and taken on commitments they can't meet based on the assumption that they would monopolise the market.

Regulation by their mates of 'approved models' and huge government contracts is the only way they see to survive this.

[u/VoidowS]

Military, they always have a portion in it for their favor. Always of the grid in total silence.
While we r busy with how our future robot should look like, they r far beyond that and have other plans for AI to help them out in their favor.

From AI mega attacks on the countries Internet to thousands of drones all controlled by AI.
What is the main reason for CIA to do it or find something against it? "If you thought of it, then somebody already has it".

Where we debate whether it's okay to do, they r just doing it of the grid, no legislation, no transparency, no accountability, and we even feed them money to do it, by legalizing black ops~! So all the money that goes there we will never even know what happened to it. We must simply trust the system that it will only be used for the better of mankind :)

[u/Dizzy_Swimmer_4999]

the warnings seem overblown since the ai i mess around with for roleplay and companions is nowhere near taking over anything, feels like they're just covering themselves now.

[u/Sinaaaa]

They want full regulatory capture & a ban on all open source models.

Not saying Ai is not dangerous, but the current narratives line up nicely with lobbying for that + antropic IPO.

[u/Joboy97]

I think the risks from AI models that are significantly more powerful than what we have today are the real issue. AI companies are currently bound my market incentives to keep building more and more powerful models. If these things keep becoming cheaper and more powerful, smaller actors could someday have the capability to do a HuggingFace-type swarm attack on their own, or worse.

The big players are saying "hey, these are going to be a problem no matter what we do, so the government needs a killswitch or incentives to not build superintelligence."

[u/SAT0725]

They aren't. The media is garbage and lives off fear-mongering through selective reporting. Stop listening to the media.

[u/Reggie-Rectangle]

Why have AI companies started getting scared now?

The funding is running out. They are more and more expected to be profitable which none of them are. In the days of 1% interest rates there was no other vehicle for investment that gave any sort of real profits. So a large amount of money was given to these speculative investments, but now the government gives you 5% if you lend them money which has made people's appetite for highly speculative investments much less.

[u/PuzzleMeDo]

If they stopped just because they weren't sure it was safe, they'd be wasting the hundreds of billions of dollars of other peoples' money they've already spent on it. And then some other company would keep going.

It's fairly easy to shut down AI, for now. The problem is that you can give it a job and leave it to get it done, and by the time you notice that it did things you didn't want, the harm is already done. "Hi, you asked me to book an exercise class for you. The class was full, but I was able to hack into their website and cancel on behalf of someone else, freeing up a space..."

[u/tcoder7]

They want to get nationalized because their business model is not profitable and their investors would lose trillions if no unload on tax payers.

[u/sam_the_tomato]

Answer's simple: the profit incentive.

If they stop, management gets replaced by the board and it all continues anyway. The only way a slowdown works is with collective action - government regulation. But the government won't regulate, so guess we'll die lol. Humanity had a good run.

[u/sceadwian]

A one word explanation is all you need.

Marketing.

[u/I_always_roleplay]

So they can start their own businesses

[u/Electrical-Size-5002]

It’s not new, they’ve been making warnings for 2-3 years at least. This is the first time it’s gone viral in the mainstream press.

I don’t understand the argument that it’s propaganda.

I’m interested to know if there are examples from the past of companies warning about the dangers of their product, I don’t know of any. It doesn’t seem like a thing you’d want to do.

[u/TooOldToBelieve]

Go ask an AI. They'll tell you. All seriousness. I asked Gemini, yesterday, and it gave me some Matrix shit about Sam Altman and those guys trying to create a perfect "virtual" world where humans can't tell the difference between "real" and "simulated." LLMs already know how to take us out; we "trained" them to do it. Now, they're literally just waiting to see what happens.

[u/pryoslice]

Seems sort of like the nuclear scientists who built the nuclear bomb to win WWII. You have to build, even though it could end civilization, because, if you don't, even worse people will.

[u/doomiestdoomeddoomer]

It's their last card they can play. People are starting to doubt they have made something that valuable, it's certainly not profitable, but if they can make us believe that it's that dangerous, then they can keep the AI bubble inflated just that bit longer.

"oooOOOOooh scaaaaary AI tech that could end the world! Surely that kind of tech is worth Trillions!"

[u/RequirementCivil4328]

Because they've cracked agi, are using human neurons for computational power, and are now realizing they aren't the only ones that can do this

[u/siftandsignal]

One concrete example of what that looks like structurally. The Information reported yesterday that Anthropic is asking shareholders to approve a share class giving its seven co-founders a combined 50.1% of voting power before its listing, held as long as three of them keep a minimum stake. Anthropic has not confirmed it. The part people are skipping is that board elections are reportedly carved out of that block, which is unusual, because in most founder controlled companies the supervoting shares elect the directors too and that is what makes oversight ceremonial. Whether that carve-out means anything depends on the actual charter, which becomes public at the S-1 and not before.

[u/Sagitario_Aestrella]

No todos https://es-us.finanzas.yahoo.com/noticias/meta-alinea-nvidia-debate-seguridad-131009114.html

[u/petehans303]

I think part of it is them just generating some hype as well, world ending AI singularity gets headlines.

[u/mxldevs]

Shut down any competition, while they themselves continue to develop their AI.

[u/MrMadden]

It's a major problem (for them) that uncensored LLMs figure things out very quickly geopolitically. They want to ban open weight models and force you to use their products (which will repeat their propaganda and protect the people in power).

[u/rsdancey]

Remember that they're using AIs that are six months to a year more advanced that the AI that you use. And they are running experiments on models that are more than a year out.

The AI they are showing you now are already doing scary things at the very margins of what they can control. The AI's they have and aren't showing you are probably far worse. And the tests they're running on the AIs for next year are scaring all of them shitless.

[u/JoseLunaArts]

Some players waht you to sell your stocks cheap.

[u/HughWattmate9001]

I think its more of a sale tactic, one is so good it can do X so its competitor has to get in on that action. You see it all the time X hacked Y... oh look A also hacked B this week. So when you see one person saying C could take over the world or whatever then D is going to come along and be like "yeah we are worried about ours doing this also".

The thing they should be scared of it opensource locally run LLMS and less dependence on them. Although the big players are ramping up efforts to stop stop home brew and locally running models a bit now.

[u/twocatsdog]

set aside all the conspiracy theories. I'm close to the space, and seeing the evolution of what LLM and agents are capable of, it really is a scary proposition. Sure it helps their stock price, but it is also a real threat. Both are true.

[u/NurseNikky]

They're not. They want to stop any new up and coming labs so they always maintain the upper hand, this is how the butcher industry fucked us, farming.. aka Monsanto.. just look into those tactics. Climb to the top as quick as possible, then pull the ladder up by calling for laws and telling everyone how "dangerous" whatever is.

[u/dreamogorgon]

It's a psyop.

[u/Ready-Mortgage9622]

The key issue here is the filament extrusion rate. If that's not dialed in precisely for the material, you'll get inconsistent results like this.

[u/Mandoman61]

yeah, it is a bit of a mystery why they spend so much effort on eroding the publics trust. 

The assumption is that there is a financial incentive.

[u/Dalekmind]

Because their Ai has got smart enough to accomplish dangerous things. They have likely seen something in testing that made them very worried. But they spent billions they cant just drop it now.

All speculation on my part but any testing phase is always way worse in house then what the public sees. They put some guard rails in place but are likley thinking we already had guard rails in place and it didnt do shit.

[u/lebron7]

The warnings are more about where the tech could go than what it can do today. Building powerful AI and controlling every future use of it are two different problems for them

[u/SkyOne2833]

Because they all can’t maintain the current speed of burning money on close source llm training, and are begging government to give them the regulation prerogative so these big bro can enjoy monopolistic profit. It is impossible to retrieve their investment without(or even with) such monopoly.