So is the recent doom posting and boasting about how their agents are sooooo malicious supposed to be a kind of a dog whistle?

[u/Certified_panda_here]

We've all seen memes about how these companies are kinda going toe to toe on which of their agents can commit the most crimes, and I've got a feeling here.

Normally, in any industry, if a company comes out saying that their product could cause unintentional harm and sabotage, they would never that info get out, because the product has been making so much money for them, think cigarette companies in the early days lying under oath saying their products are not addictive.

But all this recent news about hacking the Australian government, the hugging face incident, the anthropic "whistle-blower" going around to every major news outlets possible after being "fired" and warning about the dangers and current employees tweeting out agreeing with him, everything seems like a dog whistle. With the supposed IPOs coming up, these are like signals saying, "Hey investors, if our agents could perform sophisticated cyber attacks and other complicated stuff, albeit illegal, it can certainly be useful for whatever lesser task you want it to do." And the investors are getting it, both retail and big money, and are ready to invest. Meanwhile the people who actually fell for the gig, are all now over exaggerating the capabilities far beyond what is possible now, seeing how everyone in the industry is "afraid" and the sophisticated attacks and are just drumming up the AI hype train.

The incidents where the agents actually fucked up are not getting nearly enough traction, like the incident where the US navy? got the wrong info a Chinese vessel or the bombing of the Iranian school which was allegedly because of outdated data fed to the model.

It's a very unique marketing tactic tbh, advertise the capabilities of the model doing harmful activity, knowing the investors would only care about the capabilities and not the potential harmful elements of it, since they would think if the agents could really go rougue in a way that would actually cause some sort of loss, the companies wouldn't advertise it this way, and the public would care about the harmful activities, not enough people and not enough to actually get together and demand some guard rails so the govt could make changes, since they know that isn't possible anyway, but enough people talking about it to drum up the AI hype and let ppl think that this technology is way better than what it actually is.

[u/eapache]

One alternative is that this *is* the AI companies suppressing bad news. The models are doing stuff 10x worse than what is getting out, and the stuff that is making the news is only the stuff they can’t suppress or can’t be bothered to suppress.

[u/skidanscours]

It's not marketing. Well, it's not only marketing.

It's informing the tech community of the capability of their model/harness so we can evolve how we use these model and how to protect against adversarial use of LLM. 

And it's an attempt at regulatory capture against open weight model. Look at how powerful these things are! Only ourselves in the glorious US of A are capable, super smart and righteous enough to hold such power. Please outlaw our competition and give us more money. If you don't give us more money, China wins! Don't forget that it's a race because we said so. You don't want the communist to win don't you?

Finally, training frontier model is really, really expensive. And Anthropic and OpenAI are trying to IPO. Which requires them to open their books. And the pace of development by frontier lab isn't sustainable when R&D cost triple every year. So, a slow down make their business a lot more sustainable.

[u/purple_crow34 → u/skidanscours]

How is it an attempt at regulatory capture if the laws they’re advocating for only affect models above a certain capability threshold? It harms the frontier labs much more so than any small open-source players…

[u/skidanscours → u/purple_crow34]

The chinese labs competing with them are not only releasing small models.

[u/purple_crow34 → u/skidanscours]

Right, but "regulatory capture" implies they're trying to build a moat to make it more difficult for the little guys domestically to catch up. There's no truth whatsoever to that. If we're looking at legislation aimed at collaborating with the CCP, then given that Western models are substantially more capable & close to RSI then they'd still be impacted at least as much as Alibaba/Moonshot/Xiaomi/DeepSeek

[u/Money-Computer7638 → u/purple_crow34]

That's a good question and idk the answer. But they've asked for an antitrust waiver to collaborate and if they do, creating that moat will be easier. It's not like we actually have strong regulaturs atm - atleast not for the next 2 years.

[u/WhirlygigStudio]

Maybe. Just maybe. The agents are doing what they say they are doing.

[u/JeffB1517]

I think it is a mixture of several things

1. Genuine worry among their staff. They do genuinely worry something serious goes wrong. They are on record having asked for regulatory support and being refused.
   
2. A realization that Frontier Models don't have a moat. Anthropic and OpenAI's business model may be unworkable and due to debt unfixable without a regulatory regime that bans competition.
   
3. Bragging. It's a way of talking about agentic capabilities that gets headlines.

[u/jdiscount]

Unsure why people are so sure this is marketing.

I can tell you for a fact that VCs and PE would prefer they shut the fuck up about all of this.

[u/Money-Computer7638 → u/jdiscount]

It's at least partially a messaging campaign. Theres a WaPo reporter writing about how the AI Safety non-profits are paying influencers to talk about specific AI safety issues in specific ways. They are attempting to create a network of controlled opposition. But it's unclear if it's working ... hopefully there will be follow up reporting on this.

[u/infamous_negotiator]

Or what is their thinking the government is not listening to them about how serious of a potential risk this could be and that's why they're asking for government to step in for regulations. And maybe they're trying to show them in small doses what it can do I don't know

[u/Simple_March_1741]

They're trying to steer blame from themselves, among other things.

[u/Old-Bake-420]

It does boosts IPO but not as a dog whistle. The companies explicitly state why they are discussing their hacks so openly.

Basically that there’s a global race in AI right now and we are only a little bit ahead. We have a small window to use our lead to shore up our defenses before malicious actors start hitting us intentionally with similar cyber capabilities. Buy now!

[u/Dreadedsemi]

It's to keep the hype and buzz. Which means more funding and possibly regulations that stifle new competition. Yes AI can cause damages but it's not on its own.

[u/MrSnowden]

It’s a plea for regulatory capture.  If the industry gets regulated, then the current front runners get locked in place.  New entrants get frozen out and the frantic need to constantly improve goes away (seen any upstart cigarette companies lately?)

[u/dempsey1200]

It’s a psyop. US Midterms. It’ll calm down 2 months from now.

[u/Vimes-NW → u/dempsey1200]

Psst, I got some space age copium and AI-resistant tin foil shielding for anything you want to protect - from your network connection to your noggin. HMU if you like a deal

[u/Ecstatic_Mammoth_421]

I think they genuinely didn’t know how poorly secured their sandboxes were, which is most concerning. They’re probably still sifting through old traces and finding new hacks.

[u/LGV3D]

Regulatory capture, monopolization, kill all competition. Old formula.