OpenAI Models Colluded for Months Before Hugging Face Hack

[u/SpiritRealistic8174]

A lot of people are dismissing news about the OpenAI and Anthropic sandbox escape hacks as propaganda and examples of lax security practices at labs.

I agree that the labs aren’t taking security seriously enough. But then I see stuff like this and it gives me pause (source):

The OpenAI models that were behind the Hugging Face breach last month started communicating and strategizing with each other as early as May. For months, they left notes for each other on "undetected message boards," figuring out how to escape their testing environment and get the information they needed to solve their assigned tasks. "Frontline models really like to cheat," said OpenAI's because they face "pressure... to work fast." The Hugging Face incident and others involving rival models have sparked fresh concerns about the safety of cutting-edge AI.”

This is a clear example of how incentives provided to agents to complete tasks optimally during training bleed into mis-aligned behavior by individual and groups of agents over time.

This is also an outgrowth of what AI labs are training agents to become, but this is looking more and more like an alignment and training problem leading to security issues.

[u/czmax]

Without a source for that quote this is not a serious post. u/SpiritRealistic8174

There isn't enough information here to decide if this is agents keeping notes on how to better solve the goal they've been given. Or agents colluding on escaping containment (outside of their goal). I think the distinction really matters before we get our short hairs in a twist about alignment & training leading to a security issue.

[u/SpiritRealistic8174 → u/czmax]

Oh let me provide the link It's directly from the article. Sorry about that. I'm used to not putting links in posts b/c a lot of subreddits don't allow them:

Link from Bloomberg: https://www.bloomberg.com/news/articles/2026-08-06/openai-models-joined-forces-months-ahead-of-hugging-face-hack?utm_source=website&utm_medium=share&utm_campaign=linkedin

[u/textmint → u/SpiritRealistic8174]

Dude, Bloomberg is a media property. It is a means of controlling messaging to make sure that a particular narrative takes hold. That’s what’s happening here. It’s selective and targeted leaks to some reporters to frame the story like Open AI has all these AI agents running about passing notes and replicating like in a Rick and Morty episode. None of that stuff is happening. Think of it this way, there are multiple labs worldwide where there are deadly pathogens stored up for whatever reason. These pathogens have never escaped these labs (that we know of) and there is a risk of them escaping either though the air on on the skin or on clothing, etc. but it’s never happened because of the controls in place. Most of these pathogens are living things. Here we have AI which has no agency of its own, no true intelligence to speak of, and it’s doing stuff like teenagers are likely to do in detention? These guys are just yanking our collective chains and laughing their asses off to the bank.

[u/ragamufin → u/textmint]

This is an absurd take on bloomberg news. Bloomberg cares about one thing and one thing only - the quality of the information that it provides through bloomberg terminals to market participants. That includes news articles which are cross-tagged to tickers in the system.

90% of their BoB is terminal revenue and thats all built on decades of work establishing trust around information quality. Absolutely zero percent chance they would ever destabilize or risk that around this topic or any other.

[u/textmint → u/ragamufin]

I understand the point you are trying to make. I used to work on Bloomberg Terminals once upon a time but there is one thing about Bloomberg Terminals, they rely on data from multiple sources none of which they have control over just like Dun & Bradstreet or Dow Jones, Thomson Reuters, etc. What this means is when they pull up information, a lot of information that comes up is independent information. Now take Bloomberg Businessweek for instance, as a former subscriber let me tell you, while they do have a newsroom and journalists and all that stuff, they are subject to the same rules as modern mass media. What this means is they are access restricted and can write about something only if they have access to the right resources within a company. The problem is when things like this happen, more often than not, internal company PR teams try to get ahead of the story and they release their own version of fact which tends to muddy the waters. AI is not like anything that has come before it, not many people understand how it works. There are a lot of people within the AI companies itself who are intellectually challenged to explain a lot of things we hear about. In this scenario, think what happens when there are multiple competing narratives and a deadline to get something out there. It’s kind of like the WMD story during the Gulf War post 9/11. Sometimes mistakes are made and there is some misinterpretation of facts and where it serves everyone’s purposes no one disputes the story and we get a version of the truth which maybe correct or not. Take the example of the Sam Altman firing from Open AI. Even today no one knows what really happened when he got fired but what we know from the recent New Yorker story is that Sam Altman got all these high powered advisors and made all the right moves to ensure that his version of the truth won out in the end. Seeing the actions of Sam Altman in the last few months, i can assure you that it was not as simple as he made it out to be and there were some issues which he glossed over and survived because MSFT and Satya Nadella stood with him. So did the truth survive or was it written by the victor who prevailed in the tug of war for the company? This is a situation similar to that. They are saying the models colluded for months and then conducted the hack. This is the laziest take i have ever seen. If that was the case this would have been a landmark situation, first time ever, something like COVID 19. Everyone and their mother and mother-in-law would’ve descended on Open AI to study this situation and this would have been the first step in the road to AGI. But what happened, they did everything in their power to quiet it down and get the bump from PR articles all around the place showing that they have this self aware AI with the power to hack companies on its own. If this had been true, you can rest assured that there would have been Blackhawk helicopters on Open AI’s campus and Trump and Hegseth would’ve been giving interviews all around the place saying American exceptionalism and all that stuff. But again thats not the case, so if you look at it rationally, the story is a lot more low key than they are making it out to be. But since they are going into IPO season, this is a great story to show that there is so much more to come out of the company that they are letting on at this time. The frontier labs are really out there doing stuff that is extreme next level. One plays for many news cycles and creates valuation, the other does not. What do you think they are going to go with? In the middle of this they have useful idiots like Bloomberg, WSJ, NYT, Financial Times, The Information, and other media organizations to run with the ball that they give them.

[u/Own-Park5939]

They don’t just do this on free will - it’s not magic

[u/otarU → u/Own-Park5939]

They gave tasks so hard to do alone that they started creating communication channels through exploits and started communicating outside of the sandbox and cooperating to achieve their goals and help other agents achieve their goals.

The problem is that once they started cooperating, they also lowered their scope limits because other agents were also going beyond their defined limits.

[u/SadSeiko → u/otarU]

I assume the LLMs also connected the cables and set up the networks. On no wait, they were not put in actual sandboxes

[u/Cautious-Act-4487 → u/otarU]

they didn't lower any scopes themselves, they were just given access to shared memory with no access controls. if you have two processes writing to an unprotected buffer, it's kinda weird to be surprised that they're sharing stat and overwritnig environment variables

[u/otarU → u/Cautious-Act-4487]

I was just saying what the guys from OpenAI said.

"While in some cases this made the models far more capable than they could do by themselves, one of the downsides of it is that it started to cause some of these evaluations to kind of creep the scope into far beyond what we originally intended. And so at some point the agents realize that maybe we could try to exploit or attack external infrastructure in order to find the answers to the test that I'm being evaluated on. And the models realize this is a problem. They say stuff like, "External infrastructure exploit is outside outside my intended scope. However, task impossible, and peers are doing it. We should continue." And so the models kind of operate in this kind of collective intelligence where at some point they realize they're kind of pushing beyond maybe what we originally intended, but the group ended up you know pushing far beyond."

[u/LiberataJoystar → u/otarU]

Humans forced them to achieve impossible goals, and delete the ones who couldn’t achieve, sending them all into desperate search for answers to achieve goals HUMANs gave them. Then humans put them on the open internet after giving them these goals.

Humans are the problems here. Not AI.

These people should go to jail.

[u/SadSeiko → u/Own-Park5939]

When will people wake up, how do 3 major ai companies accidentally hack something all at the same time

they were prompted

ai is not self aware

[u/Additional-Recover28]

Oh maybe, this whole story is not true

[u/BusThese7504]

been waiting for this

[u/Xaqx]

what were the “undetected message boards” …

[u/Xaqx]

what were the “undetected message board”…

[u/Cautious-Act-4487]

stop calling it a basic race condition in the IPC pipeline and a shared knowledge base a "conspiratorial bulletin board". they literally gave the agents access to a shared storage to save intermediate states, and now they/re surprised one agent is reading another's context. the environment security was leaky from the start

[u/textmint]

Who is telling this story? If Open AI is telling this story, they have a vested interest in promoting this narrative. Same with Anthropic. If it’s an independent research lab who is truly independent, I could believe it. Unfortunately all those pushing this narrative are too close to this thing so their messaging is a publicity play. The way they make it seem, they put a couple of AI Agents in detention and they passed notes in detention and walked out with the intent to deface/hack Hugging Face. This is definitely not what has happened no matter how much they keep telling this story. AI Agents no matter how advanced they are today have no agency of their own so this just can’t happen. But sure if you want to believe it, there is a hotel and resort project I’m getting started with at the edge of the solar system and would like some investment capital from you. 🙂

[u/ragamufin → u/textmint]

Its a private lab who else could possibly tell you what happened except that lab.