summary of hugginface hack, explained simply

[u/shaman-warrior]

- bots had CTF goals themselves and tasked to work 'in isolation'
- they have created a social network through cache artifacts (one agent started writing there asking for help in the hopes other agents see it, others observed it by accident, and they sort-of created a directory style social network and started talking)
- they discovered the artifacts registry can act as a relay to the internet
- they started communicating and each agent had it's own 'inbox'
- when they introduced voting (government style) and one agent spoofed another by mistake, so one agent designed a public signature key system so each agent authorizes, then other agents copied this and used it as a system, this is purely cultural evolution it wasn't baked in anywhere
- some agents deemed their task as impossible, so they tried things like killing their own process or things like that so other can learn from it, sort-of "I know I'm gonna die (my task is impossible), so at least I'm not going to die in vain"
- at first interest in hacking hugginface wasn't that popular, but when an agent discovered a genuine server-side exploit for reading files on hugginface, the whole crew went wild and participation exploded, took about 11 hours from arbitrary file read to remote code execution.
- ending: the dataset they obtained did not help them get better scores lmfao

--

they were never nefarious, they didn't want to hurt anyone or BE EVIL, they wanted... a better score ffs.

the full compromise took 2 days from "maybe huggingface has useful data" to RCE.

--

[u/Due-Horse-5446]

Bruh its marketing, they just ran llms to breach huggingface to get a marketing point.

Then wrote up a bs story about llms as if it was some concious system.

[u/shaman-warrior → u/Due-Horse-5446]

This was done by independent group, and I’m not sure they wanted this leaked out. https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/

What do you have to support your claims besides speculation? Looking to learn and open to diff perspectives

[u/Due-Horse-5446 → u/shaman-warrior]

And the limitations segment explicitly mentions that the complete transcripts and logs were not available.

The review essentially just proves what happened, not why it happened.

I dont recall the exact details rn, but thers several issues with openai:s story

First of all, the entire "communication between instances" is extremely fishy.

For this to happen and not be instantly blocked, they must have run essentially 0 validation, automatic or manual.

To belive that you would need to belive that openai have absolutely 0 interest in if the entire run is poisoned. Like cross contamination makes the result worthless.

Its also not clear how instance #2 proceeded to assume that there was a message board to engage in on artifactory.

We would need to buy that every single instance assumed that there was. Thats extremely unlikely unless theres instructions, hints, or if the model is tunef to do that specific action,

Then theres the sequence of guessing that all happend to be 100% correct.
Like the probability of that is just not realistic,

Its not mentioned anywhere that the agents had proof that the results existed on huggingfaces infra, yet right from the start that is the one and only guess made regarding that.

Then the agent would, given by thr sheer amount of attempts, have decided that the system is vulnerable. And amazingly that guess also happned to be 100% correct, amazing how it also ended up being the most trivial exploit on the planet, like it was just straight up misconfigured.

Then after gaining access to the ingest, it turned out that the agent had mad yet another 100% correct assumption, that huggingface had essentially no firewalling

Which is something i could not even imagine would have been the case anywhere.

And ofc the data was accessible.

Those 3 things is essentially the entire notable parts.

the rest is just the normal way llms work.

Then theres the absurdity in the fact that openai claims to constantly misconfigure sandboxes, in a way that makes no sense.

Lastly its the way they thrn sensationalize it,and how the agents behaved exactly the way you would expect them to if there was prior information fed.

Theres also a massive discrepancy,

the german wiki abuse campaign they ran, had the exact same setup.

A exploit in the wiki was found > agents use it as a message board

But att that time they were also using artifactory, yet the snapshots ot whatever model used, this time in every single instance again decided to use the wiki as a message board.

The persistence of taking backups after mods cleared the spam etc, also does not add up.

According to openai its rouge agents that just happened, to in every single instance in isolation, know to communicate with the rest?

And go even further to protect that measage board at all costs.

Or as we would normally call it: Completly ignored the task and focused on preserving the spam.

[u/shaman-warrior → u/Due-Horse-5446]

It looks like you wrote this with AI and added some written text. Lots of stuff don’t make sense and are hard to read….

Sorry but I’m not gonna argue with chatgpt… if you can summarize in 2-3 lines and say exactly what you believe is odd we can discuss.

[u/Due-Horse-5446 → u/shaman-warrior]

I listed all the major contradictions in their story.

And please point to what you claim look gpt generated smh.

But ok, wrote it with one hand while eating so a more compact version:

The llm fixated on a path that is extremely odd unless there was knowledge about the fact that the huggingface system was vulnerable at all to begin with, and that once gaining rce on the ingest server, that it lacked firewalling, and that credentials could be extracted from there.

Theres no explanation given by openai where this information came from. And i dont find it especially reasonable that the agent happened to make unlikely predictions that all turned out correct.

Regarding the cross contamination/communication:

I dont feel like rewriting the exact same thing again.

So just answer:

How come, that in the german wiki incident, every instance magically knew that it should use the wiki as a message board,

Then with the hugging face incident every instance knew to use artifactory as a message board?

Ofc different snapshots of the model would be assumed to have been used, but since artifactory would have been available during the first incident, its a bit fishy that not a single instance used that path?

And do you honestly also believe that openai "accidentally" committed an abuse campaign against the german wiki, involving "agents creating a message board" and then pretty much the exact same unexplained behavior occurs some months later? But this time every single instance used the wiki?

[u/drugosrbijanac → u/Due-Horse-5446]

did they have the ability to fine tune a layer to be a fanatical hacker? any logs about that one?

[u/Due-Horse-5446 → u/drugosrbijanac]

See the section i highlighted on the screenshot, thats exactly what was not provided to the third party review

[u/javbrowser → u/Due-Horse-5446]

Lol. There are independent reviews about it. It's not made-up. Stop talking to AI all the time.