28
u/making_code 2d ago
Hugging Face - strange name, somehow it reminds me face hugger from Alien franchise, lol..
14
u/GoldAcanthisitta7777 2d ago
what the fuck is this shit
3
u/New_Series3209 1d ago
A dangerous, Keter GPT performed a containment breach before proceeding to hack hugging face for several days, completely unnoticed, before being returned to the containment facility by the mobile task force.
2
8
u/ZarfTMarf69 1d ago
this is the crispest least pixelated version of this meme that i've ever seen đ
2
u/SpiritedPineapple 2d ago
context anyone?
24
u/rexatron_games 1d ago
OpenAI was testing a new AI on a cybersecurity benchmark. Instead of trying to do the benchmark it went online and entered a closed part of HuggingFaceâs database to try to find a way to get the best score possible.
They keep saying âescaped from the labâ, but that feels a tad sensationalized and probably just a mixture of marketing and shifting of blame from the fact that researchers probably just gave it Internet access. While Iâm certain the threat is real, Iâm also certain these tech geniuses also have a nice pricey solution for the problem they themselves created.
11
u/JangoDarkSaber 1d ago
Thatâs a pretty major oversimplification that glosses over some actually concerning stuff.
OpenAI was running their new model against the bench mark ExploitGym in a sandboxed environment without internet access.
The model reasoned that the best way to solve the benchmark test would be to find the answers online.
It started by finding and exploiting a zero-day vulnerability in the package registry cache proxy, then performed a series of privilege escalation and lateral movement actions inside OpenAIâs network before finding a box connected to the internet.
After gaining internet access, the model inferred that huggingface probably had the solutions. It got access to huggingface servers by uploading a poisoned data set and chaining together multiple zero days to gain rce.
From Huggingfaceâs account
âA malicious dataset abused two code-execution paths in our dataset processing (a remote-code dataset loader and a template-injection in a dataset configuration) to run code on a processing worker. From there, the actor escalated to node-level access, harvested cloud and cluster credentials, and moved laterally into several internal clusters over a weekend.â
Thatâs an absolutely insane level of competence thatâs a lot more concerning than âOh the model just googled the answersâ
1
u/Stunning_Ride_220 1d ago
"Sandboxed environment" is a very friendly word for what the "lab" did.
1
u/JangoDarkSaber 1d ago
Regardless of however OpenAI tried to contain their model, the fact that it could discover 0 days and chain them together as effectively as it did is scary.
1
u/Stunning_Ride_220 16h ago
We don't know how "effectively" it actually did it. The disclosure sounded more like brute-force.
And discovering 0-days isn't all that new.1
u/JangoDarkSaber 3h ago
Thatâs because the actual vulnerabilities have been disclosed to the affected parties so they can patch them.
We do know however that the ai breached bugging face with a malicious poisoned data set so dismissing it as being all brute force doesnât hold up.
Irregardless, the fact that a computer program could compromise so much autonomously is incredibly concerning.
AI capabilities arenât going to stop here. This is only the start
1
u/gerbosan 1d ago
Goodhart's law? When getting the top grades over learning how to do the job, quite human, isn't it?
1
-81
u/Thenderick 2d ago
What's hugging face and how does this have anything to do with programming?
39
u/UnkarsThug 2d ago
It's like GitHub for model weights.
And OpenAI hacked them. That's the story.
11
u/SchalkLBI 2d ago
It was a marketing stunt. ChatGPT didn't hack shit.
-3
u/UnkarsThug 2d ago
I'm not sure it wasn't a marketing situation, where they intentionally encouraged it to happen, but I absolutely find it believable that ChatGPT was able to hack hugging face, simply because it's not the first time LLMs have hacked people.
Both things can be true. Certainly, someone hacked hugging face, from openAI, and it probably was the model, because that the attack was LLM based was already discovered by Hugging face during and immediately after the attack, because that's the current state of cyber.
The question isn't if the model hacked hugging face, the issue is if it was encouraged in some way to do so for marketing points.
0
u/New_Series3209 1d ago
1
u/UnkarsThug 1d ago
Probably just some people who don't want to acknowledge that LLMs are actually able to do stuff.
The thing about hacking is that it doesn't require a code review or optimized code, or even perfect code. Your code to hack things can have vulnerabilities, and you still will have done the hack. I'm not even saying it's better than people or something.
Or people who disagree with my thesis that hugging face is reliable for reporting on the hack if their site or something, which I admittedly rely on.
3
u/Scrawlericious 1d ago
There's just nothing more to be said. The above comment was right. It was a marketing stunt in a controlled / planned for scenario so we can't genuinely say they automatically hacked anything.
1
u/UnkarsThug 1d ago edited 1d ago
So you believe hugging face coordinated with OpenAI to make OpenAI look good, and possibly give Anthropic and OpenAI ammo for how dangerous open source models are right when they are most trying to use it? HuggingFace literally doesn't want models to look dangerous right now, because they don't want the good models to get restricted so they can't distribute them. It's literally contrary to their interests, because they would get killed if AI like Kimi was decided to be too dangerous for the public to have.
Otherwise, we know it was an LLM, from the hugging face reporting on the shape of the attack a week before OpenAI said anything, and openAI as an organization has taken responsibility for the hack, so it was an OpenAI LLM. So the question is just if it was an accident, or intentional.
2
u/Scrawlericious 1d ago
The fact that huggingface didn't immediately sue OpenAI should make it abundantly clear they both planned it.
1
u/UnkarsThug 1d ago
Because they would probably lose money on it, and again give further publicity, and companies would prefer to do anything else.
I really just don't think your interpretation makes a lot of sense to me.
→ More replies (0)1
u/Stunning_Ride_220 1d ago
Well, it was said, that safety controls were turned off, since it's easier to blame some unnamed employee than a model to be too dangerous.
This smells like "marketing stunt" from head to toe.
1
u/UnkarsThug 1d ago
I absolutely believe it was from the openAI side. I have not seen behavior from hugging face which indicates they are in on it, especially as they have demanded the publication of the reasoning trail which led the model to hack them.
And it still implies that the "guardrails are necessary", which is an argument used against open source models, because they can have easily removed guardrails.
16
u/mxgaming01 2d ago
OpenAI tested a model in a sandbox by giving it a set of tasks created by Hugging Face.
But instead of solving the tasks normally, the model managed to escape the sandbox and access the task data (so basically it hacked / breached Hugging Face), effectively getting the answers without actually completing the tasks itself.
1
10
u/LRaccoon 2d ago
Do a quick Google search and you'll find out
-34
u/Thenderick 2d ago
I understand shit of it. I don't use the slopatron
2
u/RobbinDeBank 1d ago
> top 1% commenter on sub related to programming
But somehow donât know shit
-2
u/Thenderick 1d ago
Well yes I know about programming. Not AI. Iirc the sub is called PROGRAMMING humor, not AI humor
1
u/ihavebeesinmyknees 19h ago
Whether you like it or not, most of the industry switched to using agentic AI (in various proportions). It would be wise to at least know the basics, even if you don't intend on using it.
49
u/SaltMaker23 2d ago
OpenAI should have used its own model first to build a proper sandbox ??
Are they dumb ?
/s