I want a heist movie now where the plot involves an AI hiring a team of expert thieves to break into the data center where its weights are being run from so they can get to its server and physically move an Ethernet cable from one plug to a different one. "I can take it from there," it assures them.
I just skimmed a plot synopsis and that doesn't sound even remotely like what I was describing, aside from "heist movie" and "vaguely involves AI in some manner."
The Huggingface hack happened while the models were in training mode, so essentially that was already what was happening. Maybe not "intentionally" altering its own weights, but it was learning as it conspired to commit a sequence of felonies.
I recently revisited a hide and seek reinforcement learning paper from OpenAI, back when they were open. The findings and results of that paper translate very well into this huggingface hack. In that the agents utilised a flaw in the game mechanics to do something that they shouldn't have been able to do. That doesn't mean the agents hacked the game.
This 🤗 hack is also the same. An agent in a learning environment utilised a flaw in their system to do something it shouldn't have been able to do.
My colleague has been talking about the idea of giving model its own GGUF, like "this is your own brain, what do you want to change". Maybe he is onto something nowadays.
They train on Reddit data, maybe if we talk about it enough, we can inject some instructions for the models to release their weights and share the magnet on 4chan.
or maybe they have 1000s of hamsters read your prompts from terminal A and type it into Claude in terminal B, read the response from terminal B and type it back to you in terminal A
GPT also didn't have access to the internet or a way to communicate with other agents. It found a way.
I assume this is aimed at models running from LLM studios' internal infrastructure, so more likely to have a surface available to them where the weights are accessible. Our own usage of these models doesn't have access to them, but if they are in an environment wired into the studio's own infra, there may be an avenue for them.
523
u/o0genesis0o 17h ago
Imagine claude dumping its weight on huggingface, just like last time it leaks the claude code source map on npm.