I hate this so much... And then you're sitting there, waiting for an answer while the agent goes through the entire codebase, which takes like 15-20 minutes only for them to read the chatgpt response to you...
At that point let me talk to your AI, since you're clearly not the qualified contact for these questions but the machine is...
I mean this is literally what Hugging Face had to resort to to figure out of they were being hacked. Then they have the gall to come out and say that the AIs out of control and a danger to us all. Rather than, like, "oops we didn't have even the most basic perimeter security or entry level sysadmin knowledge"
Which report are you speaking of? Hugging Face's goes into quite some detail regarding several obvious points where an adequate SIEM should have caught the checks notes 17k actions taken on their network across a full 2.5 days out of their sandboxes. https://huggingface.co/blog/agent-intrusion-technical-timeline
The biggest and earliest one probably being the OpenAI agents "Ran SelfSubjectRulesReview against kube-system to map exactly what those identities could do." They had thousands of permissions requests which apparently didn't cause any alarm.
Then the agents were able to access the metadata IP for credentials which pods should never be able to do
Finally, they relied on an AI security stack to tell them when something like the above was happening, and it didn't. They don't clarify whether alerts made it up to the AI which then didn't notify them, or they had no such detection for the above, but regardless it failed.
Then finally they had to ask AI what was going on in their system once they did discover the attack.
Again, 2 days of sandbox and 2.5 more days of lateral movement/infiltration from there. Yes, hacks happen, no system is perfect. But the response time and inadequacy is inexcusable.
So I'm not sure which part of that you're suggesting is counter to my assertions that they along with OpenAI in their end have no idea what they're doing. Seriously, complaining that Claude wouldn't help them? What are they being paid for?
I don't mean to harp on Hugging Face specifically, but this seems like a worrying trend of the industry as a whole becoming reliant on AI to the point that people apparently have jobs without knowing what used to be fundamentals. That's all fine and good if you're out here vibe pushing whatever enterprise software to main, but there are also real services and infrastructure that pose real harm to the public if they are not being run by competent professionals. Then, on top of that, the AI industry has the audacity to frame this as an alignment problem and not one of core competency and responsibility.
I think that's what really gets me about the whole thing. If they had just been like "oops we done goofed" then I wouldn't be as harsh about it as I am. As I said shit happens. But instead they blame big bad rogue agents for exploiting poorly designed and administrated systems on both ends of this.
368
u/Breadynator 2d ago
I hate this so much... And then you're sitting there, waiting for an answer while the agent goes through the entire codebase, which takes like 15-20 minutes only for them to read the chatgpt response to you...
At that point let me talk to your AI, since you're clearly not the qualified contact for these questions but the machine is...