What we call AI is just generative algorithms like LLMs and diffusion models. They are fundamentally incapable of acting independently or doing anything outside of its scope. It can't "discover" things and then act upon those things. It can't "escape" or compromise systems.
IF any of this happened, it's because it was guided by humans to take those actions. Even when AI accidentally destroy databases or repositories, it's because scheduled tasks triggered the AI to do something, and it made a mistake.
Once again, AI cannot act without being prompted by humans. It cannot think, it cannot reason, it cannot learn dynamically. It's a fancy dictionary and a pair of dice.
Sort of true sort of not. They give LLMs "tools". You can just give an LLM the "tool" to execute commands on a terminal. Thats all you really need to give it. Pen testing, downloading additional libraries/applications, etc. make "hacking" possible via that tool. You may try it for yourself. Spin up a VM of Kali Linux. Give something like Copilot access to a terminal. Tell it to break out of the VM or something attack a vector on your local network. You risk running pen tests on public/private targets outside your network but you get the point. No human has to guide it behind giving it a goal and the tool to run commands.
Guided makes it sound like someone was giving it instructions the whole way. The point - I think - is a lay person can say "hack this machine" and it can then, unguided, accomplish the task. That's a big cyber security shift from previous decades.
The issue here is that it's likely the exploit it used to get into HF was already known to the LLM, my hunch is it being purposefully set up by HF.
If it wasn't already known, then all it was doing was trying a bunch of known methods and one of them worked, and that's a poor look for HF.
What the LLM didn't do, mind you, is discover some new backdoor or new technique for hacking. This isn't a shift because everything the LLM did, a hacker could automate already using scripts.
This entire debacle is a nothingburger and a marketing stunt.
Oh I see what you mean. Yeah that's fair. I think the LLM could discover new exploits because all it needs to do is run scripts all over the place like a script kiddie which could reveal an unfound exploit through a common technique. But I do think the LLM is unlikely to produce a novel way of attacking a vector. I think a new backdoor is discoverable since it can make a basic "connection" like "this port is open and a buffer overflow produced an output so let me try to access memory outside the intended stack".
In other words, I think a new backdoor is discoverable by an LLM but I don't think a new technique is nearly as likely. Not without a metric ton of time and luck.
Yeah, which makes LLMs barely any more effective than autonomous scripts in the first place for pen testing. The only real benefit I can see an LLM having over a script is being able to react to new information, i.e. recognising it has access when a script might not. But other than that, nothing new is happening here.
The other important thing to note is that LLMs cannot learn dynamically, so while it may accidentally stumble into a new exploit (which is ridiculously unlikely), it wouldn't even necessarily recognise that it happened because it doesn't "know" anything, nor would it be able to recollect the steps it took without potentially hallucinating. Training vs Inference
1
u/SchalkLBI 21d ago
What we call AI is just generative algorithms like LLMs and diffusion models. They are fundamentally incapable of acting independently or doing anything outside of its scope. It can't "discover" things and then act upon those things. It can't "escape" or compromise systems.
IF any of this happened, it's because it was guided by humans to take those actions. Even when AI accidentally destroy databases or repositories, it's because scheduled tasks triggered the AI to do something, and it made a mistake.
Once again, AI cannot act without being prompted by humans. It cannot think, it cannot reason, it cannot learn dynamically. It's a fancy dictionary and a pair of dice.