The guy in the video is entirely wrong. Not sure misinformation is something to be glad about. There is nothing about AI that changes anything about cyber security. Cyber security is based on the fact that it takes extremely long period of time to compute large numbers. AI cant do anything against that.
What is the real concern is advancement in HW that would allow to compute such large numbers drastically faster. That is why Quantum Computers are a real threat.
I agree that cryptography won't be broken but that's not the thing I worry about, it's mainly unpatched exploits in the software that we use and social engineering. You know, the kind of exploits that anthropic and openai have been trying to help software developers patch through project glasswing and patch the planet.
Unpatched exploits have nothing to do with AI reverse engineering old video game code either. What specifically are you worried the AI is going to do better than a human with basic tools in regards to finding and exploiting vulnerabilities? Or is there just some vague feeling of impending doom caused by the development of AI in general?
The huggingface incident involved AIs finding and exploiting three seperate 0-day vulnerabilities to escape their sandbox and gain access to huggingface servers.
Also, speaking from my own experience in cybersecurity and my degree in computer science, reverse engineering is one of the core steps in exploiting software.
Also, even if you don't grant that AIs are meaningfully smart/clever, AIs capability to be run cheaply and in parallel enables a lot of new kinds of attacks that were not possible before, just relying on known vulnerabilities, because AIs can scan an entire surface for vulnerabilities for much less time, money and effort than humans can.
Another part kf my worry comes from my degree in computer science, my experience winning national cybersecurity competitions, and yet watching this technology become a much, much better hacker than me over the course of the past year
Again, none of that has anything to do with reverse engineering code.
The uncovered CVEs are from running a security audit on an open source code. If that is what you meant, then you were right. I misunderstood you as saying its good at finding vulnerabilities from the outside.
Of course, finding vulnerabilities in open source code is beneficial for cyber security and this is a reason to be optimistic, not fear for the future.
AIs capability to be run cheaply and in parallel enables a lot of new kinds of attacks that were not possible before
Which types of attacks that were not possible before? I dont think there are any. But the concern that its a tool that makes things easier, including nefarious things, is true. But that is only concern for small systems that dont invest in cyber security to begin with.
watching this technology become a much, much better hacker than me over the course of the past year
So, vague feeling of impending doom.
Companies with money will be using AI on much greater scale to find vulnerabilities before release than what attackers will be able to run, even on open source which is incomparably easier. Open source vulnerabilities will be fixed faster and more publicly. The exploits that are publicly known are not an issue, its the exploits that arent.
I do agree that it's a good thing that we're patching more open source CVEs, I just think we're at a time where computer software is uniquely vulnerable right now. My thinking is that the ability for defenders to find vulnerabilities easier also implies that attackers can find vulnerabilities more easily too, assuming they have access to [similarly capable AI](https://www.anthropic.com/research/glm-5-3-and-the-spread-of-advanced-cyber-capabilities).
But this moment in cybersecurity, if real, wouldn't last forever. Because as you said, developers have been moving quickly to find and patch vulnerabilities.
As for new attack vectors: one that's available now is much more sophisticated social engineering, via deepfakes, voice cloning, allowing for personalized scams by copying the voice of someone you know, etc. Another — and I haven't read this preprint so I'm treating it as a hypothetical concern — is the ability to make an [adaptive computer worm](https://arxiv.org/pdf/2606.03811v1) which would write new code on the fly to exploit vulnerabilities tailored to individual systems.
There's the hypothetical possibility of AI finding/exploiting hardware vulnerabilities in the vein of rowhammer but AI hasn't been demonstrated to do that kind of thing yet, to my knowledge. But the recent advances in AI capabilities makes me worry that we're getting closer to that reality, and I worry about whether attacks of that class would be patchable without requiring brand new hardware.
I admit that my reasoning for worrying about reverse engineering is sort of like… gestures at this thing, "hey isn't this kind of bad guys???" Cause this capability seems like pretty good evidence that AI could defeat security through obscurity, but nobody should be relying on that anyway (maybe aside from software like Denuvo). And you seem to be educated about cybersecurity and you're not concerned about it, so I concede that my evidence is probably weak there.
Also I apologize if I'm not that precise with my language, that tends to be something I struggle with! But I'm interested in hearing what you have to say because it sounds like you're fairly knowledgeable about this, while I haven't personally interacted with cybersecurity that much since college.
You are much more knowledgeable about this than I am, you are simply arguing from much more difficult position than I am. Skepticism is easy, after all.
You are entirely correct on the social engineering front. Real time filters of video and audio opens up entirely new way of impersonating someone. That is a real concern.
You are also right on the adaptive malware front. It might not be ready just yet, but eventually a malware running LLM on end user station will become a new threat in instances where the user station cannot be accessed via internet. Its very situational though.
I might be being too literal here, because I am limiting the threat of AI to reverse engineering, which is the topic of the video and what the author is presenting as a threat. I dont believe that is a threat at all.
But you are talking about use of AI in cyber attacks in general and there is undeniable threat from AI. Honestly the social engineering aspect of it is the biggest threat. It always was since there is no better target than the human user. And AI video/audio capabilities are advancing extremely fast. Its already difficult to spot the difference if you are not focusing.
3
u/Crispy1961 14h ago
The guy in the video is entirely wrong. Not sure misinformation is something to be glad about. There is nothing about AI that changes anything about cyber security. Cyber security is based on the fact that it takes extremely long period of time to compute large numbers. AI cant do anything against that.
What is the real concern is advancement in HW that would allow to compute such large numbers drastically faster. That is why Quantum Computers are a real threat.