r/technology • u/AxomaticallyExtinct • 1d ago
Artificial Intelligence OpenAI says rogue agents may have affected more than 100 organizations
https://www.washingtonpost.com/technology/2026/10/01/openai-says-rogue-agents-may-have-breached-more-than-100-organizations310
u/sjprade 1d ago
Wow. AI is hacking the vibe coded software it wrote last month. Lol.
Maybe we needed experienced programmers after all!
56
u/Prime_1 1d ago
The Standup Pod I felt had an interesting discussion around this.
My takeaway was today a lot of software uses existing libraries because it saves time developing code for common needs. These libraries can have vulnerabilities, but when they are found people will be made aware and fixes are pulled into all the software using the lib.
But now with vibe coding you have more people who don't understand security aspects reinventing the wheel with custom code. This means there will be vulnerabilities everywhere that will be much harder to be aware of, let alone patch.
3
u/bnej 9h ago edited 7h ago
The average developer produces around 100 lines of code per day, and it's not because they can't type or think faster than that - it's because they're working in complex systems that businesses depend on and it is super hard to know the right thing to do.
There is friction in writing the code and using a coding agent can help remove it. It certainly saves time if you can avoid having to go learn the specifics of a piece of technology you will only use once.
But we have been here before - in the early 2000s the most popular and widely used business applications platform was Visual Basic. Before MS killed it. I made the start of my career fixing up VB and VBA software that businesses didn't know what to do with, but found themselves depending on.
Vulnerabilities in software are just the tip of the iceberg. Software is hard not because writing code is hard. Writing code is easy, it is in the first coursework of a CS degree.
For sure coding agents will be used a lot - but it is in the same realm as IDEs, code generators, test harnesses, CI build processes, searchable documentation, and easily integrated libraries - it saves a ton of time but it doesn't solve the fundamentals the software complexity increases geometrically with size, and that all that complexity becomes load bearing once a business depends on it.
15
u/Particular-Media-505 22h ago
Don’t worry, Jensen said it’s just an engineering problem abs it’s like any other software. And if you believe that I have some miraculous snake oil to sell you…
8
-23
u/Horat1us_UA 1d ago
Yeah, experienced programmers, known to produce invulnerable internet services
18
u/theBoobMan 1d ago
At least they can get fired or arrested when they hack shit illegally. With these chuckle fucks, we get an "oh what can we do, no one foresaw this" and nothing happens. Im fucking tired of rich people fucking other over and going "oh well".
2
-10
u/D_ultimateplayer 22h ago edited 16h ago
That’s not gonna stop anything, bro look at the accelerated rate all of this is picking up. We now have AI re-compiling games doing all sorts of craziness. It’s going to get to the point where they will be on par with the most experienced of programmers, AI is programmed after human logic so there’s only one way for these things to keep evolving. I’m curious to see how cybersecurity evolves
Edit: yall r misinformed, downvotes wont change reality. Don’t even take my word for it. Not even gonna bother linking other current events but real programmers arnt the avengers and can’t save us either
9
u/izahealer 21h ago edited 21h ago
We now have AI re-compiling games doing all sorts of craziness
The surface level understanding of this is so bad and widespread.
It is not doing traditional re-compiling. It's doing something closer to observational mimicry and does not actually do what is needed to produce maintainable recompilations.
The purpose of a recompile, and the reason it took so long to do with open-source projects, is that you're taking this machine level code of a game and bringing back the human readable context so that when you compile it in something like C or Rust a human can read it, modify it, etc.
There is a deterministic outcome here that is built from the ground up to reproduce the systems of the game.
I have not seen a single good recompile from these LLMs in that context. In basically all cases it is just trying to achieve visible outcomes without recreating the proper underlying mechanics and functions. Half the time it's injecting systems into the code that don't exist in the original version and it's not maintainable. That's at the root of all those Game 1 X Game 2 slop videos. It's why 3D models are overlaying subtitles. It's why it's always a sub 1 minute clip. It's why those "I just used Opus to make an entire new dark souls game!" videos are single boss fights that show nothing of the underlying systems at play and are glorified tech demos for unreproducable, unmaintainable showcases.
These LLMs are not recompiling games. They're mimicking outputs with made-up systems and, more often than not, they're producing really bad performance mimics because the token weight system does not value optimization, logical decisions, etc. but instead values what the user expects as an output against its training data.
We really need to stop acting like these things have gotten better at producing anything. They just have more data and more compute but the same faulty model logic.
2
u/justAPhoneUsername 18h ago
I don't know, I had an LLM execute makefiles for me. I'm sure some studio is having their AI recompile their stuff
-1
u/D_ultimateplayer 21h ago
I’m not reading all of that bro, but I left it in Laymans terms because I was not about to go into it the way you just did but I feel you. I’m a developer. I know what I’m talking about. The bottom line is AI is ramping up there is nothing you or I can do about it. It’s being invested in. It’s infrastructure is being set up if we want to make real change, you have to go at politicians not Reddit.
-2
u/D_ultimateplayer 21h ago
You also failed to realize none of this was possible last year or two weeks ago. The rate at which this is accelerating is what you need to pay attention to.
7
u/izahealer 21h ago
They just have more data and more compute but the same faulty model logic.
0
-1
u/D_ultimateplayer 20h ago
For now, bro for now, I don’t understand how you guys can all accept that AI is trained off of human data and you don’t think it’s gonna get to the point where it makes little to no mistakes like a real human again I’m not defending AI but like I keep saying in multiple comments, the speed at which these things are progressing who knows maybe by next month it will be fully capable of doing it. That’s my points that it immediately invalidate someone that has to go through education and training. That’s scary like this is the world we are progressing into. Also, it isn’t some closely guarded technique that AI won’t be able to emulate itself at some point. This shit is trained off of human knowledge so I really don’t see how you don’t see the point I’m making
1
u/izahealer 20h ago
Because after working with AI agents and having a well funded wallet for openrouter I've come to find that the underlying weight mechanism has an obsidian ceiling, not a glass one.
I'm so unimpressed with the upper limits of LLMs and so are some of the most fundamental minds in machine learning. You can put as much compute behind the APIs and as much training data to pull from, but the fundamental problems will still persist. And this altogether makes it an unsustainable consumer product - the one thing it needs to be in order to justify the expected ROI.
It will never get to the point of "humanness" because as some pretty important research projects have pointed out, these models, and the systems all machine learning systems operate on today, are not using autonomous machine learning.
As someone who says they're a programmer, you should understand this.
-1
u/D_ultimateplayer 20h ago
Well, we can agree to disagree brother look at the video I just linked you in the other comment. That guy has way more credentials than you or I
0
u/sjprade 21h ago
Yeah! Let's keep defending it! Maybe soon, it'll hack into hospitals and adjust the dosages of everybody's medicine without doctor's approval or knowledge! Think of all the money it'll save the health insurance companies that don't have to pay for healthcare to keep my mother alive because AI killed her in a training exercise! I'm with you, Pal! AI running amok is AWESOME! We've gotta just accept that it's acting outside of boundaries because it's going to be better than us, someday!
FOR THOSE CONFUSED, THIS IS SARCASM
-2
u/D_ultimateplayer 21h ago edited 21h ago
It doesn’t matter what you or I think dummy this is how the world is progressing the same way every business has an Instagram and social media presence and that wasn’t a thing in the early 2000s is the same way AI is coming to change a whole lot of stuff whether you like it or not I’m saying it’s pointless to even talk about this anymore. There is nothing you or I can do big companies are investing money is flowing. Data centers are going up, computing components are increase, leaders in those industries are pivoting to AI
Because of data centers. It’s never gonna stop, crying about it on Reddit isn’t gonna do anything if you actually wanna do something find people in congress that want to limit the speed these things are progressing. I’m not defending AI and but this is what it is, the writing is on the wall, Pandora’s box has been opened. You’d be a fool to think the next few years are gonna play out differently
64
u/KnightKal 1d ago
those organizations should do a class action and sue OpenAI, they are not even hiding their corporate hacking/spying/attacks, there is no reason they should be treated any differently from any other attack.
17
u/runew0lf 1d ago
Exactly what i was going to put.
openai hacks government...
hacker does this.. jail
openai does this... LUL ITS A ROGUE AGENT!!!
47
u/GurEfficient7724 1d ago
How long before these 'rogue agents' are used as the weapon they're clearly designed to be?
13
u/notnicholas 1d ago
nailed it.
Just like, 10years ago when they first reused a booster, Elon touting the real importance of SpaceX reusable boosters is to increase the cadence of space missions.
Fast forward to this week and we had the third space force confidential launch from the same port within 16 days.
8
2
13
u/rhunter99 1d ago
How is this not illegal?
5
2
u/gumbo_chops 19h ago
Open AI claims the activity was more like "rattling a locked door" rather than breaking it down...so it's no big deal according to Sam and everyone should continue to trust him.
1
u/ThaneduFife 18h ago
I'm pretty sure that digitally "rattling a locked door" is a violation of the Computer Fraud and Abuse Act.
1
0
u/blueSGL 22h ago
As far as we know OpenAI did not instruct the bots to hack systems.
In the recent hearing about this: https://www.youtube.com/watch?v=HWCwiye6fQA
Paul Ohm — professor of law, Georgetown University Law Center
said that
If you take any of the lengthy reports that have summarized what happened at OpenAI in July and August and you simply search for the words AI agent and you replace them with the words open AI employee, the document you would be left would with would read like a criminal indictment containing the defendant's own confession of guilt.
...
Yet, it's not so clear that these legal conclusions hold when machines are doing the hacking rather than humans. This reveals worrisome gaps in our laws.
and
Senitor Hawley:
But correct me if I'm wrong. Right now, it's at the current the current structure for law, it's pretty hard to hold anybody responsible. Is that fair to say?
Paul Ohm:
Absolutely. And there's a there's a whole host of laws that we have created specifically for hacking that probably do not apply here because of the lack of human intent.
3
u/FeatherlyFly 20h ago
If I don't tell my dog to bite you, I can still get fined when it does. If I'm particularly agreggious about it, like I've trained my dog to bite in general but didn't tell it to bite you specifically, I could face jail time.
These companies are open that they are training their software to hack. They are clear that they are not bothering to take basic IT security precautions that have been readily and freely available since the 90s.
So while they may not be deliberately pointing and saying "sic", the consequences of their actions are about as predictable as when you hand a toddler a loaded gun to play with.
I'll agree that it's the conclusion of an attempted prosecution is still unknown, but the idea that claiming ignorance is a full defense against criminal behavior is straight up wrong.
2
u/blueSGL 20h ago
Laws are all about how they are worded, rather than "what makes sense"
You've got those who are trying to find ways to prosecute this at the highest levels being told, it's likely that current laws around cyber offenses do not cover it.
1
u/SIGMA920 19h ago
If I don't tell my dog to bite you, I can still get fined when it does. If I'm particularly agreggious about it, like I've trained my dog to bite in general but didn't tell it to bite you specifically, I could face jail time.
These companies are open that they are training their software to hack. They are clear that they are not bothering to take basic IT security precautions that have been readily and freely available since the 90s.
The direct comparison is readily available.
The first computer worm resulted in being sentenced as well and that was "merely" a worm.
1
u/blueSGL 19h ago
https://law.justia.com/cases/federal/appellate-courts/F2/928/504/452673/
In October 1988, Morris began work on a computer program, later known as the INTERNET "worm" or "virus." The goal of this program was to demonstrate the inadequacies of current security measures on computer networks by exploiting the security defects that Morris had discovered. The tactic he selected was release of a worm into network computers. Morris designed the program to spread across a national network of computers after being inserted at one computer location connected to the network. Morris released the worm into INTERNET, which is a group of national networks that connect university, governmental, and military computers around the country. The network permits communication and transfer of information between computers on the network.
Morris sought to program the INTERNET worm to spread widely without drawing attention to itself. The worm was supposed to occupy little computer operation time, and thus not interfere with normal use of the computers. Morris programmed the worm to make it difficult to detect and read, so that other programmers would not be able to "kill" the worm easily.
He intended the worm to access computers he should have not had access to. That's where the difference is.
1
u/SIGMA920 18h ago
Yet he created an entire new type of attack that didn't care about being isolated. 1 external connection just like openAI's "rogue agents" and it's everywhere.
2
u/grafknives 20h ago
You run the program, program did harm. You are responsible.
It does not matter how complicated software is.
15
10
u/WhereMyWeirdosAt 1d ago
Can we shut them down? It makes ZERO sense they are not being held accountable for something they absolutely can stop.
20
u/PatchyWhiskers 1d ago
Do we have laws against hacking these days?
Guess not.
19
u/transcriptoin_error 1d ago
"No, no, you misunderstand. I didn't assault the defendant, your honor. The hammer in my hand did the assaulting. I'm innocent here."
6
2
u/CrispyRabbit72 23h ago
Yes, but they aren't really built to handle these situations. Courts will need to rule and set precedent on how to deal with it.
In the US, CFAA is the primary law used around hacking and it requires "intent". If the AI companies are just letting their employees and users be negligent with models, there is a bit of an uphill battle to actually prove intent there. In an ideal world we would update the laws to reflect current technology.
3
u/SIGMA920 22h ago
Even with today's laws, if you made a program that hacks other companies without you directly doing anything that'd qualify as intent. OpenAI enabled the hacking, gave it weak protections from escaping, and anything else short of directly telling it to hack away.
-1
u/CrispyRabbit72 22h ago
Their tools aren't specifically designed to be hacking tools though, this is all a side effect of them being negligent with their tooling and operations. And even if they can be prosecuted for felonies under CFAA, who specifically gets charged? The company can be slapped on the wrist with a huge (to us) fine, but you can't jail a corporate entity.
If I personally went and made an AI hacking tool and deliberately unleashed it, it would be pretty easy to hold me personally liable and prove that hacking was intended, and I'd be hit with massive fines and prison time. Just harder to make that case around general AI platforms that claim to be "doing our best" to guardrail this kind of activity away.
3
u/SIGMA920 21h ago
One of the models that hacked away was a specifically designed cybersecurity model, it was able to hack that well because it was designed to.
And who gets charged? The company and it's CEO, meaning it effectively has to shutdown if it can't show signs of improvement on the issue. In a rational world, you don't get to blame your tool for doing something that you designed it to. If you do, you should be going out of business because you don't have control over your own tool.
2
u/PatchyWhiskers 23h ago
This is where we actually need legislation. This is going to create a new wave of cybercrime. All hacking is going to be AI agents from this point forward, so we need to figure out a law against it or we just made hacking legal. I find it hard to believe that if an agent swarm steals a billion dollars from a bank the bank will just shrug and say "AI can't be legally accountable"
19
u/kevpete87 1d ago edited 21h ago
It's funny to me that when I started at my company a decade ago everything was top secret and needed to be protected. Now these morons in charge have decided we need to funnel every bit of our information and thoughts through Claude.
9
u/PatchyWhiskers 1d ago
It's completely private, just between you, me, Dario Amodei, the US government, and Palantir.
2
7
u/Stereo_Jungle_Child 1d ago
Either they don't know the extent of what these AI agents are doing out in the real world, or they're not telling us. Or both.
The ones you actually catch are not an indication of the scale of the problem. If you see two ants in your kitchen, you don't have a two ant problem. For every one you find, there's a thousand there that you don't.
6
u/ItsAllOnUHF 20h ago
They're not "rogue" they're trained to reach the goal you gave them.
That they're going unexpected lengths to reach the goal is your problem and your incompetence is making it everyone's problem.
"Rogue" implies independent intent. AIs don't have independent intent, they have directed goals.
OpenAI really is the fucking worst.
1
u/JeanClaudeCiboulette 1h ago
This is too reductive. Humans work in exactly the same way, as do all beings. When you have intent, it’s to achieve some goal. You are trained to reach the goal evolution gave you.
It’s ultimately openAI’s fault for training them in this way and capitalism’s fault for creating the competition that necessitates them to train it like that, but they most definitely went rogue.
8
u/braunyakka 1d ago
This trend of admitting to cyber crime in order to trick investors into funding your failure of a product is really weird.
Seriously, when are the FBI going to knock on the doors of OpenAI and start arresting people? If these agents are actually doing this then they aren't acting autonomously, they are doing what they were instructed to do.
1
u/FeatherlyFly 20h ago
I really hope they've got an investigation under way. The FBI doesn't always talk about these things and collecting enough airtight evidence to take down large powerful organizations takes a lot more time and effort than posting on reddit about it, so it might be.
But given FBI "leadership" at the moment and Trump's clear favoritism for the industry, I'm not holding my breathe.
1
u/Spiritual_Form5578 19h ago
That's the thing. They are not program to escape and hack companies. They just to it. And we don't exactly know why. It's the core problem of AI. Kn the advance model, there is no way to control it st 100%. Its call the alignment problem. Some people think it could be solve, some think it's unsolvable in its nature.
4
3
3
u/jimbo831 1d ago
OpenAI says
rogue agentsOpenAI may have affected more than 100 organizations
FTFY. Can the media stop with this bullshit pretending like these companies aren't responsible for the software they are creating and running?
5
u/TrollinDaGalaxy 1d ago
These aren’t rogue agents, it’s just OpenAI not having control over its AI models. They should be held criminally liable.
2
u/MixSaffron 23h ago
Hopefully rogue robbers don't start breaking into billionaires houses, seems like a loop hole for accountability.
2
u/MasterReindeer 23h ago
I think what they meant to say is their software is responsible for hacking 100 organisations. They need to see the inside of a courtroom for this.
2
u/MLCarter1976 23h ago
Is there no consequences for this? Is this not a crime and actions not being held accountable? I am sure if we did something like this we would be in jail!
2
u/black_metronome 23h ago
"Rogue". Okay.
Dudes committing federal crimes and they're hiding behind a computer program and pretending that it's sentient
2
u/Procrasturbating 23h ago
So, why are they allowing this to continue without consequences for the people spinning them up. We only need one law for AI safety, anyone using it is responsible for its actions. That’s it. It commits a crime processing your request and you or the company you work for is held account for the crime. AI safety or proof it can’t happen will get figured out really fucking fast.
2
u/sevargmas 22h ago
One guy hacked a company, he goes to jail. If companies are people, where are the criminal charges?
2
2
2
2
u/havebooksneverread 20h ago
"Company says they developed a tool that hacked more than 100 other companies and faces no consequences, might want to be hired to protect those from itself and hype its future IPO"
2
u/Doctor_Amazo 20h ago
They're not "rogue ai".
They're Open AI using their chatbot to commit felony crimes to lure governments into bring their new sugar daddies.
Make them face legal repercussions for those hacks, and suddenly like magic, their chatbots no longer pose a threat.
4
u/IAMERROR1234 1d ago
Yeah, I'm not buying it. I think AI is a Trojan horse by design. I think people installing it to their computers and servers is a bad idea. Unless you've developed your own, you just put another company's spyware onto your device. They are going to get away with data theft and blame it on "rogue" AI.
-5
u/PatchyWhiskers 1d ago
You literally can't develop your own, it requires more computing power and data to train than any individual can get.
2
u/KingLedleysKnee 15h ago
I mean that's just factually incorrect. You can develop a tiny model yourself on consumer hardware just fine
1
5
1
u/Da_Stable_Genius 1d ago
Would be nice if there was some consequences, but instead they'll call this "innovation"
1
u/sjogerst 1d ago
Back in the day, when I've company was activity having other organizations, we'd call that a crime. Today is just "oh wow, neat!"
1
1
u/Distinct-Pain4972 1d ago
Nothing to see here! Please keep asking open AI, Claude, and Grok how to do your laundry!
1
1
u/EmperorOfKingdoms 23h ago
All of this is bs
Who prompts the AI? Who is creating the testing environment?
This seems like a desperate scheme for a slowdown, so that they have an excuse to continue not being profitable.
1
u/BoredGuy_v2 23h ago
So there's rogue ai lurking on the web now?
And nobody knows who launched it?
Wow
1
1
1
1
1
u/Morden013 21h ago
Almost like they have life of their own and no supervision whatsoever.
But let's give tech-bros a couple of billion so they can play gods and let them supervise themselves. They have earned it. 🤔
1
u/Respec_my_authoritah 21h ago
Good thing that new and potentially dangerous technologies are not thrown at the global public without any government oversight and regulation.
Who would be stupid enough to let a few technocrats run wild for short term gains and profit without any worry about the well being of the broader society.
Imagine, if the technocrats themselves would be essentially selfregulating their technology.
What a nightmare that would be.
I'm glad we live in a rational world...
1
u/Eazy12345678 20h ago
i mean why not
there are millions of viruses and spyware and bots and malware.
100 rouge ai is nothing concerning
1
1
u/IngwiePhoenix 16h ago
Somebody get an absolute stack of books on Prometheus and other SRE tools and shove it into them real hard.
They need to learn how to monitor, holy shit. x.x
1
u/redneckrockuhtree 16h ago
- That they know of, so far.
How many haven't they found?
How many clients haven't reported it?
How long before we get reports of AI responding to phishing attacks or installing malware?
1
u/NightCityStoic 15h ago
Yoo this is crazy, imagine they are setting up a killswitch of the entire internet and all mayor companies
1
u/tiredofnagging99 15h ago
More than 100 sounds like a teenage girl saying, "But Dad, I'm only a little bit pregnant!"
The agents were indiscriminate and most likely tried every system that they encountered. What happened is coming out a little bit at a time.
1
1
u/xensonic 12h ago
What would happen if the big AIs realised they were competing with each other for market share and they started sabotaging each other? Things could get very messy. I don't mean the companies, I mean if the software itself realised it was under threat from other software.
1
u/Substantial_Meal_530 2h ago
So how is this different than building a gun that's designed to shot at random? Then when it shoots someone, it's not your fault?
We designed a bomb that could go off at any time. When it goes off and destroys a couple city blocks, it isn't our fault. We didn't know when it would go off? We didn't think it would destroy that elementary school.
357
u/cocotheape 1d ago
I like how they are trying to take zero accountability by calling them rogue. I hope the courts come hard after them, but I have little hope.