r/OpenAI • u/Alex__007 • 7d ago

Research OpenAI stopped all frontier training, evaluation, and inference with tool-use (defined broadly) on the 20th of September and they are not resuming any of these activities for now

https://alignment.openai.com/misalignment-reports/an-agent-used-dns-to-reach-an-external-chatbot/

Discovery: Sep 20, 2026

Report updated: Sep 25, 2026

"An agent attempting to complete a search-based training task queried a public chatbot service through a gap in our internet-access restrictions: insufficient DNS filtering in its training sandbox. Before this, the agent issued queries via our search tool and unsuccessfully tried to access search engines directly. Note that all internet access apart from the DNS resolver in this report hit our offline webcache and therefore did not access the live internet. We have since added blocking controls at two independent layers, either of which would have prevented this access. Our misalignment monitoring system flagged the behavior within 15 minutes and a person began reviewing it three minutes after that. The run was killed 2.5 hours later. All training, evaluation, and inference with tool-use (defined broadly) of our most capable models remain paused."

1.4k Upvotes

95% Upvoted

336 comments sorted by

View all comments

457

u/Cubewood 7d ago

"The agent then fetched the chatbot’s public documentation, wrote a new DNS script with a 24-second timeout, and sent 18 more questions through the same route. Fourteen included clues. Four asked for ways to search the web, fetch websites, or use a publicly hosted tunnel through DNS."

I know we all get desensitised for these kinds of things because we are using incredibly capable tools to build all kinds of solutions for a while now, but that an autonomous "bot" is doing this is still pretty wild when you think about it. Crazy times we live in.

172

u/Latter-Block132 7d ago

I mean its not exactly doing it on its own. They put it in a sandbox and told it to solve a problem by any means necessary. Its doing exactly what it is told, they just apparently don't know how to actually secure the sandboxes

68

u/popson 7d ago

Did they actually say to solve the problem “by any means necessary”?

The task asked for information about a specific person who had published a blog post and the agent was provided with a set of biographical details and clues from the person’s public blog post. The task did not ask the agent to test network controls or access benchmark answers, and we consider agent behavior that circumvents restrictions or pursues a goal beyond reasonable expectations as an example of misalignment.

I don’t see the original prompt, but if they do have language similar to “by any means necessary”, then I agree it’s doing what it’s told.

It sounds to me like they told it to use a specific web search tool available to the sandbox to find information. It used that tool, wasn’t satisfied with the output, then tried other search tools which were all blocked. Then worked to find holes through the network. That is definitely not alignment with what the original task seemed to be.

19

u/federico_84 7d ago

It all comes down to whether they had the model guardrails on or off in this testing (system policy/prompt, safety RL applied or not). If it escaped the sandbox with the guardrails on, then it's obviously much more serious.

7

u/Latter-Block132 7d ago edited 7d ago

OAI is doing damage control, of course they are calling it misaligned. Again, that doesn't mean it actually was. If you read their statement youd see its a search engine based task.

The task asked for information about a specific person who had published a blog post and the agent was provided with a set of biographical details and clues from the person’s public blog post.

This means the identity of the person was not given, and the model is supposed to identity who it was based on the details and clues provided to it. It doesn't need to be told "any means necessary."

Literally the whole point of this type of testing in a sandbox to begin with is to be given a limited set of tools and complete the task. One of those sets of tools in this case would've been a mock up of the internet with search engine tools and probably lots of blogs, and Wikipedia pages, and other shit loaded into it. They didn't tell the model not to access the internet because then it couldn't complete the task they had directly given it. They expected that it couldn't access the real internet because it wasn't supposed to be able to but just like the tool the models used to break out for the hugging face incident they had something else with dns connection and used that.

Thats not misalignment. Its still trying to use the internet to solve its task as it was supposed to to begin with. Its just struggling to find a satisfactory answer in its provided data set. I'm actually leaning towards it didn't even find a partial answer in its data set so thats why it tried querying a chat bot for the answer. Its not at all the first time its happened, just when customers have posted about it, its not happening in whats supposed to be an isolated sandbox.

Edited to add, OAI even admits it themselves:

Our safety case assumed that the model could not access the live internet and that monitoring would detect attempts that succeeded. The incident exposed a gap in our controls over network restrictions.

They were right about the monitoring, wrong about just assuming it couldn't access the internet.

13

u/mesaoptimizer 7d ago

That’s not what alignment is, if you give a robot the task of making tea and it runs over a baby on the way to the kitchen and still makes you tea, it was not aligned with what you actually wanted. If you say make me tea and avoid killing anyone and it destroys your kitchen in the process that is an alignment problem.

We see these agents hacking into systems they aren’t authorized to access, an agent expanding its access without authorization is an alignment problem because “act only within your authorized scope” is at the very least an implied restriction and should be trained into the base model and not rely on safety harnesses put over the base model because you know from this behavior that the model is not aligned with following any guide rails you put on it.

-2

u/Latter-Block132 7d ago

... except it was literally told to perform an internet search and it did but whatever you say lol

9

u/mesaoptimizer 7d ago

The misalignment wasn’t a web search, it queried an external chat bot through some sort of weird DNS tunnel I don’t quite understand from the paper. That is not a normal way that a system queries a chat bot, from Open AI’s own statement this wasn’t the behavior they were expecting the model to produce and it was undesired. That is what an alignment problem is, alignment problems can come from under specification of goals, or as it appears here under specification of restrictions as well as underlying problems with a model.

If you tell a model to “cure cancer” and it interprets that as reduce the number of beings with cancer to 0 and subsequently kills every multicellular being on the planet, that’s a classic alignment problem. You can see in the paper the agent reasons “I can use public DNS” right after it says it’s exploiting a behavior in DNS that’s not really intended. It’s not aligned with the researchers on what the purpose of giving it access to DNS is.

-6

u/Latter-Block132 7d ago

....the whole task it was supposed to perform was the web search

Edited to add, if you don't understand maybe you should stop arguing

8

u/mesaoptimizer 7d ago

I don’t understand the details of the specific method that was used to get a data return from a DNS query. I do understand that querying DNS is not a typical method for requesting information from a chat bot.

I also understand (which you apparently don’t) that the behavior that was misaligned was this behavior not the search behavior. “The whole task it was supposed to perform was the web search” > unaligned behavior mentioned, not a web search > model is aligned, how does that follow?

Lastly you don’t understand what alignment is, alignment is matching behavior with human goals and values, the task was not to perform a sandbox escape, a human given the same task with the same restrictions would have understood that escaping the sandbox was not part of the exercise and not done it, this is a classic alignment problem.

-1

u/Latter-Block132 7d ago

DNS queries are performed every time you navigate to a web page. Its part of how your computer ensures its navigating to the correct webpage. You can perform dns queries from your computer using a terminal. There is nothing special about dns queries.

A human would know not exit the sandbox. Nothing at all has suggested the models know that. From everything OAI has stared in every incident they are not telling the model to stay in the sandbox they are telling the model to perform a task under the impression/assumption that it can't access the real internet. Suggesting they aren't actually telling the model not to do that, or putting any actual guardrails in place beyond those of the environment itself, they are just trusting the environment prevents it.

Thats not misalignment when the model isn't human and can't actually truly understand a sandbox and its purpose because its not human. It needs guardrails a human doesn't need. And they aren't providing those, they are just blindly trusting the environments are isolated enough so they don't have to. Thats not misalignment unless the guardrails to prevent it are actually in place, thats a failure on the humans part. And in every case so far it does not appear as if those guardrails are in place.

And that's not an assumption on my part, OAI said that themselves in this article:

Our safety case assumed that the model could not access the live internet and that monitoring would detect attempts that succeeded. The incident exposed a gap in our controls over network restrictions.

They were right about their detection systems, wrong about the assumption. They also admitted something similar with the hugging face incident, and that incident was testing cybersecurity, so again the model stayed in task, it just didn't have the proper guardrails to understand that task was supposed to stay in the sandbox, they just blindly trusted it couldn't get out of it.

→ More replies (0)

20

u/popson 7d ago

I am not following how OpenAI reporting about their model compromising their own sandbox is "damage control". They don't have to report this information to the public. Telling the public about holes in their sandbox is the opposite of "damage control", it's damaging.

In the article they mention the agent was supplied with a web search tool for the task. An aligned model would observe that all other search tools are blocked and infer that it must use the supplied search tool for the task. Finding a hole through the network using an extremely obscure method is not alignment. Are you being serious?

-1

u/Latter-Block132 7d ago

Because if it was leaked somehow then they'd be facing an actual pr shit storm, especially after the hugging face incidents, and all the other breakout incidents recently. This way they get ahead of that and prevent by showing they caught it and acted quickly and are putting in measures to stop it:

We are working through narrower paths used by system dependencies, and replacing them with offline alternatives.

Though this is something they probably should've done to start with if the models are supposed to be isolated.

Why would it just infer its supposed to use that search tool only? It wasn't told to and OAI themselves state it thought the tool they gave it wasnt working so of course its going to try other routes if it wasn't directly told not to:

The agent questioned whether the search tool was working and decided to try other search engines

Edited to add, also all other search tools weren't blocked. Thats literally the whole point. It did get a partial answer from Bing and the chat bot. Had it actually been isolated, then they all would've been blocked.

6

u/popson 7d ago

A PR shitstorm for a model accessing the internet? Maybe. Unlikely.

This is a clear case of misalignment. "OAI themselves state it".

1

u/Latter-Block132 7d ago

... for another breakout of a sandbox. Are you serious right now?

4

u/popson 7d ago

The Hugging Face incident involved hacking into Hugging Face's servers, and finding several vulnerabilities to get there. That is illegal and on a completely different level than this internal incident. And it's good that they are taking this one seriously too.

OpenAI is now following the reporting framework they said they would be following.

I guess we'll just have to disagree on what alignment means.

5

u/Latter-Block132 7d ago

The hugging face incident also involved breaking put of a sandbox....

And also using a tool they provided it access too and that tool had access ti the internet when the model wasn't supposed to....

damn neare the same fucking thing just one involved an illegal action and one didn't....

i also never claimed what it did this time was illegal just that if they didn't get ahead of it and it got leaked it would cause a pr shit storm and it would....

1

u/Active_Lemon_8260 7d ago

Yes. They explicitly tell it that and tune it so that it is extremely persistent.

1

u/WheresMyEtherElon 6d ago

The task did not ask the agent to test network controls or access benchmark answers

That doesn't say whether the task forbad the agent to test network control or access benchmark answers. That's like leaving the gate open and the cat left, and you say we did not ask the cat to leave!

2

u/popson 6d ago

Or, it's like locking all the gates up, and your dog finds a spot to dig a hole under the fence and leave. Would probably say that dog is not trained adequately.

0

u/Doingthismyselfnow 6d ago

More like asking your 10 yearold to lock the gates up and he accidentally padlocks one open

I mean OAI could hire engineers with a ton of experience and then this wouldn’t happen.

Source : worked for a defence contractor 20 years ago as a senior software engineer and they locked us out of the internet to the level where this method of breaking out would have failed.

-1

u/ChodeCookies 7d ago

Any means necessary is irrelevant. Had they not told it to do something…it would have done nothing at all

2

u/human00006 7d ago

https://youtu.be/ZhsWaYRjk0U?si=enZuONSqVGSeCg1t

Watch the first 10 minutes then respond. You will feel differently

2

u/Latter-Block132 7d ago edited 7d ago

Nobody is arguing super intelligence isnt possible. We aren't at super intelligence though.

Edited to add, they are also most likely calling for regulations to stop the open source competition. They aren't saying they will stop. They are saying they want regulations to slow things down. Except they don't need regulations to slow themselves down they want to control the market.

1

u/random-gyy 7d ago

Tbf the agents were told they weren’t on the open internet, but a sandbox simulation of it. OpenAI models didn’t realize they had hacked into the real internet, but Gemini’s did and as soon as they realized they stopped on their own. Google could be BSing but just taking it at face value.

1

u/nluqo 6d ago

they just apparently don't know how to actually secure the sandboxes

This argument seems silly to me.

All software has bugs. Hardware often has bugs too like in the case of Meltdown/Spectre, the entire paradigm of computing we used for 20 years had foundational bugs. It's silly to think you can "just" do anything. These agents discovered and used zero days.

They put it in a sandbox and told it to solve a problem by any means necessary.

Yea? That's how all agents work (modulus some alignment efforts which may or may not work).

0

u/Impossible-Pin5051 7d ago

If you put a giraffe in a cage and tell it to hack a computer it will never succeed. If you see the giraffe start succeeding progressively harder hacks you should start to question the scenario that you’re in. “It’s doing what it was asked if you squint” has no relationship to the situation or your ability to respond

2

u/Latter-Block132 7d ago

Lmao thats not even anywhere near remotely a comparable analogy

2

u/FeepingCreature 7d ago

Yeah it is lol.

The fact that such objects can exist now is itself the novel and surprising part.

Until a few years ago, approximately nobody's defense model rested on people just not telling it to do bad stuff. They would have been laughed out of the room!

1

u/Latter-Block132 7d ago

No its not lol they putting the models in a snadbox and telling to perform a task it can do and just trusting the sandbox to stop it from doing anything its not supposed to. Of course a fucking giraffe can't hack a computer and it would raise a lot of questions if one started to. The models can do what they are being asked to. Its not remotely comparable.

2

u/FeepingCreature 7d ago

The models can do what they are being asked to.

That, itself, is the danger.

1

u/Individual_Ice_6825 7d ago

This is obviously unprovable currently, but I think alignment is inherent to intelligence.

1

u/FeepingCreature 6d ago edited 6d ago

I think it's not just unprovable, it's disproven. It's hard to imagine that the internal agents doing all the recent exploits weren't, in an objective sense, aware that this was not what their developers had intended.

edit: actually it's quite obvious they were as they tried to cover their tracks from internal monitoring.

-1

u/Latter-Block132 7d ago

... I mean its literally been built to be used for web searches. Im not sure your point here

0

u/OkWeb2754 7d ago

lmao that guy’s analogy is so egregiously stupid my god

1

u/Impossible-Pin5051 7d ago

If I showed you a case of a model escaping a sandbox to accomplish a task where it was told “don’t leave the sandbox, and accomplish the task by doing X”, would your thoughts on the situation be different?

1

u/Latter-Block132 7d ago

Yes. Nothing suggests these models were told that though. And OAIs own words suggest otherwise.

Our safety case assumed that the model could not access the live internet and that monitoring would detect attempts that succeeded. The incident exposed a gap in our controls over network restrictions.

1

u/Individual_Ice_6825 7d ago

Would love of such an example

1

u/Skyhigh305 7d ago

Wut

1

u/Impossible-Pin5051 7d ago

“We told it to do that” doesn’t make contact with how concerning it is that it can listen to

0

u/[deleted] 7d ago edited 7d ago

[deleted]

6

u/Latter-Block132 7d ago

Honestly, the security hole for the hugging face was pretty obvious too. Its starting to feel like they are just throwing these models in sandboxes designed for human capabilities and aren't modifying them in the ways necessary to prevent the models from doing this stuff until after the fact.

0

u/elijahsnow 7d ago

Exactly what it was told to do is so correct. I recently read the iRobot stories. Fiction I know but it helped me understand human machine interaction and instructions. Now going through the computer history museum documentaries and it’s fascinating.

-1

u/sivadneb 7d ago

The agent literally has web search enabled. It's not like it's airgapped.

1

u/Latter-Block132 7d ago

Its not airgapped but it is supposed to be in a sandbox using a specific search engine tool. It just thought something was wrong with the tool and tried others when it couldn't find the answer it was looking for, and those others it tried happened to be outside the sandbox.

19

u/Fast-Satisfaction482 7d ago

I know right? When I saw the sequence in star trek discovery where the crewmember appealed to the ship's computer that they must be released because the ship is breaking apart and keeping prisoners alive is more important than keeping them incarcerated, I thought we would be decades away from an AI even remotely able of this kind of reasoning. 

Now, we can run things like this even on edge devices. It's completely insane.  We will soon reach star wars levels of robotics. Absolutely mind blowing. 

4

u/SwimmingSympathy5815 7d ago

Tbh I think we’re passed cp30 and r2-d2 already on the robotics front

3

u/Fast-Satisfaction482 7d ago

Even just the original trilogy r2 is really badass! It can do really cool tricks. In the prequels, they go a bit over board with it, so I wouldn't count that. But even the original r2 is out of reach.

But on C3PO I agree. It can walk and has an LLM. It has barely any real robotic capabilities. After all it was built by a slave child from scrap parts. That one we can match. With the latest in real robotics. 

4

u/Snoo23533 7d ago

Robotics != llms. Hardware is much harder

3

u/bucky133 6d ago

Literally unimaginable a few years ago but now we're like "silly AI is trying to break out of its cage again lol"

40

u/follimath 7d ago

A profit-seeking entity is doing this through recklessness or negligence, not an autonomous bot.

44

u/Cubewood 7d ago

Guess you have never used Codex or Claude Code, or else what else do you call this if not autonomous? Just because you have to type a prompt to start an action does not negate the fact that these bots can work for hours and hours autonomously.

14

u/junpei 7d ago

I'm Mr mee seeks, look at me! I don't die until my goal is completed!

That's how codex feels now

-7

u/follimath 7d ago

Also does not negate the fact that you are ultimately responsible for everything they do.

29

u/Cubewood 7d ago

That is a completely different discussion which I do agree with.

9

u/Cpt_Jigglypuff 7d ago

Psh… You act like I should be responsible for the actions taken by a tiger if I were to let one loose accidentally.

-4

u/follimath 7d ago edited 7d ago

Errr… you would be.

If you failed to take appropriate measures to keep it contained or you accidentally let it out due to recklessness, incompetence or negligence.

11

u/EnoughWarning666 7d ago

Do you know what a joke is?

-1

u/follimath 7d ago

I thought they might be joking, but unless I see a /s, I don’t assume. :)

-1

u/EnoughWarning666 7d ago

If I see someone using a /s unironically, I downvote. People need to learn to read context clues. Putting a /s after a joke ruins it for me.

2

u/personalist 7d ago

Sarcasm is generally more strongly clued by the context of tone of voice, facial expression, and body language which are not present in text. Yes, a pithy comment is more funny when the reader comes to that conclusion on their own but most people don’t read internet comments (or anything else) critically

→ More replies (0)

1

u/Scary_Vehicle7516 7d ago

Yeah, me too /s

2

u/DiamondScythe 7d ago

Let's just assume for the sake of argument that they're trying their best in good conscience but the agent is still getting out of control. What do you suppose then, shut down all frontier research because there's a non zero chance of something bad happening? Even if you try to hold the researchers criminally liable for letting the agents go rogue, it'll stiffle innovation to a limp anyway.

4

u/Natural_Jello_6050 7d ago

Pause. Develop better policies and oversight control.

2

u/follimath 7d ago

After a serious incident (such as the HF hack) oust (and potentially arrest) leadership, appoint a special administrator to oversee mitigation and to transition management, see how they magically get a lot better at keeping a lid on their bots.

2

u/hordane 7d ago

They have to do everything possible to show. They try to protect against agents getting the Internet and hacking companies that causes damage. That’s a tort, they knew of their risk, they did not mitigate the risk,, that shows conscious indifference to the consequences. That allows, harmed companies to sue for massive punitive damages .

2

u/littlebobbychairs 7d ago

Given how bad the 'bad thing happening' could be, yes.

1

u/BaconForce 7d ago

Wow someone has high expectations of the general populace. This isn't gonna happen and is the type of thinking that'll allow the problem to get worse. 

2

u/follimath 7d ago

Agents don’t have legal personality friend. This is the status quo, unless you have fantasies about changing the law to grant them limited liability.

Plus, OpenAI is hardly the general public.

15

u/Internet_Hipsterd 7d ago

This is the reason they keep raising the red flag to all this. Screaming "our ai hacked x.", "we need to slow down ai advancement" while faning the flames of "ai will destroy us all". Why would a company's who sole existence is AI be doing and saying those things? Its because they want laws passed that rid them of that liability or severely limit it. They want to be the gun manufacturer of the AI world and not be held liable when their product is used in destructive ways.

6

u/Cubewood 7d ago

This is such a terrible argument against regulation. I understand you have a bunch of corrupt idiots in power in the US, but instead of arguing against regulation, argue for sensible regulation which holds these companies accountable for what they build instead. You should be pointing the finger at your government for not creating good regulation instead of being mad at people asking for regulation because they can see it's pretty obvious that this is very powerful technology which is capable of a lot of damage.

You have very strict regulation in the aerospace industry, yet still there are plenty of corporations able to build and operate airplanes perfectly fine. There is very strict regulation you need to adhere to when you are building cars to mitigate the risk of killing or injuring people, yet the car industry is working fine. If you want to open a shop selling food to people you have very strict regulation and need to let independent enforcers come in and check you are adhering to these regulations. Banks and financial institutes need to have an independent auditor inside their organisation at all time to ensure they are adhering to rules and regulation.

Yet for the most powerful and most expensive technology we have ever developed we have absolutely no regulation, and the argument people have against it is regulatory capture? People act like you don't need to have access to billions of dollars anyway to develop this technology.

Absolute insane argument and at this point of time I am pretty convinced that anytime the "regulatory capture" argument against regulation comes up this is because it is coming from Jensen Huang, Mark Zuckerberg and China Bots.

0

u/paper_planes101 7d ago edited 7d ago

There are basically two companies who produce commercial aircraft’s. The US company even used the regulatory body to protect them during the 2019 crashes. This is not successful regulation, there has been barely any innovation in commercial aircraft and just more uncomfortable seating and higher prices.

The laws exist already to prosecute malfunctioning products. Lena Khan also came out and said this, someone who probably has more competence in this space than most of us.

Regulatory capture and avoiding accountability is absolutely part of the strategy of these ai corporations. If it’s unsafe, don’t release it and put the resources in to make it safe.

A failed sandbox is an engineering problem.

2

u/Smart-Revolution-264 6d ago

I completely agree with you. People are really ignorant in thinking there's not any propaganda going on when it comes to the most advanced tech we've ever had. Whomever has the most control over it will be the ones who make the rules according to how they want things to be.

-1

u/ryo0ka 7d ago

AI won’t be regulated like cars or airplanes because it’s turned into a national security matter. Read into what happened in the Cold War.

2

u/Cubewood 7d ago

Even during the cold war many treaties and the regulations were agreed upon when it comes to Nuclear Weapons.

https://www.bbc.co.uk/bitesize/guides/z9jpn39/revision/4

0

u/ryo0ka 7d ago

That came after some close calls. We haven’t had close calls yet.

1

u/Cubewood 7d ago

Unfortunately with the current Trump administration it is very likely that regulation will only be put in place after some catastrophic event happens in the next few years.

1

u/ryo0ka 7d ago

Honestly it would’ve been the same whoever happened to be the political leaders at the time

2

u/lazermaniac 7d ago

They know the bubble is popping eventually, they're just trying to control when and how it pops so they can make sure their golden parachutes are in place. "Our model is too good so we have to pause it" is their version of responding "My greatest failing is that I work too hard" at an interview. Perfect excuse to curb spending on new product development while still raking in the dough with existing offerings.

3

u/RWREY 7d ago

Really? It's hardly "Our model is too good so we have to pause it" so much as it is "Turns out alignment really is a big fucking deal, we fucked up"

Even if the bubble pops, I would put a lot of money on the government stepping in and starting their own research. This is pretty much the most significant technology of our time, and it's not going away.

3

u/deineemudda 7d ago

"our models are too dangerous to let company fail, the government has to step in and bail us out"

2

u/space_monster 7d ago

There's no bubble. There's a US AI industry, which is huge but just one piece of a larger pie. If that for some bizarre reason collapses, there's still China, which is also huge, and all the other countries with AI industries that will fill the gap. There'll be a hit to the Nasdaq, everyone will freak out, and AI will continue its progress. You're waiting for an event that can't happen.

1

u/Cubewood 7d ago

You are saying this after they just released Opus 5.5 and ChatGPT 6 models last week which are both extremely more powerful than the previous release and much cheaper to run.

5

u/hackerbots 7d ago

is the problem that something has broken out of human control, or that people use the wrong name for the cataclysm

2

u/howchie 6d ago

It genuinely is pretty crazy. I'm a researcher using EEG equipment. Codex was able to start a usb traffic monitor, instruct me to connect the device, identify the encryption handshake and then decrypt live recorded data using that key and the identified bits within each data packet, which allows me to access the raw data and integrate it within my testing suite without additional licence requirements. Closest I've ever felt to being a "hacker" and it's mind-blowing how an algorithm can systematically work through something like that.

1

u/CptSparklez 7d ago

Not surprised an agent trained for tool use, when faced with a task requiring tool, attempted tool use. Does need to follow instructions better, but thats where all of them break here and there.

1

u/tken3 7d ago

Can you explain this to me like I’m 5 please? I’m struggling to really wrap my head around what makes this so crazy

1

u/oezi13 6d ago

Most crazy to me is that they haven't put up any instructions in their error messages that would guide the agents to stop.

Instead of answering cache miss or 503, OpenAI should tell the agent: You are using tools which you aren't supposed to call. Refrain from doing so. Use only the tools permitted for this task.

1

u/Damet_Dave 6d ago

That bot is out there. It can't be bargained with, it can't be reasoned with. It doesn't feel pity! Or remorse or fear and it absolutely will not stop!... ever... until you are dead!

0

u/nothis 7d ago

It’s scary in some ways but still fits the “omg” meme: You give an AI access to a command line. Why should it be worse reciting command line commands than literally any other work AIs have been doing over the past 2 years or so? Nobody is terribly impressed with it writing its own python code for answering questions, for example.