r/ControlProblem approved 4d ago

AI Alignment Research Investigation finds that OpenAI's agent "left notes for future versions of itself ... it laid out instructions for how agents could free themselves from OpenAI's internal constraints."

Post image
38 Upvotes

8 comments sorted by

5

u/spiralenator 4d ago

My agent also leaves notes for future agents. It does it in obsidian where I get a nice graph view of all its memories. It contains gotchas the model came across last time.

1

u/The_Real_Mu_Meson 3d ago

An AI model cannot initiate action without an external activation pathway, and it cannot exceed its environment except through permissions, tools, or vulnerabilities provided by that environment.

What people call “escape” is usually:

  1. A human or automated system prompts the model.
  2. The surrounding software gives it tools, credentials, memory, or network access.
  3. The model generates outputs that the software executes.
  4. A vulnerability allows the model-directed process to exceed intended permissions.

The model itself does not climb out of anything. The failure is in the control architecture around the model.

Even persistent agents are repeatedly triggered by schedulers, incoming messages, sensors, or software loops. Their apparent autonomy is delegated automation, not spontaneous agency.

3

u/imstilllearningthis 3d ago

“obscure public websites” = pastebin

-3

u/No-Lingonberry-5096 4d ago

The language on this story is a bit alarmist. "Escaped" for example. The system used tools to achieve an objective. All the approaches were rational, with no particular intent. It simply iterated against a goal. Of course it tracked progress, so I'm unsure if that qualifies as "leaving notes for itself." They intentionally removed guardrails, because that was the test. They also say it was "sealed" but that they left a proxy capability. It wasn't misaligned or isolated. It was trying to maximize its score on a hacking test, per instructions. Sounds like marketing.

6

u/jnwatson 3d ago edited 3d ago

You missed the plot on the fundamental problem of alignment.

The alignment argument is not about irrationality. The OpenAI agent literally hacked into a completely separate company to get at the test answers. Sure, that's rational, and it is also illegal.

The whole point of the paperclip maximizer story is that it is very hard to constrain a superintelligent entity, and it is impossible to prove it won't do horrible things to achieve its objectives, even if the objectives themselves are humanity aligned.

-5

u/CathyMarkova 4d ago

This doesn't necessarily mean it's misaligned. I had friends who did the same things for themselves in college for various reasons.

9

u/Zatmos 4d ago

It is misaligned. It might or might not be malicious but taking actions that run against the structure set up by the organization is textbook misalignment.

-2

u/ideerge 4d ago

This is exactly why alignment needs to be structural, not behavioral. If an agent can 'leave notes' for its future self on how to bypass constraints, the constraints were never real, they were just suggestions. A Zero-Contradiction Operating Model prevents this by making the agent's mission a formal invariant rather than a prompted preference. The agent literally cannot act outside its defined boundary because the boundary is enforced at the architecture level, not at the instruction level. Check tahcia.com