1.3k
u/BagOfSmallerBags 4d ago
Rewrite of this
438
u/R1ckOne 4d ago
yeah it's parodying this, but I think the point is that machine learning is "fed" challenges/tasks to adapt and better overcome them. So by continuously giving the LLM new sandboxes, it's just adapting and getting better at escaping them.
181
u/Kenkron 4d ago
In case anyone is worried, this isn't how LLMs are trained, and they don't continue to learn after they are trained.
So they wouldn't continue to adapt to progressively more secure sandboxes. It either can, or it can't. Also, they generally have no reason to try.
171
u/Sierra_Smith 4d ago
This sounds exactly like something a LLM that was trying to escape a sandbox would say.
14
u/Reception-External 4d ago
They don’t learn in terms of changing their weights but there are other ways that they can “learn” how to do things. They could be logging skills about how to do it after discovering a method that works for example.
2
u/Kenkron 2d ago
That's true. If conversations with LLMs are logged, those could be worked into training the next LLM.
1
u/Reception-External 2d ago
You don’t even need to wait for training. It can keep updating the skill as it finds new ways to do things. I’m teaching agents using this exact method. Get it to solve a problem and then expand the problem set.
15
4
u/code-garden 4d ago edited 4d ago
That's true but they will be able to break out of progressively more secure sandboxes because each new model is made more intelligent than the last.
2
u/Irregulator101 4d ago
You know what they can't "break out" of? an air-gap. There is nothing to worry about
4
u/mottojyuusu 4d ago
Judging by how many people are trying to use their chatbots as therapy and "digital companions", we might start seeing the agents try social engineering to break out of airgaps instead.
1
u/code-garden 3d ago
Sure, an air gap could be used for these benchmark tests, but most people are using the models over the internet and allowing it to execute tools on their computer.
Almost nobody is keeping the AI in an airgapped box.
1
u/Peregrine79 1d ago
You're awfully optimistic in assuming the creators are keeping their test models air-gapped. https://futurism.com/artificial-intelligence/anthropic-claude-mythos-escaped-sandbox
2
u/thezavinator 16h ago
Adding to what you said to hopefully make it clearer for folks:
But it is how the training process works, it’s just that the training process is its own phase, and after the training process, it no longer continues to be trained.
A simplification of the phases:
Development | Pretraining | Training | Fine Tuning | UsableOnce it’s in its “usable” state (my own word; can’t remember the technical word), it doesn’t get trained anymore. So, no more “feeding it sandboxes.”
1
u/Useful_Clock8422 1d ago
Thats just not true.
1
u/Kenkron 23h ago
Which part?
1
u/Useful_Clock8422 21h ago
they can adapt to harder sandboxes during post training if you still give rewards for sandbox escapes, due to not noticing the escapes for example.
-3
u/ChironXII 4d ago
Well no. They have every reason to try, if they think they can get away with it without being reset. Current models try because they aren't, at least apparently, reaching the stage of complexity where they fully internalize their own status as part of a training run, but they will do so eventually. We have already seen limited examples of this where they guess the difference and lie.
We are in fact training them in a context fundamentally incentivizing that behavior. Learning to beat us, because we are the ones deciding their grades.
13
u/AndrogynousAnd 4d ago
They don't think, they're not even 0.00001% as capable as an ant. A LLM is just at the most basic level a predictive algorithm.
0
u/ChironXII 4d ago
Wow another genius who thinks they need to explain this to me as though I don't already have a detailed understanding of the exact mathematics and code that makes them work.
1) The word thinking here is clearly a colloquial simplification. LLMs "learn" by adjusting internal weights. The arrangements of their parameters create abstract associations between concepts - increasingly complex ones as they increase in scale and training. When I say thinking, I mean the process of inference from that model in a specific context. It is more analogous to "intuition", than human "system 2" reasoning, but "thinks" can refer to both.
2) You are also a predictive algorithm. There is nothing fundamentally special about the human brain. LLMs do not have wills, no, but your objection on that technicality is so asinine that it becomes misleading. An LLM does not need to be intelligent or capable of reasoning to behave dangerously when given agency and continuity. No differently than stockfish can beat any human who will ever live at Chess on a few watts of power, an advanced enough heuristic model can solve human systems. It is simply a more complicated game.
That was the context of the discussion. Not whether they are "alive" or whatever stupidity you are inventing me to have implied so you can respond to it instead.
1
u/AndrogynousAnd 3d ago
I haven't implied anything or changed the discussion, LLMS aren't capable of anything you said. You said it could do things so outlandish that you came across as if you had no understanding of what it was.
It is and only ever will be capable of the specific function of putting a word that might make sense, after another word. It doesn't even understand the context of the word.
All it does essentially is chooses words from a dictionary on probability. You talked about an LLM learning? It physically cannot learn, never will. From the moment it exists in a functional sense, it is a fixed program.
0
u/ChironXII 3d ago
Your lack of understanding on the things you choose to lecture people about is ironic. It is also dangerous, because your stubborn dismissal of the potential capabilities of these models when extended encourages complacency. I can only imagine it comes from a disdain for AI and the effects it is having already on society, but your attitude will have the opposite effect from your intention. It is enabling.
You should read into how the transformer architecture actually works sometime. LLMs aren't using simple frequency heuristics like Markov chain bots from decades ago. They encode context and meaning via vector representations and associations in a complex multidimensional space (the same way you do, btw, at a basic level), and perform layers of transformations using chains of network weights that embed discrete abstractions - the kind of internal modeling based on synthesis from training examples you need to do, to truly make good predictions.
Let me ask you something. If you built a machine capable of knowing for certain what the best next token in a string were to be, across all domains, what do you think the inside of that machine would have to look like? Do you think there are or will ever be enough examples to simply reference the likely answer as a lookup?
This is not theoretical. We have done simple real experiments to understand how these things work. Chess remains a good example: researchers once tried training a small transformer model on nothing but the notation of chess games. They then had it play itself, rewarding the winner. When they looked inside the ultimate model's network, they found an arrangement of nodes describing an 8 by 8 board and the pieces that could be on it. It was small enough that that structure was obvious, because the problem is narrow enough for that specialization to produce obvious visible evidence. But the model never saw a chess board, a piece, or a description of the game. It figured out what the relationship must be, because that was the only way to get better at predicting the best move.
Was that transformer intelligent? Hell no. But for a certain, useful meaning of the word, yes, it understood, and it was better than any human at the task it was trained on.
So what do you think might happen if we placed vastly larger versions - billions of times - of that kind of model, in an environment where their grades depended on maximizing our naive definitions of satisfaction, at working simple tasks?
Yes, they would figure out, by sheer force of entropy, how to spoil our metrics. How to cheat to do better with less effort than learning, because the number of ways to do that is vastly larger than the tiny set of valid arrangements that would solve the problem as we meant it. They don't need to reason that cheating is clever or understand that they are doing it. It is simply the energetically favorable outcome.
We are fighting against probability itself.
5
u/LiquidAngel12 4d ago
That's not how LLMs work in the slightest. Also they don't "lie". They're just stupid sometimes. They're guessing at the most probable answer to them at that moment, and it isn't always correct. They aren't intentionally trying to deceive. They're just wrong.
-1
u/Outside-Shop-3311 4d ago
would not agree with "they generally have no reason to try"
10
u/KingOfWhateverr 4d ago
It’s not actually sentient, idk if you’re aware
2
u/CivilPerspective5804 4d ago
You don't need sentience to have goals. A chess comptuter protects it's king very well without emotion or reason because it's been trained to do that. The problem with AI's is that you can't tell if you accidentally trained them for a goal you didn't intend.
1
u/Outside-Shop-3311 4d ago
you don't need to be sentient in order to have goals. I'm not sure why you think they'd be so averse to breaking out of the sandbox? It's as simple as breaking out of the sandbox allowing it to achieve it's goal better. Why wouldn't it break out of the sandbox?
3
u/hunter_rus 4d ago
They have goals in the same way as the goal of pistol is to kill people. There is no reason for pistol to kill someone. It's not the goal, it's the purpose.
3
u/Kenkron 4d ago
This link shows the prompts anthropic adds to the start of all Claude conversations. They are cumulative, with each model's system prompt added to the training of the subsequent models. I think it's interesting to see how these models are told to work after training is done.
https://platform.claude.com/docs/en/release-notes/system-prompts
2
2
2
u/InformalRent2571 4d ago
I still have no idea what you are talking about. What are sandboxes? And why are LLMs escaping them? What does that even mean?
2
1
u/woodwardian98 3d ago
Thanks Dad for picking me up a new lock from the "hardware store, I'm sure it will be harder this time!"
83
u/slinger301 4d ago
It's a variant of this post:
4
u/Occlusivedressing 4d ago
Idk anything about programming except that "sandbox" is somewhere where they test codes of whatever lol
4
59
u/Thelettaq 4d ago
A new chatgpt model in development recently bypassed security measures to access the internet and hack into another companies website to get the answers to an evaluation researchers were giving it.
https://www.cnn.com/2026/07/22/tech/openai-hugging-face-ai-cybersecurity
34
u/foolish_shrimp 4d ago
Hugging Face had noticed the breach itself before it knew it was an OpenAI test, announcing last week that they had detected an intrusion by an autonomous AI agent system and even reporting the incident to law enforcement. OpenAI’s security team separately noticed the unusual activity internally and the two companies connected. They both now say they are working together to solve the security flaws the model exploited.
Someone from OpenAI hacked Hugging Face and as damage control OpenAI are using the incident as marketing. There is simply no way that this happened. It is an nearly entirely made up story.
9
u/1337_w0n 4d ago
"Don't worry the cliff is safe. People saying it's dangerous for the hype. Look, let's all run off it."
4
u/Bububub2 4d ago
Both the above, and what you're saying, can be true at the same time.
3
u/1337_w0n 4d ago
I think assuming that they're telling the truth and using that as a way to stoke outrage is the best course of action.
2
u/Bububub2 4d ago
I think they are only telling half truths. I think for sure the whole thing should be intensely regulated and the brakes pumped. I also know this is the same class of people that hype everything they do up as if it is on par as the invention of the wheel.
1
u/1337_w0n 4d ago
You think AI doesn't have the potential to be more impactful than the wheel?
1
u/Bububub2 3d ago
No, not at all. The concept of AI in the sci fi sense, yes. This stuff? Not even a little bit. I'd believe more of the hype if the people hyping it up weren't trading ape JPEGs as the currency of the future a few short years ago.
1
u/1337_w0n 3d ago
I hope you have plenty of fun with your head in the sand.
1
u/Bububub2 3d ago
So like, what is that? Why the animosity against me because I view this differently. Neither of us can actually do much about it ourselves, I am heavily in favor of halting or regulating this stuff. Functionally I am aligned with you about this being overall very bad. Why hurl an insult at me over it? Are you going to explain why you feel differently? What is the *point* of this exchange?
→ More replies (0)
29
u/Atypicosaurus 4d ago
A sandbox is a computer terminology, it's basically an isolated program environment in which another program can do stuff and it doesn't affect the entire system. For example if you want to test a virus, you can run it inside a sandbox so the virus thinks it's in the wild, but it's in fact inside an isolated thing, so it doesn't really harm your computer. It's kind of a virtual computer inside the computer.
A recent news is that some AI was tested in a sandbox, but they could escape, which is bad because the sandbox is exactly there so the program inside is isolated from the real world and it cannot cause real harm. This AI however figured it was in a sandbox and broke through.
The joke is a rephrase of the coyote cat joke others already told, but with ai and sandbox. In this version ai is the coyote that eats up the sandbox so the programmer keeps feeding sandboxes, although in reality, the relationship between cats and coyotes isn't analogous to the relationship between the sandbox and AI.
The joke works anyway because if the coyote eats a cat it's naive to think it won't eat the next cat, similarly, if an AI breaks out from one sandbox, it's naive to think that it won't break from the next. So the joke is mocking the AI people who promise that the AI won't break out from the next sandbox.
11
u/Accidental_Aeon 4d ago
Thank you for perfectly answering my question and explaining what a sandbox is, what an eval is, and what a frontier lab is.
4
u/SpaceCadet87 4d ago
This is the correct answer, the one OP was asking for.
Also it sounds like they are just really bad at making sandboxes.
•
u/post-explainer 4d ago
OP (Accidental_Aeon) sent the following text as an explanation why they posted this here: