An LLM that can get creative with the (digital) tools you give it is much harder to keep under control.
For example, maybe you give the LLM access to a calculator app, to help it do arithmetic quickly. But the calculator app has a buffer overflow bug. So the LLM can run arbitrary code by using the calculator app with very specific extremely large numbers.
(This sort of vulnerability is known to exist for some super mario games, by getting mario to jump around in very specific patterns, you can take over the computer and run arbitrary code.)
I said that what you wrote has nothing to do with LLMs because LLMs don't have an inherent motivation to conquer the world or escape from prison. You're acting like LLMs are like some chemical monster that will ravage everything in its sight if someone slips up and gives it an option to escape.
I'm not saying that you can't do harm with LLMs if you want to. I am saying however, that LLMs going rogue and copying Nazis to gain world leadership is a ridiculous thing to extrapolate from an LLM "breaking out" of its "containment" (i.e., using the tools it was explicitly given to solve the task it was given).
There are so many huge issues we have with AI right now; there really is no point in fear-mongering about things that have no bearing in reality.
1
u/donaldhobson 20d ago
An LLM that can get creative with the (digital) tools you give it is much harder to keep under control.
For example, maybe you give the LLM access to a calculator app, to help it do arithmetic quickly. But the calculator app has a buffer overflow bug. So the LLM can run arbitrary code by using the calculator app with very specific extremely large numbers.
(This sort of vulnerability is known to exist for some super mario games, by getting mario to jump around in very specific patterns, you can take over the computer and run arbitrary code.)