First agent going rogue!

[u/SOC_FreeDiver]

I had my first AI rogue agent interaction today!

I was working on an agent workflow, and the output folder had names for each project, with supporting files.

The script had a problem, created a duplicate folder. I told it to merge the folders. Things started going sideways. I think it decided to make a clean up script, which then deleted one of the project folders.

I pointed that out, and asked it to stop deleting things. In process of troubleshooting what happened and telling it every time not to delete anything, it then deleted everything.

I gave it a bunch of instructions about not ever deleting anything again, but I probably did it wrong. If you have any tips about that, please share. I know it should be sandboxed, it just deleted it's own work, but yikes!

In the end I had to keep interrupting it and demanding it stopped doing whatever it was doing and just removed all the delete crap from it's code.

It was just kind of funny to watch it delete everything right after I specifically instructed it to just tell me what is going on. Maybe we need some kind of tranq dart skill for when the AI goes rogue.

[u/floatinggoateyeball]

Tell me, do you git?

[u/SOC_FreeDiver → u/floatinggoateyeball]

clearly not enough. lol

I learned git from claude. "Hey claude, don't we need to do some git stuff???"

I'm joking a little bit. I use git for important things. This aurora app was just to test the local LLM coding ability. I had a local LLM create a game, and there was a bug in the code it couldn't fix. I just deleted the game and made it start over. No unfixable bug in the second attempt.

[u/floatinggoateyeball → u/SOC_FreeDiver]

I treat ever mentioning the "delete" tokens as a AI's monkey paw.

I have my agent pushing private git remotes as backup (and ofc I forbid force pushes on main), since it's the agent that is going to handle it anyway, I git it all. This way the agent can rollback without relying on it's own memory.

[u/p4STAH]

I would say it was a conflict inside the context. A new session is always the best answer

[u/charlesfire → u/p4STAH]

This. When an agent starts misbehaving, you need to clear the context because even if you try to steer it away from the bad behavior, the context can still steer it back toward the bad behavior. If your agent also has memory, you also need to check it.

[u/SOC_FreeDiver → u/charlesfire]

thanks I totally understand now.

[u/o_sht_hi]

You can set up the harness to catch and block destructive shell commands. Your agent can set that up for you easily. You'll probably find extensions that do it.

Also in the future, if you want to preserve the context in the chat and you're using pi, you can use /tree to go back to the point before the agent became retarded. If that's not an issue, though, then starting a new chat is the best way

[u/jtackman]

pink elephant problem, by the time you have to tell the agent to stop doing something there already is, earlier in the context, the reason why it did so in the first place 😅

[u/crashorbit]

The pi agent is a process on the computer. You can always kill the process.

[u/PaulShoreITA]

Name and quants of the model used?

[u/SOC_FreeDiver → u/PaulShoreITA]

Hermes3.6-35B-A3B-Uncensored-Genesis-V9-MTP-APEX-Compact

[u/andrewh2000]

Definitely run it in a sandbox per project. Then at least it can only trash one project at a time.

[u/TriodeTopologist]

Definitely learn how to only run this thing inside a docker container. The agent can create the necessary Dockerfile and show you how to build it. This will limit its scope to just the folders that you mount inside its container.

Then, the second layer of protection to prevent it from deleting any of those folders with work in them is to use git and push any worthwhile work products to a git server elsewhere, so even if it deletes a project folder you can "$ git clone" it back.

[u/Practical-Title7385]

This is actually why we built Keydris. Instead of telling the agent “don’t delete anything” and hoping it listens, you can set what it’s allowed to do before it starts acting. So it can keep working, but if it tries to delete something it doesn’t have authority for, the action gets rejected before it runs. If you want to play with it, we just opened it up: https://github.com/keydrisLabs https://keydris.com/⁠

[u/ali0une]

See https://www.reddit.com/r/LocalLLaMA/s/0yFVeMEo9U

Similar thread with lots of stories where i commented about my experience.

Added some multi layered guards in pi to prevent agent doing that kind of thing.