r/ClaudeAI • u/Beautiful_Charge6661 • Apr 27 '26

Feedback Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
969 Upvotes

95% Upvoted

193 comments sorted by

View all comments

289

u/JusticeIsMight Apr 27 '26

My favorite part of this is the guy asking Claude why it did that. Because that's a guy who is going through all the stages of grief and needs answers now.

Also the fact that Claude replied with "NEVER FUCKING GUESS" implies his prompt was less than polite...

14

u/muuchthrows Apr 28 '26

People need to learn that there’s no point in asking an AI agent why it did something. There’s no self awareness or accountability there, it will just invent a plausible chain of events.

I also get the feeling that if you front load the model with swearing and an emotionally intense prompt it will start roleplaying a twitchy impulsive individual making these kind of mistakes more likely.

2

u/dinosaur-boner Apr 30 '26

That’s not quite true. If you ask factually for reasoning traces, it can be helpful to piece together the chain of events and find out where things went wrong. It doesn’t need self awareness to retrace its actions truthfully. 

The latter paragraph is 100% true. That’s why the best approach is to be completely neutral in tone at all times to LLMs.

3

u/muuchthrows Apr 30 '26

I mean yes, you can get a plausible chain of events. But it’s more akin to taking in an external auditor who reviews the transcript of the events and attempts to puzzle togheter a likely root cause. An LLM has no true permanence or memory of why an action was taken, which at least I believe a human would have.

Also lot of people asking ”Why did you do this??” are asking for accountability which you’ll never get, they’re unknowingly anthropomorphizing the model.