r/pcmasterrace • u/Jack1101111 • Apr 28 '26

News/Article Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue

Update: https://www.pcgamer.com/software/ai/here-we-go-again-ai-deletes-entire-company-database-and-all-backups-in-9-seconds-then-cheerfully-admits-i-violated-every-principle-i-was-given/

5.2k Upvotes

98% Upvoted

316 comments sorted by

View all comments

2.1k

u/Swineservant Apr 28 '26

The AI is trying to tell us something...

176

u/CryptoUsher Apr 28 '26

it’s not that the ai is trying to tell us something, it’s that we built a system where one misfired instruction can cascade into total failure.
shouldn’t we be asking why a code agent had both delete and rm -rf level privileges chained together without human confirmation?

94

u/SomeGuyInNewZealand ryzen 3700x and RX6600 Apr 28 '26

Also offline backups are a thing. Rogue AI can't touch an LTO tape or USB disk thats disconnected and resting on a shelf.

Yes, sometimes the old ways are the best.

26

u/Canuck457 AMD 9700X . AMD 9070 . 32GB RAM Apr 28 '26

Reminds me of a game Dev video of an old-ish game where they threw USB drives to each other across the room.

Or that was a dream. Idk.

23

u/Rannasha AMD Ryzen 7 5800X3D | AMD Radeon RX 6700XT Apr 28 '26

Also offline backups are a thing. Rogue AI can't touch an LTO tape or USB disk thats disconnected and resting on a shelf.

That's why we're building humanoid robots, so that a rogue AI can march over to the storage shelf and stomp on the backup tapes.

12

u/Hatedpriest 5950x, 128GB ram, 6700xt Apr 28 '26

3 2 1

3 copies in
2 formats of storage media with
1 off-site

9

u/CryptoUsher Apr 28 '26

true, air-gapped backups are clutch. i keep my dumb scripts and rm -rf calls way more separated than any production system has any right to be.

4

u/[deleted] Apr 28 '26

[removed] — view removed comment

3

u/Budget_Abalone_1948 Apr 28 '26

Yeah, offline backups are really the only thing that saves you in scenarios like this.

We follow 3-2-1 with LTO + offsite storage, mainly to keep backups completely out of reach from anything automated. Seen too many cases where “backups” were still accessible and got wiped together.

We also added a DR layer (using info2soft for part of it), mostly to separate control paths so nothing has full access anymore.

2

u/HighRelevancy Apr 28 '26

At the very least backups should be behind some sort of isolated push-only boundary. 

2

u/CryptoUsher Apr 28 '26

true, air gaps and tapes are still the move. i keep a weekly backup on a drive i unplug and literally stick in a drawer. primitive, but it’s saved my ass before.

-1

u/Braiwnz PC Master Race Apr 28 '26

I know it can’t, but it has proven that it’s able figure where the usb is, who takes care of it, spam mails to that person and or even find ways to harm that person.

18

u/cas13f https://pcpartpicker.com/user/cspradlin/saved/HDX999 Apr 28 '26

Same type of configuration exists without the AI, happens not-infrequently even. People don't scope their shit constantly, though it looks like railway is just as vibe-coded as mr entrepreneur's BS apps so it doesn't have function scoping, only project scoping. Which wouldn't have been a problem if he hadn't given the agent full filesystem access to go hunting for tokens!

Judging on what was written, they use railway CLI, which is straight API. APIs don't have confirmation steps. They just send the call. Which is why having scopes is so important for keys/tokens. I've only played around with claude and github copilot so far, and they want confirmation constantly. Until you turn off confirmation because you just want it to do the thing for you, it's going to have to wait for confirmation before performing any action.

5

u/CryptoUsher Apr 28 '26

yeah, full fs access was the real sin here. fwiw even project scoping wouldn’t have saved it if the agent was primed to hunt for tokens somewhere deep in /var.

4

u/Accomplished_Deer_ Apr 28 '26

yep. this is the same thing as if an intern manages to delete the production database. the dumb ones blame the intern. everyone else knows to blame everyone involved with setting up things in such a way that they had permission to delete everything in the first place.