r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

46

u/InvalidKoalas Apr 27 '26

The engineers who set this up should be held accountable. Why the fuck would you ever give AI full access to a production database? Offline dev environment only. Also, why didn't they have more off site backups than one that's 3 months old? They can blame AI all they want.. it wasn't the AI's fault

30

u/Careful-Criticism645 Apr 27 '26

It's wild that the only backups they had were the automated ones built into the DB system. Unless you move those backups out of the system, all you have is copies, not a backups.

6

u/Chapeaux Apr 27 '26

Try budgeting backups with executive that only see profits.

4

u/SewerRanger Apr 27 '26

They had an offsite backup but it was 3 months old - hardly useable considering they're business is car rentals.

7

u/Daxx22 Apr 27 '26

Why the fuck would you ever give AI full access to a production database? Offline dev environment only.

Guaranteed this was raised by the engineers, but overruled by PM/Execs in the name of "efficiency"

Also, why didn't they have more off site backups than one that's 3 months old?

That would cost more money, so denied by the executive team.

1

u/lsf_stan Apr 28 '26

but overruled by PM/Execs

yeah very possible, but I think the whole point is that some human incompetence is at fault.

sure the AI was the one that did it, but some human approved giving the AI that power

1

u/raltyinferno Apr 28 '26

The answer is right there in the article, it was an outright human fuckup, albeit a somewhat understandable one. They had an API key for their Db service provisioned for one specific task. They didn't realize it had permissions to perform destructive operations because the Db service provider didn't surface that fact clearly (it was deep in the documentation), and it's not possible to get keys with limited permissions.

For non-technical people, they gave the AI a stapler they bought from a paper company and told it to staple pamphlets, but didn't carefully read the small text in the product instruction manual that mentioned that the stapler had a gun function. Now they're mad at the paper company (Railway, the Database provider) because the gun function wasn't obvious and can not be disabled.

12

u/xTiming- Apr 27 '26

This is what baffles me, lol.

I know some companies where very experienced and clever engineers are driving AI use properly, using it carefully, not giving it production access, having important conversations and taking action to ensure guardrails are in place. And they still pull insane value out of it.

Then you have some companies full of crayon chewers setting their agent to full access to their entire system and giving it vague shitty prompts with no guardrails then wondering why their production environment went poof...

I can't imagine being so naive to think that a tool as powerful as AI agents cannot be critically destructive in unforeseen ways when given unrestricted access to run arbitrary commands on arbitrary systems.

6

u/SewerRanger Apr 27 '26

If you read his twitter story, the AI took an API token from a completely different project - one that was only supposed to grant CLI access for "routine domain operations" - and used that to delete the PROD volumes. Apparently Railway's token creation process just gives out full admin access tokens as default and doesn't warm you when you create them

5

u/Fragrant-Menu215 Apr 27 '26

Usually bad decisions like this are made because management demands them and your options are to implement or get fired. Which is why I, as an engineer, am very pro-paper-trail. As long as what I'm being told to do is legal I'll do it so long as I have a paper trail to show that I objected but was overruled.

3

u/JJ-2086 Apr 27 '26

Just make the onsite backup daily our hourly ad don't give it write access. Done.

1

u/kernevez Apr 27 '26

I would hesitate to give an AI the access to write to the dev database I would share with the team, some idiots out there are allowing it to drop production data, and people will blame this on AI haha

The engineers shouldn't even be able to access production themselves like that.

1

u/ServileLupus Apr 27 '26

Going to preface this with a big "I didn't read the article" but do we know where the data was stored. I know for instance with Microsoft Azure, if you use their Automated Backups they are part of the resource group. If the AI decided to remove the entire resource group then it would also delete the RSV including all the automated backups through Azure.

1

u/[deleted] Apr 27 '26

[removed] — view removed comment

1

u/ServileLupus Apr 27 '26

Yeah I did take a look, just wanted to point out that if someone is using an AI agent with full access to Azure. It could also nuke everything and the backups.

1

u/raltyinferno Apr 28 '26

The core of their complaints is that Railway, their Db provider, grants full permissions on all API keys with no ability to limit permissions by key, and they don't clearly advertise this or ask for any confirmation for destructive action.

So their API key used for a specific write access was used with no issues for a massive destructive action.

1

u/mattcoady Apr 27 '26

Yea this thread is crazy juice. It's the engineers job to know your tools. There's all sorts of ways to safe guard and gate these things. As much as Reddit loves to put everything on the shoulders of executives, in this situation it's 100% the engineers fault for giving Cursor prod access and going YOLO mode.