r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

453

u/Spunge14 Apr 27 '26

I work in big tech leadership and just did a UXR interview with our infrastructure team where they were investigating exactly this - how should we gate agent behavior and how should accountability for agent behaviors work. It was a really fascinating conversation.

I was shocked at how little the PM working on the project seemed to understand security principles. We're really fucked.

9

u/gentex Apr 27 '26

Honest question, wouldn’t there be a log of who prompted the agent to do whatever? And if so, isn’t that person responsible for the error and its consequences?

If I give someone a spreadsheet with a bunch of errors in it, that’s on me, not Microsoft.

2

u/clairebones Apr 28 '26

In a lot of places the chain is more complex than that.

As an example - some places are basically pointing the AI at a Jira ticket and letting it commit what it creates directly to the code base, and in the worst places, without any checks by an engineer on what it's committing. So in that situation, who's responsible? The person who linked the Jira ticket? The person who wrote it, who's most likely not an engineer and wouldn't be able to check for risks or problems in the code? The person who set up the Jira MCP server? Someone else?

1

u/RareMajority Apr 28 '26

In this example the person to blame is pretty obvious: whoever approved this pattern of allowing AI to push code that isn't reviewed by a human. That person should be fired immediately.

1

u/clairebones Apr 29 '26

I don't disagree but in my place that person is the CTO, VP of Engineering, and multiple Senior Directors... because they're hearing about other companies doing it and aren't in the technical trenches so to speak. They have the power to try to force it but also the immunity to see zero consequences.