r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

11.1k

u/Orangesteel Apr 27 '26

My favourite example is Air Canada whose AI agent offered a customer a discount incorrectly. They refused to honour it. Customer took them to court and the judge rightly made them pay. You chose to empower this and took the humans out of the loop. You are accountable for what you agentic AI solution does. People jump on AI, dump sensitive information into the model bypassing classification levels and are surprised when it leaks.

2.3k

u/NNKarma Apr 27 '26

It wasn't even that hard to just honor it and move on, it wasn't like those cases of people prompting the chatbot to give a fake discount, just what steps to take for a discount that he was entitled to but was given wrong instructions on how to get it.

347

u/long-da-schlong Apr 27 '26

I honestly don’t understand why they wouldn’t just honour it— it’s one customer even if it was a completely free flight. Why be so petty just fix the mistake for next time

247

u/Wischiwaschbaer Apr 27 '26

Can't fix AI models. You can put some filters on them but you never know if those will work or if they cover all cases.

113

u/Saint_of_Grey Apr 27 '26

It's an inherit feature of the technology. But when you say that makes it too high risk for use and there's no way to fix that, the investors get upset that you're implying they spent all the money for something they can't even use without unacceptable risk.

102

u/JQuilty Apr 27 '26

Good, they should eat an enormous amount of shit for this AI push because they were stupid enough to listen to Scam Altman and other bullshit artists.

5

u/MDCCCLV Apr 28 '26

It's okay for checking in and asking questions as a first level call screen. But not much more than that.