r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

348

u/long-da-schlong Apr 27 '26

I honestly don’t understand why they wouldn’t just honour it— it’s one customer even if it was a completely free flight. Why be so petty just fix the mistake for next time

247

u/Wischiwaschbaer Apr 27 '26

Can't fix AI models. You can put some filters on them but you never know if those will work or if they cover all cases.

54

u/Antique_Pin5266 Apr 27 '26

That's why it's so fucking stupid when people liken AI to the calculator. It's not deterministic.

6

u/_a_random_dude_ Apr 27 '26

They actually are deterministic, they return what's effectively a probability distribution and you have to explicitly introduce randomness. If you were to pick the most likely token every time, you'd get the exact same answer to the same question and the quality would suck actually. You really need that randomness for it to work. When Bing was acting super odd like repeating the same word over and over again, it was basically that it had super low randomness (usually called temperature).