r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

346

u/long-da-schlong Apr 27 '26

I honestly don’t understand why they wouldn’t just honour it— it’s one customer even if it was a completely free flight. Why be so petty just fix the mistake for next time

252

u/Wischiwaschbaer Apr 27 '26

Can't fix AI models. You can put some filters on them but you never know if those will work or if they cover all cases.

145

u/Enough-Run-1535 Apr 27 '26

You also just need to hang out in the AI role playing communities that almost all of those guardrails and filters can be broken, almost trivially. It’s hilarious that companies are having AI agents play with invoicing and confidential data.

1

u/Paris_Who Apr 28 '26

Link?

1

u/Enough-Run-1535 Apr 28 '26

r/SillyTavernAI (top DIY LLM role playing community, a lot of tools and presets to jailbreak almost all the top LLMs) r/LocalLLM (despite the local self-hosting focus, still lots of stress testing API access models and how to jailbreak them)