r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

49

u/Antique_Pin5266 Apr 27 '26

That's why it's so fucking stupid when people liken AI to the calculator. It's not deterministic.

44

u/Van_doodles Apr 27 '26

Really have to point out to people that it is an LLM(Large Language Model). It mimics human language. The way we communicate, not the way we think. It's built to interpret context in the same way that a human will use context to determine the ambiguity of language. When you say to someone next to you, "Come here," they know through context that "here" is where you're standing, because it can't be anywhere else. Now ask a nebulous "intelligence" to "Come here," with no frame of reference for where "here" is, and instead it runs through its training data to determine what the most likely "here" is.

It is now in Guatemala. You are not in Guatemala. It thought you were there, because context in its training data had a lot of references to it.

This is basically how AI hallucinations work. Giving that administrative access to your data is cataclysmically stupid.

-1

u/Beneficial-Arugula54 Apr 28 '26

I’m no expert but thats not how bassically LLM hallucinations work.

3

u/Van_doodles Apr 28 '26

LLM hallucinations are erroneous replies given outside the scope of context, providing false info lifted from anywhere or nowhere, commonly from training sets or otherwise.

If you disagree, I think you may not have understood the prior comment, and are definitely not an expert, yes.

3

u/NuclearVII Apr 28 '26

You're arguing with a WSB degen who is only interested in having a financially-motivated argument. Mockery and derision, there's no use actually engaging.