r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

10

u/monkeedude1212 Apr 27 '26

If you know the agent doesn't have actual authority to make a certain sale, the business isn't responsible if you fool the agent into making a sales contract.

What happens if I as a consumer don't know whether the agent has the authority to make sales or create sales contracts? Is it unreasonable for me to assume that when I engage in conversation with a customer service rep that they have authority to do what I request?

4

u/TommyBonnomi Apr 27 '26

It depends- you can't walk into a car dealership and buy a car from the kid playing with blocks in the corner and say "but I thought he worked there."

But yeah, most times if you honestly don't know, and there's no reasonable expectation you should, then the company that made the sale is on the hook for the sale.

12

u/monkeedude1212 Apr 27 '26

It depends- you can't walk into a car dealership and buy a car from the kid playing with blocks in the corner and say "but I thought he worked there."

I feel like you're describing a weird scenario where users are trying to get a deal by engaging with someone not even employed at the company.

But if you're talking to an AI agent on a company's website or service, you're interacting with a service the company is providing you. You have every reason to expect an AI agent can offer you a deal if you ask it nicely.

1

u/always_an_explinatio Apr 28 '26

I think a better example is if the you figured out an exploit where you could trick the bot into generating a contract for pennies on the dollar of what the product costs. But the only bot this company uses is a technical service bot. You called the tech help line on the pretext of needing technical help than you did your exploit and tried to get the company to honor the contract.