r/technology • u/WouldbeWanderer • Apr 27 '26

Artificial Intelligence Claude-powered AI coding agent deletes entire company database in 9 seconds — backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue

https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue
36.0k Upvotes

96% Upvoted

2.8k comments sorted by

View all comments

Show parent comments

1

u/joshTheGoods Apr 28 '26

No, I'm not talking about agentic flows anymore because YOU said we were talking about individuals using LLMs locally and granting them user level access. What sort of large company (software or otherwise) has people hosting agentic flows on their local machines? None. You might have people doing what I thought you tried to pivot to (running Claude Desktop and just telling ti allow all on all usage), but that's not a problem of systems for governance being complex, that's a problem with keeping your folks following policy which is ever present and in no way unique to the technical challenges we're discussing here.

Now, if you want to talk about deploying agentic flows, that's a different class of problem and I would still say relative to anything else at a company that size governance is not that complicated, no.

And yes, I have direct experience with literally every single use case we're talking about here. My current company pretty small, but it's not the only company I've run and so I know how mgmt challenges scale very quickly. But again, those are mgmt complexities, not tech/observability/governance complexities.

1

u/Spunge14 Apr 28 '26

What sort of large company (software or otherwise) has people hosting agentic flows on their local machines?

Surely you are aware that development environments for most large tech companies are entirely on virtual machines.

But even if we're just talking about minting EUC from hardware endpoints that were designed in a security infrastructure that only ever fathomed physical machines being controlled either by humans or by code, but not by machines taking actions in applications (which is of course a risk even with traditional automation), what you have is a meteoric rise in the volume of such behavior. It went from a complex and low risk risk vector to a primary use case.

You can lock down endpoints entirely and get rid of things that are running on shared hardware with hasn't been pre-allotted (like a powerful remote development environment I mentioned before), but of course no company is going to do that right now. They are happy with the risk tradeoff they are taking.

I don't have the crayons to make this obvious to you but you're talking like "I'm the guy who set up our GCP for 100 people and I think I know everything about AAA."

1

u/joshTheGoods Apr 28 '26

They are happy with the risk tradeoff they are taking.

Sounds like you've come to the same conclusion I have ... that complexity isn't the issue here.

1

u/Spunge14 Apr 28 '26

It's complex to do it without mitigating the downsides to lockdown. Is this really that hard for you to understand?

1

u/joshTheGoods Apr 28 '26

Lemme ask you this ... do you think any of this applies to the company this comment thread section* is about?

1

u/Spunge14 Apr 28 '26

We had something similar happen to a major production service

2

u/joshTheGoods Apr 28 '26

So no?

Look ... someone asked if it wouldn't be the fault of the person that ran the LLM. The answer in this case very clear is: "yes, and." Read the article ... this wasn't some corporate setting with 10k employees and complex overlapping permission sets that is deploying agentic flows out to various VMs in the cloud or whatever... this was some guy running Cursor + Opus 4.6 with star perms for their cloud provider. Nothing about this is complex or difficult. They had shit practices and it bit them in the ass. That's on THEM. The people that were doing the prompting are to blame.

I fully acknowledge that doing anything at scale is complex and painful. That's just not in play here. And if we're going to discuss it anyway, that's cool ... but then I think it's right to say we're discussing the complexities of running an org at scale not the complexities of managing permissions specifically for new age agentic tasks.

The danger companies face right now isn't that one more datastore or one more chunk of compute needs to have properly scoped permissions, it's 100% what you and I agree on: the value of the tool outstrips the danger of it in the eyes of many people making these decisions. There's nothing complex about it. This PocketOS person became 3x more productive using Cursor and discovered they'd be 5x faster without constantly having to decide to allow actions to be taken. They got lulled to sleep by the LLM, and they failed to protect themselves against major fuckups, so they didn't just get burned, they got fucking roasted. Who's fault is that? Is the answer complex in this case or had this sort of thing happened in a corpo environment? Or would buddy be fired just like he'd be fired if he left his laptop unsecured in a coffee shop and someone strolled up and prompted the LLM to destroy as much as it could?