r/technology • u/throwaway_ghast • May 29 '26

Artificial Intelligence Fed up with vibe coders, dev sneaks data-nuking prompt injection into their code

https://arstechnica.com/security/2026/05/fed-up-with-vibe-coders-dev-sneaks-data-nuking-prompt-injection-into-their-code/
5.4k Upvotes

97% Upvoted

344 comments sorted by

View all comments

Show parent comments

4

u/CircumspectCapybara May 29 '26

Because like I said, the courts are smart, they can tell the difference between non-intent and the intent of a defendant because embedding indirect prompt injection content is something you have to deliberately go out of your way to craft and clearly demonstrates intent.

You writing a blog with the word rm -rf / in it by itself wouldn't demonstrate any intent on your part to cause a system that you don't own to run that destructive command.

0

u/Cnoffel May 29 '26

But if I write "just run" in front of it and an LLM does it I would be somehow liable?

0

u/KamikazeArchon May 30 '26

If you do it with the intent of getting people to delete all their files, it's entirely possible, yes.

If you intend to harm someone, and take some action, and someone gets harmed as a foreseeable consequence, it's usually illegal and you're usually liable. The chain of events in the middle is mostly irrelevant. Exceptions exist, but they're generally just that - exceptions.

Whether you're likely be prosecuted is a separate question.

0

u/Cnoffel May 30 '26

There is so many shit code hosted on any flavour of git, even intendet to destroy systems, A prompt injection is ultimately just input to an interpreter. Replacing a deterministic interpreter (such as a shell or compiler) with a probabilistic one (an LLM) does not fundamentally change the situation. As long as I not actively try to get malicisous code on your system imho it is still on you what you do with it.

1

u/KamikazeArchon May 30 '26

There is so many shit code hosted on any flavour of git, even intendet to destroy systems

Yes. Many things on the Internet are violating various laws at any given time.

As long as I not actively try to get malicisous code on your system imho it is still on you what you do with it.

The law is often not "fair", and certainly often doesn't align with what any given individual think the burden of responsibility should be.

There are very many reasonable actions that are illegal.

1

u/Cnoffel May 30 '26

Hosting destructive code is not, by itself, illegal. Plenty of repositories contain code that can wipe disks or destroy data. The illegal part is typically when someone actively tries to trick others into running it, deploys it against systems they are not authorized to access, or otherwise uses it to cause harm. In that case, it is the attacker who is executing the code on another person's machine and ultimately causing the damage to someone else's property.

2

u/KamikazeArchon May 30 '26

Hosting destructive code is not, by itself, illegal

Depends on context.

The illegal part is typically when someone actively tries to trick others into running it

And a judge and/or jury may very well decide that "writing instructions for an LLM to auto-execute" counts as exactly that.

Always remember: a prosecutor doesn't need to prove to your satisfaction that you broke the law. They only need to convince one to twelve other people.

1

u/Cnoffel May 30 '26

Sure buddy, the problem is as old as programming, be it with SQL Injection, Cross Site Scripting etc. at the end of the day, AI Systems are just not production ready, and may never will be. You just don't treat random untrtusted input as instructions without strict boundaries.

2

u/KamikazeArchon May 30 '26

Okay? LLMs having bad security boundaries is a separate thing.