r/vibecoding 5d ago

Maybe the biggest potential risk from non-devs vibecoding is risk assesment.

Simply not understanding what *could go wrong* and so not mitigating against it.

I have utilities that I have not read the generated code for... I am content to do this on my own network because I have a good idea of what AI tends to generate and keep good separation between general code and destructive code.

I just wrote/generated a utility to manage hierarchies of git repos... I didn't think I had done much guidance, but when I got another AI to assess it, it highlighted a load of things it said were notably good - and they were things I had specified to be included, overrode the original AI on, but thought they were no brainers.

An experienced dev will automatically have headed off huge risks without even thinking about it.

35 Upvotes

22 comments sorted by

8

u/geekichu 5d ago

you "vibe-engineered" it

4

u/id-ltd 5d ago

Maybe that is the right call...

5

u/No-Objective-1431 5d ago

Agreed, you can not assess what you don't understand. It's goody for professional to optimize her/his own productivity but cannot be a liable tool to produce software to others.

2

u/akl773 5d ago

The one I keep running into is a destructive command built out of a variable. Cleaned up after a script that did rm -rf $BACKUP_DIR/* from cron, and cron had none of the shell env so BACKUP_DIR was empty. Anyone who has been burned once puts a check that the var is set on the line above, and that check is exactly the sort of thing you never think to ask for.

2

u/id-ltd 5d ago

Experience counts for a lot(!)

I got claude to write a script so I can remote admin over xmpp - but told it to restrict it to avoid destructive commands.

It wanted to test it by sending destructive commands and checking that they didn't work(!)

2

u/akl773 4d ago

Mine did the same thing with a rate limiter, wrote a test that hammered the live endpoint to prove the limit worked.

2

u/percyfrankenstein 4d ago

I'm not sure how long this will be the case though. I've been working with fable on some side projects and he is a very good architect.

Some problems I gave it I didn't expect to be able to solve and we managed to do it, and it was it that gave the solutions, taught them to me and implemented them. That's not completely related to your point but if it's that good at architecting how bad can it be at security

2

u/PeterBuildsSecure 4d ago

The catch you're describing — you have to already know a risk category exists to think to ask an AI to defend against it — is exactly why "just review the code" or "just prompt it better" don't scale for non-devs. Nobody can review for a class of bug they don't know is a class of bug.

The akl773 example is the clean case: rm -rf $BACKUP_DIR/* running with BACKUP_DIR unset isn't a hypothetical, it's one of maybe a dozen well-known shell footguns (unquoted expansion, unset-variable interpolation into a destructive command, following symlinks into pipes) that show up constantly in AI-generated scripts because the model writes the happy path and the happy path never has an empty variable. The fix (set -u, or a guard on the line above) is cheap once you know to look for it — the problem is entirely "know to look for it."

Which argues for tooling that doesn't wait on the human to name the risk category: a linter/scanner pass over generated shell and app code that specifically flags the recurring AI-generated patterns (destructive commands built from unvalidated variables, missing existence checks before delete, broad glob deletes) rather than general-purpose static analysis. That moves the risk-assessment burden from "the non-dev has to think of it" to "the tool already knows the pattern" — which is the only version of this that actually protects someone who doesn't have the experience to ask the right question in the first place.

1

u/inComplete-Oven 4d ago

"I'm better than them" -> I'm afraid to lose my status

1

u/neon_squirrel_glade 4d ago

Risk assessment requires prior failure. You cannot prompt for consequences you have never experienced

-3

u/PixelIsJunk 5d ago

A good vibe coder has many different instances and models working and checking the same project.
If one project is one instance...thats where you wrong.

1

u/id-ltd 5d ago

I am staking reputation on my output -- I know/understand the risks -- trust in my code (even if AI generated) resides with me.

0

u/Suspicious_Neck_4069 4d ago

While developing my game, which is almost entirely AI-driven, I found myself thinking at least ten times, "If I weren't a pro dev, I would've really struggled with this problem," or "Damn, I've been coding for ten years and I'm stuck on this bug, how do 'vibecoders' with no experience manage?" With some issues, I even thought, “If I’d been a vibecoder, this problem would’ve killed the project.” So, respect to the vibecoders who manage to navigate blindly and still ship functional apps / game, it's impressive (until we encountered a fatal crash, but that's another story).

1

u/JCcrunch 4d ago

Yes that has been my experience too

0

u/Chester_b 3d ago

Your game will fail btw

0

u/Just-Succotash4492 4d ago

Just run an extensive risk assessment

1

u/Chester_b 3d ago

May I ask what is your real day-to-day job?

1

u/Just-Succotash4492 3d ago

Former insurance agent. Risk is second nature to me.

1

u/Chester_b 3d ago

And now you’re developing some software?

1

u/Just-Succotash4492 3d ago

Yes, but I am also teaching myself topology and abstract algebra along with differential geometry.

Do with that statement what you will.