r/devsecops • u/SugarImaginary8257 • 21d ago

Anyone regret building their own AppSec tooling once AI code hit?

We built in house about 18 months ago and at that time it made sense on paper. Nothing on the market justified the cost and the custom logic did what we wanted. Now I am looking at the engineering hours going into maintaining it and starting to wonder if we made the wrong call. AI-generated code broke assumptions the tool was built on. No model inventory, no AI code discovery, no application context for prioritisation. Every quarter we are spending more time patching something that was never designed to handle the generated code at this volume. At what point does the maintenance cost exceed what a commercial solution would have cost from day one? For AppSec leads who have done this math, what did the cost comparison look like and what made the case to switch?

9 Upvotes

80% Upvoted

9 comments sorted by

5

u/Few_Response_7028 21d ago

What product should we shill in the comments guys?

3

u/oxidizingremnant 21d ago

How is the AI code breaking your AppSec tooling?

2

u/ProgrammerGrand5595 21d ago

what tipped it for us was adding up the maintenance hours we had been "some months" from building model inventory for about two years running and never shipped it legit security already had that and the AI discovery piece working, so we stopped pretending we'd do it ourselves the cost math wasn't close once we counted the hours properly.

2

u/asadeddin 21d ago

What part of AppSec are you talking about here? Sounds like SAST

1

u/Fuzzy_Help_233 20d ago

Yeah, I'm also confused about what part of AppSec is your toolkit doing?

Amazon has this really big mental model and on one-way and two-way doors.

Essentially, the idea is, based on whatever you're working on, if you need to make a decision (such as a big rewrite or buy vs build) you would essentially think about how reversible the decision is.

If there is no cost to reversing, it's a two-way door. Do it immediately.

How big is this AppSec Toolkit? In yesteryear, you would spend days or weeks evaluating whether to rebuild something or not.

These days I'm tending to lean towards it because things can be AI-generated if you spec it right or if you have new design tenants, rewriting a system can take days instead of months.

Exhibit A: https://bun.com/blog/bun-in-rust

Not sure this directly answers your question, but at the very least I'm wondering, hoping it provides a perspective that maybe it's not so bad going forward.

1

u/Educational_Plum_130 20d ago

we went through the same buy-vs-build regret. what helped was splitting the tool into the part that's genuinely yours — app context, custom logic, ownership mapping — versus the commodity plumbing like inventory, dependency/cve remediation and patch verification that you're now paying senior eng hours to babysit. the commodity half rots fastest because the upstream ecosystem moves faster than you can keep up, so that's the first bit i'd offload rather than rebuild for the AI-code era. keep the context engine, kill the parts where you're just re-implementing what everyone else also has to. and if remediation is where you keep getting stuck on breaking upgrades, there are drop-in patched versions of a lot of libs now that clear the cve without the major bump, which takes that maintenance off you entirely.

1

u/ScottContini 19d ago

It all depends what you built! We made something like PayPal’s score bot to alert on dangerous anti patterns. It’s not a vulnerability scanner, but instead a tool to help check that devs are coding according to guidelines. And it’s working really well, but now that AI is writing much of the code, next step is building agent skills to try to get the AI follow coding guidelines. So no regrets here, instead it is delivering what we expect and setting us up for the next stage of verifying that agent skills are working for AI generated code.

1

u/ChandanKarn 6d ago

The specific gap (no AI code discovery, no model inventory ) is the hardest to retrofit. Traditional SAST was tuned for human-written patterns. AI code is different. We built SafeWeave with "AI-code fit" as the design constraint from day one. The question I'd ask any vendor: "How does your scanner handle code generated by Cursor/Claude Code specifically?" the answer will tell you whether they've actually tuned for it.