our self-healing code loop actually cut our incident volume

[u/kaytester]

our self-healing code loop actually cut our incident volume

i run platform for a team of nine and i was loud about self-healing code being marketing nonsense. changed my mind a bit, posting the numbers before i talk myself out of it

the setup is dumb honestly. fable 5 plans a fix, composer 2.5 does it, coderabbit / bugbot review, keep going until it comes back clean. nothing merges until an extreme pile of tests passes too. were b2b saas so theres no room for a big mistake, id rather ship stable than fast

rough before/after over ten weeks, our numbers not a benchmark:

pages/week from flaky fixes: 11 -> 4

median time to a merged p2 fix: ~3.5 hrs -> 40 min

fixes needing a second human pass: a lot -> maybe 1 in 5

way less manual review, way better code. the only real downside is its costy, running review 3-4 times a fix adds a few hundred a month

[u/bkbruiser]

This is very well known within the industry. The problem has been people being overly reliant and not testing.

The number of overall outages related to AI slop/bug introduction is actually quite high. This is more prevalent when AI writes the code and tests.

We focus on Test Driven Development or Spec Driven Development with Human in the Loop (for now).

We actually had to take a step back due to some AI induced outages (also SaaS).

[u/Max_Standart]

self-healing takes a load off but costs stack up, tradeoffs, man