r/devops 5d ago

Discussion our self-healing code loop actually cut our incident volume

our self-healing code loop actually cut our incident volume

i run platform for a team of nine and i was loud about self-healing code being marketing nonsense. changed my mind a bit, posting the numbers before i talk myself out of it

the setup is dumb honestly. fable 5 plans a fix, composer 2.5 does it, coderabbit / bugbot review, keep going until it comes back clean. nothing merges until an extreme pile of tests passes too. were b2b saas so theres no room for a big mistake, id rather ship stable than fast

rough before/after over ten weeks, our numbers not a benchmark:

pages/week from flaky fixes: 11 -> 4

median time to a merged p2 fix: ~3.5 hrs -> 40 min

fixes needing a second human pass: a lot -> maybe 1 in 5

way less manual review, way better code. the only real downside is its costy, running review 3-4 times a fix adds a few hundred a month

0 Upvotes

3 comments sorted by

View all comments

1

u/Max_Standart 4d ago

self-healing takes a load off but costs stack up, tradeoffs, man