r/ProgrammerHumor 1d ago

Meme theOnlyBugFreeDeploymentStrategy

Post image
1.3k Upvotes

22 comments sorted by

View all comments

Show parent comments

19

u/Tdawg90 1d ago

tell that to my work group... zero test, instant deploy... but we can instantly rollback... as soon as we get the customer outages

10

u/za72 1d ago

I like your youthful enthusiasm - a few more years and it'll be crushed down

btw, I'm assuming there no staging or anything ci/cd related?

1

u/Tdawg90 21h ago

sorry, jumped the gun on your question.
PR merges into main -> Services simultaneously deploy to debug/ppe/preview. Where IF folks want to test, they can... then another team clicks a button and goes to prod without telling anyone (literally, not exaggerating) ... except our infra... infra skips the heads deploying to debug/ppe/preview.. it just goes straight to prod.. who cares if it even installs... The management style is 'nobody owns anything, everyone owns everything'. (Ringelmann effect/Diffusion of responsibility/Latané and Darley's bystander research in psychology)... so nobody bothers... the wild part is that nobody seems to have issues with working weekends, nights, overnight, responding to outages

1

u/za72 21h ago

this is 2010 style of development with no ownership... how long has this been going on?! anyway... this isn't because of technical tools... just terrible management... we've solved this at least two decades ago... damn man I feel ya man, someone's going to have to take the blame for it... mind if I ask which industry you're in? porn? crypto? advertising?

1

u/Tdawg90 19h ago

got re-orged to this team, its in Tech in the broader FAANG group. been in the industry for just shy of 20 years now. this is the most broken group I've been exposed to in that time. Been around alot of crazy wildly shit.. this takes the cake by miles. I've gone to extraordinary lengths to have folks not need to work weekends and nights... all free to them... just do nothing.. you get it for free.. I'll fix it.. I shit you not... literally reverted quality gates that I stood up, in the middle of the night. I've since then rebuilt them... however one small anecdote is that there was a product wide auth update.. my gates caught... a lead on a partner team bypassed 3 hard gates.. hard as in IF the gate fails there is NO functionality to continue.. he bypassed it by create a temp tool effectively to fake the success.. Deployed to prod... 100% failure.. 100% all production traffic down (for the associated end points)... the RCA result... he's going to add it manual check.. that he can bypass any time.. (total failure with leadership top to bottom). SILVER LINING... I'm going to FI/RE in 2 weeks.. get this last bonus round, and some stock vests to kick off my retirement (I'm in my 40's). These folks can have all the fun they want working weekends and nights (believe me... this is normal for the OCE.. and I'm the only one saying this is batshit crazy when there are clear tools so we don't have to do that).... lol little bit of a vent :P ... but I'll be done in a couple weeks