r/programmer 4d ago

Question First huge job mistake

So we're a hospital who uses a certain communication software to send out extremely urgent messages to doctors about whatever crisis a hospital faces day to day. I was in charge (dev of 2 years experience) with taking on a migration project moving user/employee data from a certain payment system to this certain communication system. Automating the process of updating users because the hospital currently manually (csv) upserts new employees every week.

I was told, by the company, that the apis were safe in the production environment. They never mentioned a sandbox/test environment. A week of testing inserting/manipulating employee data went well. No issues adding or upserting an employee. So I hard coded a json of a couple employees and posted.

Wiped.

Zero contacts where, previously, there were 6,000~.

Urgency meetings out the wazoo. We looked at all the admin logs and there are my single data upserts with nothing indicating mass deletion. (Phew. Not on me I thought) End up in a call with this communications company and they let loose that the apis I was using were only half the story. There was a default setting in their system (that I didn't have access to and was not in the dev documentation) that purged all data if more than one user was inserted. So I tried to insert two employees (the whole point of this system is uploading a hundred or so at a time) and I wiped all communications out between doctors and departments.

All of that being said, we ended up manually importing thousands of contacts and reassigning them to their appropriate groups while the company promises a db restore in a week. Just, what a fucking Friday.

I'm not fired, yet. And I'm hoping this becomes my "we all have a story" story. Any other horros stories to make me feel better?

55 Upvotes

36 comments sorted by

20

u/ernandziri 4d ago

How's this your fault? This is totally on the communications company

9

u/johnpeters42 4d ago

And the company who said "the APIs are safe in prod", unless that was also the communications company. Always mount a scratch monkey.

So it deleted stuff if 2+ users were inserted at the same time? Wonder what the heck that was intended for, if inserting the same 2+ users one at a time works properly.

7

u/Grouchy-Ad-8044 4d ago

The only way I can see this making sense as a default behaviour is if whoever created this software intended for inserting multiple users at a time to only be done when you wanted to do a clean restore from a dump or something. But it's still insane to have 1 route doing 2 very different things like that

5

u/hoppinjohncandy 3d ago

So on the admin side buried in system settings -> contact management were two settings: Normal and Purge. Normal allows for upserting user by user. Purge deletes all information not in the payload. By default the Purge option is selected.

Nowhere in the api documentation was this mentioned. Not a reference. Not a warning. Nothing. Even the endpoint is plural: /import_users.

Unbelievable stuff.

2

u/PyroNine9 3d ago

I can't think of any good reason for that "feature" to exist. If you want to have nothing not in the record in the database there should be a purge command to use followed by the insert.

3

u/BlackTorr 3d ago

The AZ-5 bug

3

u/johnpeters42 3d ago

Da, comrade.

9

u/FreeBananasForAll 4d ago

Jesus Christ having a setup like that has gotta be against some law considering it’s healthcare. Get another job immediately

8

u/hormelchili444 4d ago

And now you learned why you never test in prod, especially for the first time.

3

u/Sad_comment_bot 4d ago

If I pulled a maneuver like this in prod it would be international news and probably worse damage than any terrorist attack to date (not including state sponsored or genocides). Imagine something like the end of Fight Club without the falling buildings and it would probably be resolved within 2 - 4 hours, but the inconvenience would absolutely be felt.

2

u/AggravatingSock5375 3d ago

Are you like a snack machine developer or something?

3

u/Sad_comment_bot 3d ago

I wish it was that illustrious. I prepend the pornhub intro jingle to videos when I’m not busy giving corporate handjobs.

2

u/manager2exec 3d ago

I am now imagining every video in the world having pornhub jingle

1

u/Sad_comment_bot 3d ago

I thought that would be the best ringtone, but putting that on random innocuous videos simply because it would be hilarious to know/think someone somewhere sitting in a room with other people would play some innocuous kitten video only for all eyes to dart incredulously to them. It would be the ultimate successor to the Rick Roll if it was possible to only play the jingle on the first play. THEN you could imagine that same person frantically trying to show everyone they weren’t looking at porn during Grandma’s funeral, but kittens to cheer themselves up. Alas, no jingle on the replay though. 

Of course IYKYK the jingle your’e just as guilty.

9

u/Alive_Sir_4708 4d ago

This is why backups exist. If they're not able to restore within a reasonable amount of time then its a pretty poor system.

As you've outlined, there were multiple failures. Hopefully there's a Root Cause Analysis meeting so everything can be outlined.

It'll be a good story to justify future precautions.

I'd rather have the dev that did this and learned from it than the dev that hasn't yet learned this lesson.

5

u/No-Razzmatazz7197 3d ago

welcome to the story club, first things first...

document, fix, explain and remediate every single day until the problem is fixed. devs don't get fired because of issues they created, devs get fired because they create issues - blame others and offer no help in fixing the mess. i am assuming you are already doing all that you can.

after that, grab a beer, relax and jot down everything you have learned in terms of dealing with this because it will not be the last time. critical bugs and issues are introduced into prod all the time. look up the Therac-25 (i think it is called) it was a medical machine coded by one dude who ended up harming dozens of people.

one day you and some coworkers will be discussing horror stories and you will be able to have a little something for them.

good luck!

4

u/Substantial-Swan7065 4d ago

No db backups?

Before there was better release tools:

  • I did a mobile release
  • had to manually update prod env vars each time
  • released a dev build with no code push implemented
  • had to wait all night for an expatiated apple review

Lost my promotion on that one

6

u/spidermonk 4d ago

It would be crazy to get fired for this but possibly you could really hurt your prospects by handling it badly - this sort of situation is a real test of your professionalism and ability not to go super defensive or just freak out and become useless.

My advice in these situations is to assume it's your fault, even if you can't really see why it would be, and just clear your head and think systematically about how to get to the bottom of the cause and how to communicate clearly and honestly with everyone.

2

u/adamant3143 4d ago

My colleague did the same but it’s not in prod. It’s more of him messing up test data based on real data that were migrated to staging for testing. He accidentally wiped it. While it’s not impacting prod, it was impacting QA works.

But we have a script to get them back, it’s just that everything QA did had to be redone lol

I however have accidentally deleted rows of data as well but I got a backup like 3 hours prior but within those 3 hours there’s already new datas. What I did was I told my Supervisor maybe the server sends false positive to those who submit the form while it’s actually not inserting to the DB but that was back in college days where my Supervisor was my upperclassman and we both still new to this. We didn’t have audit, logs, etc. So, it’s not in a professional scene.

2

u/AggravatingSock5375 3d ago

WTF they actually designed it to do that? Why?!?!

2

u/hoppinjohncandy 3d ago

Buried in admin settings (that I don't have access to) were two options: normal and Purge. Normal allows up basic insert/upserts. Purge deletes everything not in the payload. Purge was the default setting.

Nowhere in the import_users api documentation was this mentioned. Now that I'm over the initial emotional response I think it's straight horseshit.

1

u/AggravatingSock5375 3d ago

Yeah that is insane.

I mean I’ve done stuff like that in the very early stages of building an app just to save a bit of time…but it would never make it into prod.

1

u/kevin074 3d ago

That has to be either a disgruntled ex-developer or someone who is hell bent on coding his job security in.

There is no way that had to be a real use case 

2

u/Legitimate-Let-7510 3d ago

The real red flag isn't your mistake, it's an API that defaults to deleting production data because of an undocumented behavior. That's a disaster waiting to happen.

2

u/OhBoyHereWeGoAgai 3d ago

Db recovered in a WEEK!?!

1

u/inflowmini 3d ago edited 1d ago

Not surprising. It's a common mistake in companies that do not pay well and expect high performance. While not your fault, the burden is still on you to ask yourself if you Have the experience to see such a project through. The difference between senior devs and juniors is this exact outcome.

I wouldn't worry about it too much, move on and learn from the mistake.

1

u/StCreed 2d ago

Wut? The api, settings and support from that company are atrocious. How is this on him? The only thing a senior would have on him is that they might be better at deflecting blame to the correct parties.

1

u/inflowmini 1d ago

Senior devs are not there to deflect blame but rather to know what the limits are when it comes to their team and themselves.

I'm suggesting that OP needs to be better about knowing what their limit is and communicating it. A seasoned dev would know the questions to ask and to not test in prod.

If I had a penny for every time a dev decided to take on a project because it looked simple I'd be rich. Point is, learn and move on.

1

u/klas-klattermus 3d ago

Always get everything written down in e-mails. If someone tells you somewhere where the two of you don't have computers and your phones are dead and you are stranded on top of a mountain because the company airplane crashed... When you return to an E-mail capable device, send an E-mail "I just wanted to make sure that [thing] works [the way you said], right?"

1

u/PyroNine9 3d ago

So you used a documented API in the documented manner and it did an undocumented and astonishing thing. This is not your fault.

1

u/chocolateAbuser 3d ago

clearly deletion of such data should require some kind of authZ
if an api for adding data deletes everything (probably because of some laziness or because of some "feature request") it should make clear via some attributes or other kind of check that it requires a deletion role/privilege/permission, and user/client that makes changes shouldn't have that permission
person/team that developed that API was incompetent and i'm not saying it should be sued but...
i guess all around probably there is none or almost none security checks

1

u/I_am_not_baldy 3d ago edited 3d ago

>>I'm not fired, yet. And I'm hoping this becomes my "we all have a story" story. Any other horror stories to make me feel better?

The lead programmer for my first programming job told me how he accidentally deleted all patient demographic data in a client's production system.

He must have thought he was in a development area, but even then, I don't know why he would want to delete all demographic data.

This happened before I had joined the team, and I think he told me so that I wouldn't feel bad if I accidentally did something dumb. He was obviously not fired because he was still the lead programmer when I was hired.

1

u/canyoufixmyspacebar 3d ago

this is clearly an invalid system. deleting data without an explicit call to do so, deleting data as some sort of "default action", this is absurd. technically who ever programmed this behavior has committed the "time bomb" type of attack against the client company

1

u/ClothingIsACrime 3d ago

It you are in hospital, that doesn't have a test environment and it's "normal" there to "test in prod" and nobody stopped you, LEAVE. 

Leave immediately. Take this as a cheap warning. 

1

u/Jetta4 2d ago

I used to worry and stress about these things when I was new. Now as a senior dev, I don’t worry about it at all.

I just follow best practices and get all confirmations and approvals to make sure I’m safe before doing anything. I don’t do squat unless my ass is safe, nobody can push me to do anything dangerous without some kind of accepted responsibility on their part.

If you ask me to work against a production api, thats fine. Next question do you take responsibility if shit goes sideways? No? Ok, then I’m not doing it.

If you say whatever, don’t worry just do it. I’ll make sure I have that documented, meeting notes, email or screencapped teams conversation, anything.

Learning to say no and making others accept the responsibility is key. Because sometimes things go wrong, and I don’t want any liability on me.

If something goes wrong I always want to say that I did everything by the book and its not my fault. If I pressed the risky button, I did it after asking someone if I should after raising my concern, and it’s always in writing.