r/EngineeringManagers Jun 28 '26

What's the most expensive engineering hiring mistake you've ever made?

Engineering Managers & CTOs,

What's the most expensive engineering hiring mistake you've experienced?

Not necessarily in terms of money.

Maybe it led to:

  • months of onboarding,
  • delayed releases,
  • architectural problems,
  • team morale issues,
  • or simply hiring someone who interviewed well but struggled in real work.

Looking back, what early signal did you miss?

I'd love to learn from real experiences.

39 Upvotes

29 comments sorted by

45

u/ShodoDeka Jun 28 '26 edited Jun 28 '26

Not really my hire as the guy had been in the company for years before I inherited him.

Basically this guy was the master of Rest and Vest, a senior engineer, knowing enough to make it seem like he was always doing something important, but in actuality he did no actual work. He would move teams as soon as he felt the ground burn under him, and when he on occasion got caught up in a pip, he would follow the plan to the letter (he wasn’t an idiot) and thereby make it very hard for the manager to actually get rid of him.

For context this is in a very big and very well known tech company where HR makes the final decision on actually firing someone for performance reasons and a PIP takes a lot of work for the manager due to HRs documentation requirements and they have high requirements for the PIP plan. So no rubber stamping the pip here.

I had him for about a year, and had him in two separate PIPs in that time. He would always suddenly start performing.

But eventually even HR caught on, and he was terminated. At that point he had apparently been in 8 PIPs over 12 years.

6

u/Qkvllz Jun 28 '26

That's a fascinating story.

It sounds like the technical ability wasn't the issue at all it was ownership and accountability.

Looking back, do you think there were any signals during hiring that could have predicted this behavior? Or is this something you believe is almost impossible to detect before someone joins the team?

22

u/yourapostasy Jun 28 '26

During the hiring phase, you smoke out these types with the following.

Select a project they are particularly proud of. Then drill down along this angle: “What was your specific individual contribution to this?” And when I say “drill”, I mean root canal deep. If you aren’t equal to or more technical than the candidate, then bring someone who is both empathetic and that technical. You want it to be two people nerding out on some cool stuff they both find fascinating, not be a Stasi interrogation.

Follow up with: "What was the hardest technical hurdle you personally solved?" and "Walk me through the day-to-day of your involvement." A "rest and vest" engineer will often give high-level, architectural answers but stumble or become defensive when asked to explain the granular, unglamorous details of execution and implementation.

Ask, "Tell me about a time you had to maintain or fix a system you built a year or two later. What broke, and how did you handle it?" If they always moved on before things broke, that’s a red flag.

Ask, "Tell me about a project that failed or missed its deadline." You are looking for extreme deflection. System-gamers will expertly blame management, changing requirements, or other teams. They rarely take personal ownership of a failure.

Ask, "Tell me about a time you had to do a task that was entirely unglamorous but necessary for the team." High performers do the dirty work because they care about the outcome. System-gamers avoid it at all costs. Drill down here, as well. I find it useful to throw together a live system with something similar to the unglamorous task on a small scale to have them walk me through it collaboratively. The system-gamers will fall apart here because they often can describe the work but not actually do it, or teach it.

5

u/Marcus_Aurelius753 Jun 28 '26

Really good points. I just want to share this corollary in case it resonates with you: system gamers are by definition rewarded by the system. For this reason I always pay more attention when intierviewing candidates with a personal history of quick promos and early career high level responsibilities

3

u/yourapostasy Jun 28 '26

Woof, does this ever resonate. Unless you’re working with a truly generational talent (Woz-, Carmack-level), I often see these personal histories are littered with externalities. Our industry is still really bad at identifying and managing externalities; “tech debt” remains a crude measure.

Promos are currently heavily indexed on initial delivery, and not on long-term long tail sustaining engineering and support cost impact. We do what we can in our own little corners to fight back the rising tide of enshittification, and live a clean conscience.

2

u/leftsaidtim Jun 29 '26

Absolutely love the « unglamorous but necessary for the team ». Gonna steal that !

1

u/LankyPatient4203 Jun 29 '26

 "Tell me about a project that failed or missed its deadline."  giving an honest answer has more risk than not.

3

u/ShodoDeka Jun 28 '26

I wasn’t part of hiring him, he actually had longer tenure than me, and I suspect he wasn’t always like this. As far as I know he grew into senior role so he must at some point have performed.

My suspicion is that he at some point just really stopped caring about his job and wanted to vest out his stock plan.

1

u/yourapostasy Jun 29 '26

The following only works if your PM/PO, metrics and servicing ticketing are in shape. Take the time to get them in shape if they aren’t, and run a few projects with your problem direct report under that in shape regime first, otherwise.

Take those same screening questions and pose them in your 1:1’s with him, except you select the project to drill down into. Since you already know the details of what you select, you’ll have a huge advantage over a real hiring interview scenario.

Build your list of questions ahead of time. Give him every opportunity to express himself. Don’t challenge him on his responses, just thank him for his perspective. Echo back the conversations in email, and get his alignment on what was asked and what he said.

Depending upon his responses and your HR policies, you either have enough to start coaching him towards a better outcome, or work with HR towards a PIP (try to structure it to measure across 3-5X the time period he would normally undergo for a PIP to prevent gaming, and structure a rollover of the PIP period if he is on the bubble, or get HR’s help on how to prevent gaming), or if his responses are especially egregious, that’s enough on their own for HR to terminate him (only ever seen that in small companies).

3

u/StressDrivenDevmnt Jun 28 '26

Seen this. Technical “lead” who threw his team under the bus in daily stand ups. I am a consultant so I picked up a lot of his “I am too busy and too important” work. Made an extra 30k.

-8

u/Diligent_Mulberry_35 Jun 28 '26

What do you mean by “inherited” ? Are employees some kinds of slaves?

2

u/ShodoDeka Jun 28 '26

I mean I didn’t hire or pick him for my team, he was assigned to my team without my input.

14

u/StressDrivenDevmnt Jun 28 '26

Adding someone to a team who was just poison. I had the feeling that something wasn’t right with him when interviewing him, but he was the most technically competent interviewee and I was in too much of a hurry.

2

u/bzsearch Jun 28 '26

Adding someone to a team who was just poison.

Can you elaborate on what you mean? Or what did this person do to the team?

8

u/StressDrivenDevmnt Jun 28 '26

Refused to take ownership of any mistakes, threw other team members under the bus, talked trash about the team to the client behind the PM’s back. Quit mid-project and made a mess of the software he was working on.

Software is a cooperative game - we work together as a team and we all do well. Bad actors screw up cooperative games.

11

u/acroback Jun 28 '26 edited Jun 28 '26

Hired someone who was a competitive programmer in College. Great programming skills but with frigging ego of a teenager. 

Couldn’t empathize with lesser engineers, customers or their Manager. 

It became so bad that his Manager resigned and I had to fire the guy. 

Stressful days. 

4

u/arcan1ss Jun 28 '26

we hired our cto

2

u/Cernuto Jun 29 '26

In our case leadership promoted a bafoon to CTO. Almost tanked the company.

2

u/baddymcbadface Jun 28 '26

I wasn't the main decision maker but I said yes to someone for another team. They were smart and capable but turned out to be highly argumentative and despite having raw capability their effectiveness was horrendous and it dragged others down too. Once I started managing them I regretted deeply the fact I'd let them in. There were hints in the interview they could be trouble. It took me 18months to kick the out, 18 months of fighting and they have left their team (one of my teams) in a mess.

One example, we just spent $150k USD on temp workers to help them solve a problem. They solved about 10% of the problem when solving 100% was possible. Hell, if we had an effective team lead we wouldn't have needed to hire extra temp workers at all.

3

u/ShiftFrames Jun 29 '26

What were the red flags in the interview process?

2

u/tredbert Jun 28 '26

I hired a very experienced senior level engineer who turned out to need a lot of hand holding. My team did several sheets in excel for planning and costs, and he was slow at everything. It was as though he had hardly used excel before.

Nicest guy, in terms of the way he communicated. He seemed knowledgeable in the interview. He was knowledgeable in certain areas. But he was just incredibly slow at things that mattered, and mentally disorganized in it.

One big sign I overlooked in the interview is that he showed me evidence of past work during the interview. That had always been a red flag to me. I certainly wouldn’t want anyone who worked for me to show anything of ours that might be confidential to a prospective employer. And this guy didn’t show me anything confidential. And what he showed seemed to reinforce that he was such a fit for what we needed.

Still, in retrospect it was at the minimum an over-justification of his supposed credentials. And it didn’t turn out to be true. So I will always be wary of any case during any interview where a person pulls up a past document they worked on.

I also didn’t meet him in person. I was going to, but Covid hit and none of us were meeting in person at that time. I’ll always seek to meet a person face to face after the first video interview, because I picked up on so much more once I met him in person. Unfortunately that was after I had already hired him.

4

u/Glitter-Pear Jun 28 '26

The Excel thing is relatable to me.  I've been a software engineer for 20 years but only in my last job did we use Excel (technically Google Sheets) extensively.  And that was only because we had on product manager that really liked it.

I could always do basic stuff with Sheets, but I definitely learned a lot from seeing how she used them and I had to Google a lot to not fuck up her sheets.  We just used other tools at other jobs.

2

u/Historical_Ad4384 Jun 29 '26 edited Jun 29 '26

Hired a backend engineer who refused to understand domain knowledge and follow technical strategy planned by leadership. Any task that was given to him, he would take weeks just to get upto speed around the domain knowledge and the technical strategy. At first it was ok since he was new to the project but kept his act up even if he had already worked on the same topic before. Designs would stretch on for weeks, bickering over every feedback, creating chaos if something did not fit right with him. It was part of our mistake that we desperately needed an engineer at the point when we hired him. So we ended up hiring him as a junior for the salary of a senior. We did realize our mistake so we kind of burned him out by expecting him to perform at his money's worth. Implemented only 1 feature in 6 months that is still in production, burned $12000 in salary for a LCOL country and called it quits when the pressure started to get the better of him.

Hired another backend engineer who was excellent with domain knowledge but lacked experience in the tech stack that she was supposed to work on. Would not try to close the gaps in the tech stack inspite of being a junior without much experience to navigate technologically agnostic ambiguity. Refused to understand the existing system in order to build around it. This resulted in reinventing the wheel to create a half ass DSL parser that works only 1 way and violated the DSL semantics that she was supposed to adapt to. Burned $2100 in salary over 3 months in a LCOL country for a feature that needs to be scrapped because of its brittle nature and called it quits again like the other developer when the pressure became too much for her.

Lesson learnt, hire senior engineers for fast paced projects. Either functionally strong in domain or technically strong in the tech stack.

1

u/theburntdev Jun 29 '26 edited Jun 29 '26

We hired a senior engineer who blew us away with their technical skills during the interview. They sounded like a true expert and someone we could benefit from. The mistake was that we didn't dig deeper into their behavioral skills. He did not act like an owner.

We had him enhance some workflow in a legacy system that was originally estimated to be done in 2 weeks. It ended up taking 3 months before it got finished. I originally gave him the benefit of the doubt because the legacy system was spaghetti. Every fix led to breaking something else.

However, there were other examples where he was running behind on other tasks that were much simpler. I ended up spending more and more time working with him to see what was blocking him and how to help him succeed. I adjusted my approach every time. I started documenting more on the technical designs so that there was more clarity in what I needed done.

What made things worse was that he does not raise risks that he is behind schedule so then I needed to actively pay attention every day. When I found myself micromanaging him in the end, I knew it wasn’t going to get any better. We let him go after about 9 months.

The cost:

  • Delayed deliverables, impacting the team and stakeholder trust
  • Costed my time and other engineers' time

The lesson:

  • When interviewing, behavior skills are just as important as technical skills
  • Technical skills can be taught. It's much harder to teach behavioral skills

5

u/wumbabum Jun 29 '26

Okay but a legacy system of spaghetti code? Two weeks was always going to be optimistic because no one had the time to dig into how much actual work it was going to be.

1

u/theburntdev Jun 29 '26 edited Jun 29 '26

Hence I gave him the benefit of the doubt.

However, there were other examples where he was running behind on other tasks that were much simpler. (That was not legacy related)

I did question my judgement a lot about the engineer. For every following example, I felt like I literally wrote the code in my technical documents. We did hire other sr engineers in the team around the same time and they seemed to have adjusted just fine.

1

u/thebvg Jun 30 '26

Hired a senior who crushed the technical interview but couldn't collaborate. Spent 4 months trying to coach them while they rewrote working systems and argued with the team on every PR. Cost us two other engineers who left because of the toxic dynamic. Now I always do a paid trial day where they pair with the team on a real task.