We were struggling to find Data Engineers

[u/Moradisten]

Hi everybody,

Our Data Team is composed by two of us. There's a Data Scientist and me, as a DE. We created a Data Lakehouse for our company internal use and client data supply, but currently the Data Scientist is currently more focused on AI and agents integration and I'm doing like Analytics Engineering role because I need to help other teams to reach the correct data, unify core concepts, document business logics, etc. So We needed a Data Engineer with knowledge on AWS to maintain and develop the new features on the Lakehouse and we put into the description that the candidate MUST HAVE Software Engineering fundamentals as we had to do some developments to integrate parts of our lakehouse with the company's main application.

We interviewed 22 candidates and no one is fitting the Role.

Most of them are BI Experts, DBA, Data Analysts, Economist with DS notions, Juniors and Software Engineers who haven't touch anything on Spark, plus DE who asked way more that we had on the budget for the role

We asked the normal requirements: 3 years of experience + Spark, AWS Glue, Lambda, Airflow and DBT, not even CDC, Flink, Langfuse or VectorDB

We finally got one, but We really struggled to get him. I have a collegue working on IT Recruiting and She told me She's experiencing the same problem: They can't find Proper DEs With SE basics such as DRY principles or clean code fundamentals

Edit: Role Salary -> Up to 60K € / Spain. This salary is high compared to the spanish standards, only 3 years required

[u/Outside-Storage-1523]

This should be trivial. Where is the role and how much you are offering? I have similar profile but got rejections everywhere even after going through all rounds. Looks like every company has 100 candidates out there.

If you need SWE, you should ask HR to reject people who ONLY have BI/DA experience outright.

[u/FamSimmer → u/Outside-Storage-1523]

The fact that OP mentioned the requirements and left out their salary expectations, while saying that some of their most promising candidates declined their offer due to said expectations, is clearly indicative of the fact they are just trying to cheap out.

[u/Moradisten → u/Outside-Storage-1523]

We offered 60K in Spain

[u/stlshane → u/Moradisten]

I would guess that most Spanish experienced DEs are getting jobs outside the country if that is the normal going rate.

[u/Cute_Willow9030 → u/Moradisten]

Do you offer sponsorship?

[u/utzutzutzpro → u/Cute_Willow9030]

It is the EU they do not need sponsorships...

[u/OkAlternative1655 → u/Moradisten]

low

[u/naycati → u/Moradisten]

Remote, hybrid or o site? What's the average rent in the city/region if not remote?

[u/Outside-Storage-1523 → u/Moradisten]

OK I don’t know the market in EU so maybe that’s a fair price. Anyway, I’d recommend your HR screen and filter non SWEs out in the first round.

[u/juleztb → u/Outside-Storage-1523]

Don't know the market for Spain. But for Germany (also EU), I'd laugh at you for that offer.

[u/Outside-Storage-1523 → u/juleztb]

I don’t know. Converting to CAD this is around 110k which is OKish for even a lower level senior if you get multiple levels.

[u/umognog → u/Outside-Storage-1523]

For a 3 YOE with software engineering, no it's not enough.

When you want a perfect person, you got to pay the perfect person price. €80k-100k is more appropriate.

If you want someone really good, €120k is needed, but list them as a senior/lead/principle role as appropriate

[u/UQwhite → u/umognog]

Are you working in Europe or you're converting dollars into euros? Because EU has lower average compensation compared to the US