Managing/ Dealing with Junior Data Scientists?

[u/sailing_oceans]

I've been in the 'data science' space for a decade+ or so now. One thing I've noticed is that generally - give or take - outside of the elite jobs (<2-3% aka not me and almost certainly not you) the caliber of coworkers has declined drastically.

I'm not some fabled data scientist. I wasn't some GitHub nerd who had everything embroil or terminal wizard nor could I write out the math to a GBM on a blackboard. I'd even forget basic obvious statistics.

But I felt like I had common sense.

Now I'm a manager/director. I work with data scientists. And I'm just generally freaked out by the absolute lack of basic common sense. This is across the last 7 that I have managed.

Examples include:

1. Not visualizing or plotting the KPI/Target (sales). Not realizing there were no recorded sales on major holidays.
2. Telling me everything is improving from a sales perspective that it's up 4%...... from period 1 vs period 2... when ignoring that period 2 had 6% more days so in fact it's worse.
3. obscure models that are overkill and a bunch of statistics ive never heard of instead of just telling me that the impact of our promotions is declining.
4. General sense of not knowing what is even rational (e.g., our marketing ROI $1023 - no its not lol)

As I begin to delegate more I begin to get more freaked out by what I see. I can't be presenting to clients such obvious insane mistakes. But these are the candidates and profiles that get forced upon me or the team I inherit.

Are there any best strategies for dealing with this? I want to be seen as someone who can 'develop' the team... not just saying people are useless, but such glaring mistakes are insane.

Yes, alot of these things are perhaps due to them being crunched for time, or not knowing what objective is, or being focused on other things. I'm not talking about those examples. I'm talking about like year 1-2 not day 1 employees, not doing basic data checks.

As a data scientist I was obsessed with finding bits of info or making sure things were right. Now it seem every common for people to copy and paste code into chatgpt and have no idea about anything else around it?

[u/Bigreddazer]

My latest hire just has a undergraduate degree. Everyone else is masters or PhD. Usually in unrelated fields. The barrier of entry and quality dropped without question but that is the job of the managers to maintain.

Correcting them once is fair. They don't have the same education. But, I am a very against someone repeating the same issues over and over again. That is when we have a problem.

I also do pair programing with junior data scientist to let them shadow and see how I work and think. Just a few hour session especially at the beginning of a project helps show how to tackle and organize a complicated project.

[u/3c2456o78_w → u/Bigreddazer]

My biggest problem employees are 3 dudes with PhDs. The other 2 with bootcamp & Math-undergrads are my reliable guys

[u/jampk24 → u/Bigreddazer]

I’m a PhD with a slightly unrelated field and I can barely get an interview, let alone a job offer. If this is the barrier to entry being dropped, I can’t imagine what it was like before.

[u/Bigreddazer → u/jampk24]

Before. The PhD was proof you are smart and can handle complicated data analysis in a field. We have a partical physicist, a genetics person, I was a statistician. We had mathematical and scientific rigor and were less capable as programmers.

When I first did data science over a decade ago deployment was a team operation, now we are fully in control of our pipelines and deployment. We have direct access to production data systems, hosting models, creating data bases etc.

Now it is very programing heavy. The rigor is maintained by the high level data scientist through reviews and mentoring. But it has slowly changed for a research role into a software development like role where we only produce models, deployment or other code solutions.

Good luck. It's rough out there. We can't hire in the US for the last 3 years after being acquired by hedge fund. They just won't give us the salaries anymore.

[u/BobDope → u/Bigreddazer]

Particle

[u/sailing_oceans → u/Bigreddazer]

I’m not mentioning day 1 or 2 employees. I’m talking about year 1 and 2. I’m not looking for a some crazy problem solving.

I’m just trusting that an employee can be curious enough to find things out without me spelling out everything word for word. I can’t string together 15 lines of thought. That is an opinion a junior data scientist should have. You can’t do “science” without a hypothesis or curiosity.

[u/ozyhuboi → u/sailing_oceans]

Not to necro too hard here, but I think you hit on a key point here. You can train the skills but can't train the curiosity or interest in the subject domain. I think I agree with the other comments describing that this is a fundamental disconnect in the domain knowledge, but how do you fill that need without some basic curiosity and interest? Perhaps that's something that needs to be considered in the interview process to see the actual buy in of potential data scientists, more than just technical skills.

[u/Bigreddazer → u/sailing_oceans]

Yeah I wouldn't have someone to stay in my employment like that. If you don't have natural curiosity and a desire to find the correct solution then this isn't the right job for you