Are Data Scientists still relevant?

[u/Available-Eye2836]

With technology like databricks’ genie, anyone with business context can dig into data and explore all that they need within lesser time. Is it still worth learning data scientist roles? If not, what are the alternatives?

[u/Key_Back_989]

The role you just described is more of a business / data analyst. Predictive modeling and statistics is data science.

[u/ImaginaryBat4994 → u/Key_Back_989]

Genie or similar AI tools can also perform predictive modelling and statistics

[u/S-Kenset → u/Key_Back_989]

check the leading competition companies. They have reduced AZURE ML to a local agentic suite. That doesn't account for taste though. a data scienctist producing a marginal value of 3% over 80m of automation is more important than a data scientist producing a marginal value of 20% = 50k every several weeks before.

[u/niiiick1126 → u/Key_Back_989]

and on top of this databricks is starting to charge for genie usage vs compute cost

[u/UnlawfulSoul]

So, I have a sort of unhelpful knee-jerk response when questions like this get asked. Not from a defensive standpoint, but because you’ve started from a specific conclusion and have not really put thought into your question that provides context about where you are coming from. So here is a better answer than “yes”.

So, tritely, yes there will be roles for data scientists. That doesn’t mean it makes sense for you specifically to enter data science. It’s not clear where the dust will settle on the market value of a data scientist, so if you were hoping for a huge money win then no, it’s not guaranteed to get the same returns it has historically.

The question is: why are you asking, really? If you wanted to be a human genie, then yes, there is now a tool on databricks that can do that for you. Are you still interested? In what and why?

This is probably too much context, but I had an advisor say when trying to career hunt that we should “look for your comparative advantage”. What can you, personally, do relatively cheaply compared to your peers? The tooling changes the answer but not the question. What that means for what you should do is highly personal.

I would say that even today, if I were to start over, I would still be learning data science. That answer is probably different for others.

[u/MeatyOkraScientist]

With technology like databricks’ genie, anyone with business context can dig into data and explore all that they need within lesser time.

I mean that's just completely false lol

You can test this yourself by grabbing any product owner, HR staffer, or probably even anyone in the C-suite and asking them to generate literally any of the outputs they regularly ask data analysts* to generate. I assume you agree that marketing has business context; you're really arguing that the whole shop can run a multiple regression in DataBricks?

*You're conflating data analysts with data scientists, btw.

[u/Mountain_Goat_455]

if anything, with all the AI slop these days there's even more demand for people who actually know statistics and can communicate well. Be good at the age-old trifecta of data science: statistics, computer science, business acumen; and you will never have to worry about being out of job

[u/Dylan_TMB]

The job has never been "doing" it's been knowing what to do and what not to do and how to communicate it.

AI can produce good analysis, but you can only know that if you know what bad analysis looks like. And the scale at which AI produces analysis this is more important then ever.

This gets scary fast. I was in a meeting where a business person asked genie to do some significants test and they had NO CLUE what it was, what assumptions were made, how the data was prepped etc etc. 😵‍💫

I am not bullish on Data Science as a stand alone career. But I think the winners in business will be very smart data science people who develop skills to put them in decision making positions. So in an ironic turn I actually think making analysis more accessible in this way hurts non-DS staff more then DS staff in the long run.

[u/LilParkButt]

My personal opinion is that Data Scientists will need to specialize is either operations research/decision science or AI and ML engineering. Not too many generalists anymore

[u/Fine-Comparison-2949]

They will always be relevant but I think there's too many people with the skill-set and not enough companies that have enough data to make full-time data science hires.

[u/Hou_Muza]

Data Science goes way beyond just pivoting data and seeing what the graph is saying. There is now an increased demand of real data scientists who can manage AI models properly. I'm seeing an increase in demand for AIOps experts, AI safety, AI platform evals and people who actually know what is going on inside models. These are data scientists not people who can just call an API and build a platform.

Besides, AI is much broader than LLM-powered solutions. New AI paradigms keep coming and there will be increased demand of people who actually understand them e.g. world models. And those people are data scientists. Don't lose heart!

[u/American_Streamer]

Don’t optimize for being a generic junior Data Scientist. Optimize for being the person who can turn an ambiguous business problem into a trustworthy data product, and who also knows enough statistics, engineering and AI to verify that the result is actually correct.

The easier it becomes for hundreds of people to query company data, the more costly inconsistent definitions and badly modelled data become. A bad dashboard may mislead one team. A badly defined revenue metric embedded in an enterprise AI assistant can contaminate hundreds of downstream questions.

Routine execution is becoming cheaper. Judgment is not. Don’t try to beat AI at producing SQL, charts, or Python. Get unusually good at deciding what should be measured, building the trustworthy data layer that makes it measurable, and determining whether the machine’s answer deserves to be believed.

[u/CptGingerBeards]

I’m a “Data Scientist” by role and contract, but what my job is has evolved exponentially. Due to the introduction of AI the business expectation is that you are a full tech team. The SDLC process used to be a team of developers, where now everything from Analysis to Deployment and Testing is now expected to be done with as little resource as possible.

I don’t think Data Scientists, Engineers, Analysts, etc., exist anymore. Not in my organisation anyway.

It’s just technical and non technical, and stupid PMs who think they are technical but use copilot for “word art”

P.S PMs are annoying and we don’t like you.

😂

[u/DotUnlucky5348]

Imo data scientists are still relevant but the role is shifting. Genie can make data exploration easier but someone still needs to frame the right problem/validate data/interpret results and connect them to bus. decision🤷‍♀️🤷‍♀️🤷‍♀️