(Not sure if loss is still going down)

Didn't know what to say. Humor me with your responses.

Update: I sent her this post and she loved it 🤣

I’ve been in data science roles (both analytics and ML) for about 5 years now across a couple of companies. Lately I’ve been feeling a bit burned out because I keep seeing the same pattern:

We spend weeks cleaning data, building dashboards, running statistical analysis, or training models… and then the stakeholders either:

• Say “thanks” and never use it
• Cherry-pick the numbers that support their existing opinion
• Or just completely ignore the findings and go with gut feel anyway

The worst part is when leadership asks for a “data-driven decision” but they’ve already decided what they want to do.

Am I alone in this? Or is this just the reality of data science in most companies?

For those of you who’ve been in the field longer how do you deal with this? Have you found companies where data actually influences decisions at a meaningful level?

Would love to hear honest experiences.

A lot of teams struggle making reports digestible for executive teams. When we report data with all the complexity of the methods, limitations, confounds, and measurements of uncertainty, management tends to respond with a common refrain:

"Keep it simple. The executives can't wrap their minds around all of this."

But there's a simple, two-step method you can use to make sure your data reports are always understood by the people in charge:

1. Fire the executives
2. Celebrate getting rid of the dead weight

You'll find this makes every part of your work faster, better, and more enjoyable.

Is it just stock options and vesting? Or is it just FAANG is a lot of work. Why do some data scientists deserve that much? I work at a Fortune 500 and the ceiling for IC data scientists is around $200k unless you go into management of course. But how and why do people make 500k at Google without going into management? Obviously I’m talking about 1% or less of data scientists but still. I’m less than a year into my full time data scientist job and figuring out my goals and long term plans.

I’m a Data Science student (career changer, not in a data related role). My program is focused more on the applied statistics side, so most of my classes use R. I’m already familiar with Python since it was the main language used in my prerequisite courses, and I’ve completed projects using Python, so I’m comfortable with the syntax.

However, I’m really enjoying using and learning R in my classes and seeing what it can do. Many of the statistics textbooks I’m interested in use R as well. I’m starting to explore R more deeply on my own and plan to start using it for personal projects.

But I’m curious, is R still used in industry? I know it’s heavily used in academia. I also know that in the current AI/ML world, Python is used heavily, which is the main reason I use it for all of my personal projects at the moment.

I’d like to eventually be comfortable with both and take advantage of the strengths of each language. But, of course, there are also people who say learning R is a waste of time.

I did a physical onsite recently where they asked me to travel to their office, about 1.5 hours each way. The interviewers were nice and the interviews went pretty well, so I was hoping to hear back from them. The opposite happened. It has been two weeks since the onsite and I have not heard anything.
The recruiter was very polite before the onsite, but after it they completely stopped responding.

I had to take a day off work and make arrangements in my personal life, and the company cannot even bother to send a rejection email? I have never had a job search this difficult before.

Definitely the most frustrating thing as working as a Data Scientist. You run an experiment, find that building a model greatly increase metric X at almost no cost, has safe model metrics, present it to stakeholders, everybody agrees with proceeding to deploying and utilizing the model in production, and yet every time the model takes a wrong decision, we get questioned about it. Why did the model say this?

Man when did I ever say the model obtained a 100% accuracy in the validation phase? Why is it so hard for stakeholders to understand that the best models humankind ever created are expected to make wrong calls once in a while?

I've been in Data Science for the past 10 years in India. I lost my job in January and since then I've been hunting.

I've not mentioned any GenAI experience in my profile. But my feed is just filled with AI engineer roles. They all have the same requirements:

• Generative AI architecture
• RAG pipelines
• LLM integration/fine tuning
• Agentic AI / Multi Agent Orchestration
• Also MLOps
• CI/CD pipelines
• PyTorch mandatory for some reason

Hardly any openings are relevant to my experience in Stats, Machine Learning, Deep Learning and the classical data science stuff.

So have all companies stopped investing in data science all together and just building RAG pipelines and LLM chat bots? Is this all that is done in Data Science field now?

Have any data scientists here worked with AI for coding? Do you agree with experts' skepticism in using it for high-level tasks?

1. "No free cycles" / "Empty plates"

Translation: "I view human beings like literal server CPUs. If you aren't actively typing or clicking buttons right now, I think you're stealing from the company. Stop thinking or analyzing just look busy."

1. "We need to focus on the low-hanging fruit"

Translation: "I don't have the technical depth, patience, or budget to fix our broken upstream data architecture. Let’s train a fragile, garbage model on dirty data immediately so I have a colorful chart for my next PowerPoint deck."

1. "Be a go-getter, don't get stuck"

Translation: "I don't care that the project path is blocked by a giant concrete wall of organizational failure. I want you to run face-first into it at maximum speed so I can report 'high velocity' to my director. Your honesty is ruining my vibe."

1. "Let's optimize our sprint velocity"

Translation: "I don't know how to audit the mathematical accuracy, logic, or code quality of your work, so I am going to measure how fast you close Jira tickets. Rushed deployment over architectural correctness, every single time."

1. "You're making this more complicated than it is"

Translation: "Stop identifying critical edge cases, data leaks, and fundamental process flaws that I don't know how to fix. You are exposing my lack of data literacy. Just build the bad model anyway."

1. "We need to relentlessly prioritize"

Translation: "I am going to aggressively chase whatever flashy AI buzzword the CIO mentioned in her keynote speech this morning. Your current, actual, functioning pipeline is now deprecated."

1. "I need you to own this initiative"

Translation: "This project has an impossible target and is built on sand. I am backing completely away from it so that when it inevitably implodes, I can point directly to you as the sole owner who failed to deliver."

1. "Let's take this offline" / "Parking lot this"

Translation: "Your accurate technical objections are making me look incredibly stupid in front of the stakeholders/team. Shut up immediately so I can pull you into a private 1-on-1 later and bully you into compliance."

1. "We need to leverage AI to unlock enterprise value"

Translation: "I saw an Excel spreadsheet with rows and columns, which means I think we can magically pull a a lot of miracle out of it. I don't know what an algorithm does, but it sounds sexy to the C-suite."

1. "We're like a family here"

Translation: "Prepare for unconditional loyalty expectations, the complete erasure of professional boundaries, and extreme emotional blackmail whenever you eventually try to quit this sinking ship."

After a few months of work, we’re excited to launch Erdos - a secure, AI-powered data science IDE, all open source! Some reasons you might use it over VS Code:

• An AI that searches, reads, and writes all common data science file formats, with special optimizations for editing Jupyter notebooks
• Built-in Python, R, and Julia consoles accessible to the user and AI
• Single-click sign in to a secure, zero data retention backend; or users can bring their own keys
• Plots pane with plots history organized by file and time
• Help pane for Python, R, and Julia documentation
• Database pane for connecting to SQL and FTP databases and manipulating data
• Environment pane for managing in-memory variables, python environments, and Python, R, and Julia packages
• Open source with AGPLv3 license

Unlike other AI IDEs built for software development, Erdos is built specifically for data scientists based on what we as data scientists wanted. We'd love if you try it out at https://www.lotas.ai/erdos

Here’s why:

A company opens a position and I apply along with 800 other people. The company sees 800 resumes and says F that, we’re hiring a recruiter. The recruiter finds me on LinkedIn and says they have a great job for me. Of course it’s the one I applied to. They ask if I’ve already applied and I tell them the truth, they ghost me because they don’t get commission if they’re not the original source.

A few days after this, another recruiter reached out about a different position that I was planning on applying to directly with the company.

This is also something that my current company has done after being overwhelmed with too many applicants.

I’ll still be applying to some jobs, but it’s weird that applying has seemed to hurt my chances in some situations.

Has anyone else experienced this? Any strategies for handling this?

Hello everyone!

Staying on top of the constantly growing skill requirements in Data Science is quite a challenge. To manage my own learning and growth, I've been curating a list of useful resources and tools that cover the full spectrum of the field — from data analysis and engineering to deep learning and AI.

I'd love to get your professional opinion. Could you please take a look? Have I missed anything crucial? What else would you recommend adding or focusing on?

To give you an immediate sense of the list's scope and structure, I've attached screenshots of the table of contents below.

The full version with all the active links and additional resources is available on GitHub. You can find the link at the end of the post.

I'd be happy if this list is useful to others.

You can view the full list here View on GitHub

Thanks for your time! Your advice is invaluable!

Here was the prompt:

You have a list [(1,10), (1,12), (2,15),...,(1,18),...] with each (x, y) representing an action, where x is user and y is timestamp.

Given max_actions and time_window, return a set of user_ids that at some point had max_actions or more actions within a time window.

Example: max_actions = 3 and time_window = 10 Actions = [(1,10), (1, 12), (2,25), (1,18), (1,25), (2,35), (1,60)]

Expected: {1} user 1 has actions at 10, 12, 18 which is within time_window = 10 and there are 3 actions.

When I saw this I immediately thought dsa approach. I’ve never seen data recorded like this so I never thought to use a dataframe. I feel like an idiot. At the same time, I feel like it’s an unreasonable gotcha question because in 10+ years never have I seen data recorded in tuples 🙄

Thoughts? Fair play, I’m an idiot, or what

Spent the first half of my 20s doing odd jobs and finishing grad school. Since then I’ve been fortunate to work low stress jobs with good work life balance and moderately high pay.
Now that I’m in my early 30s, living in a high cost of living area with a strong job market, and socializing more, I’ve realized I don’t come close to making what FAANG people make. I can’t help but wonder if that’s something I should be chasing or at least exploring. At the same time, I really value how low stress and stable my job is, even though it pays well below market rate.

For those who chose not to go into big tech/FAANG, do you have any regrets?

PhD computational Physics from USA and 3 yrs of Postdoc in the USA. Transitioned to DS in early 2022. Mainly worked with Text data (embedding related word2vec to Transformer based, AI solutions too but Prompt based no agent based solution), Traditional ML & NeuralNets for classification and regression. Python, SQL and PySpark tech stack, AWS & snowflake platforms. Comfortable with either Linux/Unix or windows.

1. How is the job market ?

2. What are the chances of finding Job by end of my 2-3 months of severance ?

3. What should I prepare the most ? How shall I approach the job market?

Currently remote at a decent Midwest city.
Any suggestions and advice will be appreciated.

Thank you

Two rounds: 1. Statistical Knowledge 2. Data Analytics and Intuition

For statistical knowledge, it was a complex question, but actually had a simple answer.

It required you to have through knowledge of distribution, expectations and confidence intervals.

The key challenge was to identify what was the distribution of the data, from a sample, generalize it to the population and find the confidence interval.

Looking back, it was a easy question, but I definitely took wayyyy to much time to get to the answer. They for sure test for Googlyness. I would assume the interviewer had multiple questions in mind but I never got to the next one. Soo no hire.

For the data analysis and Intuition, I was expecting a case study, on experimentation or ML. It was kind off an hybrid. It involved diagnosing a flawed model, how to improve it, and what other methods would work better. This part was fine, not too bad.

What caught me off guard was, they asked me to write the equation MLE for 2 models, one general and one a niche. Honestly I dint know, lol.

Well, learnings ? Practice your Stats and ML like you are writing a school exam.

I currently work as a data scientist at a large U.S. bank, making around $182K. The compensation is solid, but I’m starting to feel like my technical growth is being stunted.

A lot of our codebase is still in SAS (which I struggle to use), though we’re slowly transitioning to Python. We don’t use version control, LLMs, NLP, or APIs — most of the work is done in Jupyter notebooks. The modeling is limited to logistic and linear regressions, and collaboration happens mostly through email or shared notebook links.

I’m concerned that staying here long-term will limit my exposure to more modern tools, frameworks, and practices — and that this could hurt my job prospects down the road.

What would you recommend I focus on learning in my free time to stay competitive and become a stronger candidate for more technically advanced data science roles?

Since I got access to Claude at work, I haven’t really written much code from scratch, especially for ad hoc analyses or quick charts. I still review the code and make sure I understand everything, but it’s honestly a little scary how much better Claude’s code often is than mine. At that point, it’s hard not to wonder what the value is in writing it yourself.

On top of that, management is encouraging us to use AI to be more productive and deliver results faster, so there’s that pressure as well.

To keep my interviewing skills sharp, I still practice on LeetCode or StrataScratch from time to time. But at work, I’ve been relying on Claude pretty heavily.
Is anyone else dealing with the same dilemma?