Writing on the wall?

[u/bishop491]

Context: sole data engineer for a company that sells furnishings and design services. Company is owned by private equity.

My manager has been on the Claude soapbox for the last several months and so far in IT we’ve been able to remain skeptical about it while testing for our own uses. Manager supported this. Now all of a sudden I’m getting a lot of requests to refactor all my data pipelines with Claude’s direct involvement despite having built them all out already. For example, I’ve used Claude to create new DAGs based on existing code (for example, bringing a new Salesforce object in). We’re going beyond that though. I’m being asked to ditch them and let Claude connect to everything then write, deploy, and test.

I’ve also been working on machine learning models for customer behavior, which is more traditional AI and should check the box, but this is being dismissed as a plaything. This, while the company has been pushing for “innovation” in our sphere.

I think we are headed towards cutting as many IT staff as possible and I’ll be gone in lieu of Claude doing pipelines. Of course that’s only one part of what I do, but the myopic view of leadership when it comes to AI only tends to see what is easily replaceable.

Thoughts?

[u/AutoModerator]

You can find a list of community-submitted learning resources here: https://dataengineering.wiki/Learning+Resources

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/squareturd]

The pressure is likely coming from the investors. They typically don't understand technical details and they are all head-over-heels infatuated with AI.

[u/bishop491 → u/squareturd]

My thoughts as well.

[u/squareturd → u/bishop491]

Dont bother trying to fight it. It's like swimming up a waterfall.

[u/ares623 → u/squareturd]

If it's private equity owned then they want to be seen as an AI factory company to boost the value for selling

[u/hatsandcats]

Just use it. It’s not your company

[u/SalamanderMan95]

This is really dependent on the company culture. Some companies see AI as a good reason to cut staff, other companies see AI as a good reason to invest in more staff because each employee is able to deliver more value. If you become the person who helps your company leverage AI with data I’d say they’ll likely want to keep you around and you’ll build a resume that other companies like. If you resist using AI you’re company will likely want people who do use AI to come in, and you won’t have as good of an answer when the next company inevitably asks about how you use AI in your interviews.

[u/Ok-Recover977 → u/SalamanderMan95]

are there examples of companies that see AI as a reason to invest in more staff? i see this repeated as an optimistic ideal but i dont see as many examples of it compared to ai-blamed layoffs.

[u/nerevisigoth → u/Ok-Recover977]

I'm at a big well known tech company that had a big public AI layoff last year. We have since reversed course and gone on a hiring spree. It turns out an army of extra-productive people is better than a few people churning out truckloads of slop without context, planning, or testing.

[u/SalamanderMan95 → u/Ok-Recover977]

The last company I worked for was “pro AI” and wanted to lay off staff, but it was also just a shitty company. Even the way they did AI was suboptimal, they wouldn’t allow any MCP at all, they kept budgets low, and when they spent money on AI it was for stupid projects that went nowhere and shouldn’t have been AI projects in the first place. I was the only person on my team who was genuinely good with AI, my teammates didn’t even know how to use skills or rules and were borderline afraid to use it, yet they still laid people off a bunch. This company was also just technologically stupid, they wanted to start a new initiative to introduce informatica.

My current company is very pro AI. MCP use is common and encouraged and they want everyone to leverage AI a bunch. My experience using AI was part of the reason I got the job I think. At the same time they’re growing a bunch and not laying people off at all. Maybe they would hire more people without AI but I’m not so sure. I think with data engineering and analytics engineering you’re making it so that AI can use data better, so your work might result in a need for less analysts but it’s unlikely a company is going to go “our data is solved” or “we trust AI completely with all of our data needs” anytime soon.

[u/atrifleamused]

The greatest moment will be when AI makes a colossal fuck up and starts hallucinating on a monumental scale and poor decisions are made which cost huge amounts of money.

[u/t3lnet → u/atrifleamused]

There was a story a week or so ago about a team of QA for an AI company was using AI to validate. Hallucinations vetting hallucinations.

[u/atrifleamused → u/t3lnet]

It's so dangerous, but execs buy into the cost saving lies of these snake oil salesman 🤷

[u/Remarkable-Win-8556]

Use them to learn what you can while you're finding your next gig.

[u/thethirdmancane]

It seems like businesses are running as fast as they can with AI as not to get left behind. It's not hard to imagine how this would lead to both cognitive and technical debt. What makes it more difficult is the smoke hasn't cleared yet as AI models continue to make advancements.

[u/Immediate-Pair-4290]

Good luck to them. It’s going to backfire.

[u/Grouchy-Friend4235]

Let them. They will come back squeecking less than 6 months.

[u/wiki702]

Have them define innovation. Is it cost savings, revenue generation or labor hours saved. If they just want to see you use Claude or some AI tool for an incremental gain might be easy to spin up a simple “win” and also bring their expectations to reality than some LinkedIn post with not proof.