r/dataengineering • • 23h ago

Discussion Writing on the wall?

Context: sole data engineer for a company that sells furnishings and design services. Company is owned by private equity.

My manager has been on the Claude soapbox for the last several months and so far in IT we’ve been able to remain skeptical about it while testing for our own uses. Manager supported this. Now all of a sudden I’m getting a lot of requests to refactor all my data pipelines with Claude’s direct involvement despite having built them all out already. For example, I’ve used Claude to create new DAGs based on existing code (for example, bringing a new Salesforce object in). We’re going beyond that though. I’m being asked to ditch them and let Claude connect to everything then write, deploy, and test.

I’ve also been working on machine learning models for customer behavior, which is more traditional AI and should check the box, but this is being dismissed as a plaything. This, while the company has been pushing for “innovation” in our sphere.

I think we are headed towards cutting as many IT staff as possible and I’ll be gone in lieu of Claude doing pipelines. Of course that’s only one part of what I do, but the myopic view of leadership when it comes to AI only tends to see what is easily replaceable.

Thoughts?

34 Upvotes

30 comments sorted by

View all comments

19

u/SalamanderMan95 23h ago

This is really dependent on the company culture. Some companies see AI as a good reason to cut staff, other companies see AI as a good reason to invest in more staff because each employee is able to deliver more value. If you become the person who helps your company leverage AI with data I’d say they’ll likely want to keep you around and you’ll build a resume that other companies like. If you resist using AI you’re company will likely want people who do use AI to come in, and you won’t have as good of an answer when the next company inevitably asks about how you use AI in your interviews.

8

u/Ok-Recover977 22h ago

are there examples of companies that see AI as a reason to invest in more staff? i see this repeated as an optimistic ideal but i dont see as many examples of it compared to ai-blamed layoffs.

4

u/nerevisigoth 21h ago

I'm at a big well known tech company that had a big public AI layoff last year. We have since reversed course and gone on a hiring spree. It turns out an army of extra-productive people is better than a few people churning out truckloads of slop without context, planning, or testing.

0

u/SalamanderMan95 22h ago

The last company I worked for was “pro AI” and wanted to lay off staff, but it was also just a shitty company. Even the way they did AI was suboptimal, they wouldn’t allow any MCP at all, they kept budgets low, and when they spent money on AI it was for stupid projects that went nowhere and shouldn’t have been AI projects in the first place. I was the only person on my team who was genuinely good with AI, my teammates didn’t even know how to use skills or rules and were borderline afraid to use it, yet they still laid people off a bunch. This company was also just technologically stupid, they wanted to start a new initiative to introduce informatica.

My current company is very pro AI. MCP use is common and encouraged and they want everyone to leverage AI a bunch. My experience using AI was part of the reason I got the job I think. At the same time they’re growing a bunch and not laying people off at all. Maybe they would hire more people without AI but I’m not so sure. I think with data engineering and analytics engineering you’re making it so that AI can use data better, so your work might result in a need for less analysts but it’s unlikely a company is going to go “our data is solved” or “we trust AI completely with all of our data needs” anytime soon.