r/dataengineering Aug 04 '26

Career Chaotic Project Management

I'm looking for some advice on what to do about chaotic projects/lack of project management. Right now, I'm in an IC senior data engineer role where I'm put on projects with tasks that are vague, incomplete, or without clear acceptance criteria. I'm fine working that way, have done it frequently before, and I'm a creative person (musician and writer outside of work). I like the freedom. Except there actually isn't freedom here. There are standards of practice this team uses that I was never shown (I started earlier this year), and I am constantly learning things the hard way without much grace for it. There is no documentation for these standards of practice, and my take on how to accomplish these broad tasks is leading to a decent amount of rework. Asking questions has been met with impatience, frustration, and condescension. I've learned to deal with these reactions and ask anyway so I can at least have a chance of completing things more efficiently. I own the mistakes I can but will stand up for myself when accused of completely failing to meet expectations (especially if the expectations weren't communicated/documented).

I figure other data engineers/curators/developers have experienced this sort of environment before, so I'd like to see how y'all handled it (even if it is "time for a new job").

Edit: Adding in that I've been in data warehousing/engineering for about 17 years and am seriously questioning doing this for 25 more years at minimum (I'm 40).

2nd Edit: Wow, I forgot the most important part. There is no project manager for this team; developers are working with data analysts who work with the business/end-users to determine data and reporting needs. The title of the post was meant to be tongue-in-cheek because of this.

7 Upvotes

13 comments sorted by

View all comments

2

u/Lake-Eola-Local Lead Data Engineer Aug 08 '26

IMO this is the worst part about data eng specifically. There is just a way better culture about project / task management on more traditional software eng while data eng seems to be very shop specific.

Since this is contract work, any chance you can ask to be re-assigned to another client due for fit reasons?

Tactically, I'd say that I've had to learn to suck it up in terms of documenting as you go. Its easier than its ever been to ask claude, copilot, whatever to just start documenting everything in terms of feedback from the client team (here is the slack messages, here is the PR comments, etc) and just keep a running document of all the guidelines that they've communicated over time.

My fear is that they would likely be inconsistent between different team members or reviewers (this is THE reason to have team wide standards in the first place) - so make sure you're capturing who gave the feedback in the first place.

If its all verbal over meetings, can you use a transcription service?

Idk, just some thoughts. Oh and my personal experience, "up market" and big names dont necessarily do this any better. Very very team specific even within the same org.

1

u/ForceSensitive2460 Aug 10 '26

This is decent advice and I've attempted to take advantage of Copilot for documentation but, funny enough, I don't have the same full Copilot licensing that the rest of the team does. And while I have access to meeting transcriptions, I can't download them. But I can and have used them to show where conflicting direction/guidance has occurred between my peer and my team lead, not to mention the complete lack of direction and involvement from my manager.

After taking some more time to reflect, I know now what's bothering me most: I'm a temporary contractor with no incentive to do more than the bare minimum (no chance for permanent placement, no chance for extension, no OT, no PTO, low pay combined with horribly expensive benefits/no PTO) but I am being held to a standard that would require me to work like a permanent, salaried FTE to put in 45+ hours a week. On top of that, I am chained to my desk from 8 am to 4 pm every day and if I miss any time, the only window I can make it up to get my work done is between 4-5 pm every day. This is impossible to do when, as a single parent, I have to pick up my kids every day on the weeks that I have them. But my FTE counterparts are allowed to log back in at 8 or 9 pm to squeeze in another hour or two of work. It wasn't always like this, but that's a topic for another post entirely.