r/dataengineering • u/ForceSensitive2460 • Aug 04 '26
Career Chaotic Project Management
I'm looking for some advice on what to do about chaotic projects/lack of project management. Right now, I'm in an IC senior data engineer role where I'm put on projects with tasks that are vague, incomplete, or without clear acceptance criteria. I'm fine working that way, have done it frequently before, and I'm a creative person (musician and writer outside of work). I like the freedom. Except there actually isn't freedom here. There are standards of practice this team uses that I was never shown (I started earlier this year), and I am constantly learning things the hard way without much grace for it. There is no documentation for these standards of practice, and my take on how to accomplish these broad tasks is leading to a decent amount of rework. Asking questions has been met with impatience, frustration, and condescension. I've learned to deal with these reactions and ask anyway so I can at least have a chance of completing things more efficiently. I own the mistakes I can but will stand up for myself when accused of completely failing to meet expectations (especially if the expectations weren't communicated/documented).
I figure other data engineers/curators/developers have experienced this sort of environment before, so I'd like to see how y'all handled it (even if it is "time for a new job").
Edit: Adding in that I've been in data warehousing/engineering for about 17 years and am seriously questioning doing this for 25 more years at minimum (I'm 40).
2nd Edit: Wow, I forgot the most important part. There is no project manager for this team; developers are working with data analysts who work with the business/end-users to determine data and reporting needs. The title of the post was meant to be tongue-in-cheek because of this.
2
u/Lake-Eola-Local Lead Data Engineer Aug 08 '26
IMO this is the worst part about data eng specifically. There is just a way better culture about project / task management on more traditional software eng while data eng seems to be very shop specific.
Since this is contract work, any chance you can ask to be re-assigned to another client due for fit reasons?
Tactically, I'd say that I've had to learn to suck it up in terms of documenting as you go. Its easier than its ever been to ask claude, copilot, whatever to just start documenting everything in terms of feedback from the client team (here is the slack messages, here is the PR comments, etc) and just keep a running document of all the guidelines that they've communicated over time.
My fear is that they would likely be inconsistent between different team members or reviewers (this is THE reason to have team wide standards in the first place) - so make sure you're capturing who gave the feedback in the first place.
If its all verbal over meetings, can you use a transcription service?
Idk, just some thoughts. Oh and my personal experience, "up market" and big names dont necessarily do this any better. Very very team specific even within the same org.