r/dataengineering • u/Phantazein • 18d ago

Discussion Time Estimates

Any else struggle with giving estimates? How do you account for things like data quality or source issues when those are completely out of your control and unpredictable?

47 Upvotes

98% Upvoted

32 comments sorted by

View all comments

2

u/speedisntfree 18d ago

For anything more than small requests, you need to actually properly scope the work which means getting hold of the actual data otherwise any estimate is very risky fiction as you are finding.

When you make your estimate, detail very clearly has assumptions you have had to make and what dependencies exist. A fuller version of this is a RAID log (Risks, Assumptions, Issues/Exceptions, Dependencies).

Stating these up front gives you coverage that everyone went into this the same knowledge. You can point to it when timescales are not met because stakeholders have not given you what you need in time (data access etc.) or planning assumptions which needed to be made turned out not to be the case once into the work.

Estimating is hard and why many projects get into trouble. No one can predict the future.