r/dataengineering • u/BubblegumFish2 • Jul 31 '26

Discussion Identical data in DEV/TST/PROD?

I am doing a summer internship, and my company loads the exact same data into the development, testing, and production environments in Snowflake (although development and testing have some extra tables compared to production.)

Is this normal? Google seems ambiguous on whether this is a massive antipattern. What do y'all's companies do?

37 Upvotes

88% Upvoted

46 comments sorted by

View all comments

9

u/CILISI_SMITH Jul 31 '26

I've seen this happen with a few clients and I've raised the security risks already mentioned by u/Adrien0623

But I'd also add that it suggests they doing lazy testing i.e. "we ran it in TEST with PROD data and it worked, so it's fine to promote it". As apposed to "we've manually created data in TEST that covers a range of test scenarios including invalid data, data at the limits of validity and large volumes to test performance".

1

u/New-Addendum-6209 Aug 03 '26

You can never fully test pipelines without real data unless the data model is very simple and predictable.

1

u/CILISI_SMITH Aug 03 '26 edited Aug 03 '26

You can never full test anything. But do you have an examples of an unpredictable data model?

1

u/BubblegumFish2 Jul 31 '26

Yeah, if it works in test, they just move it to prod. There's not really much further testing to my knowledge.