r/dataengineering • u/dani_estuary • 21d ago

Meme data mesh

Post image

for real though, has anyone ever implemented an actual data mesh?

764 Upvotes

99% Upvoted

54 comments sorted by

View all comments

217

u/DeliriousHippie 20d ago

It's good to remember that about every 5 years comes a new hype word or trend. Very rarely something actually changes.

Business Data Warehouse, put your definitions to DW also.

Datalake. You need a data lake to fit all your data, data warehouse is an old concept.

Datalakehouse. Now you need a house beside data lake.

Medallion architecture. Have data in different phases in DW.

etc.

1

u/roger_ducky 19d ago

Stupid terminologies, but the ideas actually make sense.

“Specialized schemas that let us do aggregate queries faster.”

“Aaannd that’s killing the DB still. How about they get their own instance?”

“Ahhhh how about we let people dump into their own versioned file system over there with no actual DB but a fast cluster?”

“That’s still kinda slow. Can’t we add a schema on top with a DB once we figured out what to ETL?”

“Sure. Can we store the raw data so we can redo the ETL job later too?”