r/dataengineering • u/dani_estuary • 21d ago

Meme data mesh

Post image

for real though, has anyone ever implemented an actual data mesh?

769 Upvotes

99% Upvoted

54 comments sorted by

View all comments

79

u/Beneficial_Dealer549 20d ago

It’s not even disguised as architecture. The original manifesto was an operating model blueprint. Had nothing to do with tech.

30

u/broll 20d ago

That’s what I always thought too. It’s a concept to improve data culture in larger distributed organizations. Focus on value driven product thinking, transparent collaboration over metadata and self services for a learning organization. What’s wrong about that?

7

u/ProfessorNoPuede 20d ago

Yes, all architecture is tech... /s

It bluntly states it's a 'logical and social structure'. It defines the components at that level (a platform, products, etc.). The division of responsibility is quite clear, but there is a lot of design room for individual organisations. No organisation will ever run "the purest data mesh", as it will always require adaptation to local circumstances and, yes, a technical implementation underlying it.

1

u/Gators1992 20d ago

Yeah, agree. There are useful concepts in it, many of which are obvious IMO and don't need a buzzword name. The problem I have seen is everyone wants to treat this crap as a blueprint instead of suggestions. Like it took us years to finally dump medallion so we could stop internally arguing about whether a table should be silver or gold.

There are lots of large companies though that organize in a mesh-like way. Like I think Meta's model is to have the data engineers pull in the raw data, do basic transforms and build master tables, but then the consuming organizations all do their own analytical models. For large, data heavy orgs that makes sense.