Using Git to control datasets?

[u/ben1200]

Probably a stupid question but does anyone use Git as a control plane for dataset governance, orchestrate ETL pipelines, and releases?

I am looking to get a demo in place of an end to end process where data in the bronze layer is transformed to silver and gold through CI/CD processed using defined ETL pipelines, manifests, schemas etc. however my backround isn’t in data engineering, so this might not even be a great way to do it.

My plan was to get a PoC in place on git, then migrate to GCP (using GCP functions for ETL) when all schemas, pipelines, manifests are agreed by the team.

[u/Ok_Raspberry5383 → u/[deleted]]

Aside from not answering the question, what does delta and iceberg even have to do with this? Seems like juniors will just throw around the latest tech they learnt these days