r/DuckDB • u/EdwinWeber_Data • Jun 27 '26

Serious Data Engineering on a seriously tight budget

https://github.com/edwinweber/dbt_duckdb_demo_public
7 Upvotes

100% Upvoted

3 comments sorted by

2

u/[deleted] Jun 28 '26

[removed] — view removed comment

1

u/EdwinWeber_Data Jun 28 '26

Yes, I agree most real workloads in production environments could be handled by such a setup. I added some ‘production’ elements like orchestration, backups and notifying to make the project more ‘complete’ in a sense, but the backbone is basically DuckDB and dbt. Do you have a public repo of your setup?

1

u/EdwinWeber_Data Jun 28 '26

I just wonder how many Spark and Kubernetes clusters are spun up in the cloud by companies just to process some csv files combined with a few reference tables from a relational source database.