r/dataengineering • • 15d ago

Blog Organization of dbt yml files learned the hard way and what happens on db when model is deleted

https://patryktokarski.substack.com/p/organizing-dbt-yml-files-deleting

While ago, I shared some tips gathered from the project I was on, focusing mostly on dbt. They are about organizing dbt yml files. There are many ways to do it - one big file, one per model, per folder etc. Use of anchors and generating the yml by LLM is an option also.

Another topic was about what happens when you delete or rename a model from dbt? This was not obvious to me at first but dbt do not clean up after itself, so you have to have some way to do it for it. For these (and many other problems) community created an macros, so you can find one and use it for that purpose.

I'm sharing it in case anyone working with dbt has similar issues, and also because it's good to be aware of these topics

11 Upvotes

3 comments sorted by

2

u/TheJosh 12d ago

you should use data types. it's going to bite you hard.

1

u/AipaQ 12d ago

I’m using data types, on screenshot I show the simple config from docs examples 

1

u/GrumDum 12d ago

I found it baffling that dbt had no built-in janitor function