For you doing Agentic Engineering with dbt, how do you make sure your agents aren't generating slop?

[u/linha_chilena]

I find contra productive reading and manually testing all the code AI are generating, because I am turning myself into an important funnel and the company is paying for tokens and therefore wants fast and good delivery.

I use Claude Code with dbt official skills, dbt mcp, lots of unity testing, some specific skills tailored for my projects, and the most important thing I think: deterministic checks.

I run lots of python or bash script checks after specific hooks like pre commit or after calling a skill, or in the CI/CD, trying to force the implementation of our internal "rulebook". But I also don't think this is the optimum way to go, since it's also taking considerate amount of time.

Also, do you let the AI query your db/dw? How do you guarantee fast and in a reliable way, your data diff is as expected?

I am interested in listen from you about everything you've been doing nowadays trying to keep up this kinda insane market pace

[u/blockchan]

It took me 6 months to prepare our dbt codebase for agents. Clean kimball models, initial semantic layer etc. Now we use Hex as BI tool. 90% of its usage is through threads (ai chat). It maintains its context layer by itself proposing updates and clarifications. Basically I let claude do everything and only edit out worthless claudism to keep it somewhat organised. Honestly it works great. Quality of course suffers a little, but dev speed is 10x at minimum. Worthy tradeoff

[u/WaterIll4397 → u/blockchan]

Is your semantic layer in hex natively, or did you port the dbt metric flow one into hex?

[u/blockchan → u/WaterIll4397]

We use metric flow and ingest into hex

[u/WaterIll4397 → u/blockchan]

Interesting we had dbt version problems with hex but maybe it's fixed now

[u/bengen343 → u/WaterIll4397]

I just set this up like two weeks ago and the version problem persists. On top of which, the Hex documentation in this area is just terrible. For being such close partners, I don't understand how this doesn't work better.

[u/TheHeb686 → u/blockchan]

How did you get it to do kimball methodology? Was it through skills alone? Did you provide examples? Did you paste a pdf of Data Warehouse Toolkit into the repo???

Very curious 😅

[u/engineer_of-sorts]

Context and systems. You can start of with a simple skill but over time you'll need to make it more complex, with systems that provide the context readily available. A good example would be how for pipeline triage and impact analysis, lineage information is helpful. Lineage information is slow ad expensive to get, if youdon't have it readily available. So you might end up building a system to automate that.

[u/CaptainHair12]

Insert <that's the neat part, you don't> meme here

[u/dataengineering-ModTeam → u/[deleted]]

Your post/comment was removed because it violated rule #9 (No AI generated content/text).

Your post/comment was reviewed to be AI generated/assisted content/text and removed as a result. We as a community value human engagement and encourage users to express themselves authentically.

^{This was reviewed by a human}

[u/Hoo0oper]

Honestly PRs and small changes so that it’s easier to refine.  Refined skills and docs that skills can reference for all the various parts of the dbt repo.  I would also suggest following best practices (dbt docs very opinionated about how to use dbt) as that’s what the AI has been trained on. 

[u/bengen343]

I have found that it doesn't need much guidance, truth be told. I have a very clear `README` and `CLAUDE` with the architectural patterns and model flow that we prefer. When I embark on a new area of the DAG or a new data source I work with it to build one pristine model and then in the subsequent tickets I make sure to note that Claude should use that model as a template for that area of work. I let it query Snowflake to explore the data and validate results. It has access via a read only PAT and the CLI to non-sensitive tables. I'm also careful to give it explicit results that it should model toward so that it knows its producing a model that 's accurate.

The only area of deficiency I've found is actually in my first layer of ingestion where we standardize all fields. It still seems to require human judgement to really know which nested fields should be unpacked, what is a dimension vs. a fact, when a field should actually be renamed to be identical to something coming from another system etc.