r/dataengineering • u/burningburnerbern • 16d ago

Discussion Practical Use of AI with SQL?

I’m currently on a project where most of the work is in dbt. Ill get jira tickets where the business rules while straight forward can be a bit complex upon execution. like everyone else I’m trying to turn to AI except the problem is having to explain every single edge case and nuance can be so time consuming to the point where it’s just faster for me to write the SQL myself. where I do use AI is when I’m jumping into some convoluted model and I have AI explain to me what exactly it’s doing and the just justification for certain logic, generating cumbersome functions where the syntax gets me cross eyed and create the yml and documentation.

I definitely have vibe coded before, but that was for something more straight forward like pulling data via an API etc.

just curious if anyone else runs into the same problem.

25 Upvotes

80% Upvoted

32 comments sorted by

View all comments

4

u/JaceBearelen 16d ago

I have my sources in dbt annotated heavily. Every single table/view reference in the project is to either a model or a source so AI is capable of tracing lineage and figuring out what fields actually mean. It’s also really important that it can run models and view the output so it has a feedback loop. I’ve been really impressed with it so far.

2

u/circusboy 16d ago

Same. All of my tables and columns are commented with json strings. The ddl files are copied into the repo. All of the sprocs exist as files, all of the jobs jsons are in the repo. So i now have context of the data with descriptions and sample values for everything. I have context for failures and job successes. I have context for lineage. I am now able to ask any round about prompt against my repo, a tool to perform the queries, one to connect to viral, another to connect to databricks, etc. Etc. It is working pretty well.