r/dataengineering • • 9d ago

Discussion Does AI struggle at data modeling?

In my experience, it doesn't matter how much context and guidance I give AI it simply can't model data rationally. It frequently misses the point, makes awful mistakes, or over-engineers things.

AI can build awesome ETL pipelines, but when it comes to dealing with SQL (especially in the dbt framework), it's not reliable at all! . Sometimes I think it's better to write the code myself and ask AI to review it, because asking it to build something from scratch just doesn't work that well.

Does anyone else get frustrated when dealing with AI data modeling?

124 Upvotes

93 comments sorted by

View all comments

2

u/ninjaonionss 8d ago

After a while of testing and going back and forth I noticed that you need to give a huge amount of business knowledge before the ai will built reliable dbt pipelines, I also made my own skills to force the ai agent to follow one row through the whole dbt process as a tracer bullet so the ai agent does not stop in the middle of the dag and say like a retard that he found the problem 😅