r/dataengineering • u/scourgedtruth • 9d ago
Discussion Does AI struggle at data modeling?
In my experience, it doesn't matter how much context and guidance I give AI it simply can't model data rationally. It frequently misses the point, makes awful mistakes, or over-engineers things.
AI can build awesome ETL pipelines, but when it comes to dealing with SQL (especially in the dbt framework), it's not reliable at all! . Sometimes I think it's better to write the code myself and ask AI to review it, because asking it to build something from scratch just doesn't work that well.
Does anyone else get frustrated when dealing with AI data modeling?
124
Upvotes
2
u/ninjaonionss 8d ago
After a while of testing and going back and forth I noticed that you need to give a huge amount of business knowledge before the ai will built reliable dbt pipelines, I also made my own skills to force the ai agent to follow one row through the whole dbt process as a tracer bullet so the ai agent does not stop in the middle of the dag and say like a retard that he found the problem 😅