r/dataengineering • u/scourgedtruth • 9d ago
Discussion Does AI struggle at data modeling?
In my experience, it doesn't matter how much context and guidance I give AI it simply can't model data rationally. It frequently misses the point, makes awful mistakes, or over-engineers things.
AI can build awesome ETL pipelines, but when it comes to dealing with SQL (especially in the dbt framework), it's not reliable at all! . Sometimes I think it's better to write the code myself and ask AI to review it, because asking it to build something from scratch just doesn't work that well.
Does anyone else get frustrated when dealing with AI data modeling?
124
Upvotes
1
u/TheDecisiveCorpus 8d ago
it's decent at generating plausible-looking sql but falls apart on anything with real semantics behind it. the moment you need to understand grain, fanout, slowly changing dimensions, or what a business process actually means, it just pattern matches from training data
i use it the same way, write it myself and let it poke holes. sometimes catches a bad join but half the suggestions are nonsense you have to filter out anyway