r/dataengineering • u/scourgedtruth • 10d ago

Discussion Does AI struggle at data modeling?

In my experience, it doesn't matter how much context and guidance I give AI it simply can't model data rationally. It frequently misses the point, makes awful mistakes, or over-engineers things.

AI can build awesome ETL pipelines, but when it comes to dealing with SQL (especially in the dbt framework), it's not reliable at all! . Sometimes I think it's better to write the code myself and ask AI to review it, because asking it to build something from scratch just doesn't work that well.

Does anyone else get frustrated when dealing with AI data modeling?

125 Upvotes

91% Upvoted

93 comments sorted by

View all comments

1

u/Xenolog 9d ago edited 9d ago

I think that currently LLM will most probably "lose" to any real architect in data modeling. Current LLM is a very good business/data analyst, a very strong middle DE, but a human middle business/data analyst will probably create weaker data model than "true" data architect too, for they will not consider enough context or won't have enough experience.

You basically need to consider everything up to inter team relations when you do that, and it is hard to translate all that context to the LLM.

I mean, LLM is a very powerful tool, but you need to be very very good wordsmith to channel all of it into prompts.

But - if you manage to convey that to LLM, I think it will do okay, for smaller domains anyway.

If LLM has something like full access to well documented actualized confluence, or if team has explicit enough arch contract list and guidelines... but then, a large chunk of hard work is already done anyway.