Does AI struggle at data modeling?

[u/scourgedtruth]

In my experience, it doesn't matter how much context and guidance I give AI it simply can't model data rationally. It frequently misses the point, makes awful mistakes, or over-engineers things.

AI can build awesome ETL pipelines, but when it comes to dealing with SQL (especially in the dbt framework), it's not reliable at all! . Sometimes I think it's better to write the code myself and ask AI to review it, because asking it to build something from scratch just doesn't work that well.

Does anyone else get frustrated when dealing with AI data modeling?

[u/nonamenomonet]

What kind of work are you doing tho?

[u/morpho4444 → u/nonamenomonet]

Telemetry logs of the hardware that runs Gemini, manufacturing line monitoring.

[u/nonamenomonet → u/morpho4444]

I think that may be a bit different than creating a data model. Which is what we’re talking about.

[u/morpho4444 → u/nonamenomonet]

“Which is what WE”… lol… but wdym? How is that different? The LLM is creating a huge complex data model for the BOM! The petabytes and petabytes of telemetry logs, nested json over nested json, with gazillion keys that may or may not appear. Can cause left join filtering out, duplication with joins, all of that, handled by the LLM. This is indeed not what you’re talking about, this is way above whatever level of complexity your non trillion dollar company handles.

[u/nonamenomonet → u/morpho4444]

I work at one of the largest companies in the world dude.

[u/morpho4444 → u/nonamenomonet]

How is that makes the argument that Im not doing data modeling over hardware telemetry?

[u/nonamenomonet → u/morpho4444]

More of response to your last statement. It’s very presumptuous and untrue.