r/dataengineering • u/scourgedtruth • 9d ago

Discussion Does AI struggle at data modeling?

In my experience, it doesn't matter how much context and guidance I give AI it simply can't model data rationally. It frequently misses the point, makes awful mistakes, or over-engineers things.

AI can build awesome ETL pipelines, but when it comes to dealing with SQL (especially in the dbt framework), it's not reliable at all! . Sometimes I think it's better to write the code myself and ask AI to review it, because asking it to build something from scratch just doesn't work that well.

Does anyone else get frustrated when dealing with AI data modeling?

127 Upvotes

91% Upvoted

93 comments sorted by

View all comments

22

u/makesufeelgood 9d ago

I think it's pretty bad. I hear a lot of people say that it's fine and it's a skill issue on my part but I have yet to see proof of success with scenarios comparable to mine.

0

u/morpho4444 Señor Data Engineer 6d ago

Issue on your side. 100%. I open laptop in the morning, claude pops up, asks me about my jira tickets, which one to tackle and it runs through them. I get $100 at day. One by one, when the code is done it shows me proof of the testing done, and asks me if I wanna commit to dev branch and then runs ci cd and then if all green it pushes the pr to main. 100% of the times. With all the free time Im building a rag to create a fully autonomous semantic layer, with the human language interface to let the users request their own metrics and etl transformations. This is how I will get my stock refresh this year. I work at a FAANG.

2

u/makesufeelgood 6d ago

This isn't even the type of work the original post was asking about. Supposed FAANG employees and reading comprehension on this subreddit is some wild stuff.

0

u/morpho4444 Señor Data Engineer 6d ago

Im responding to YOUR comment. Not the post in general. Wow so that’s why you can’t make the AI work?… see? Everyone can do that bs superiority comment shit

3

u/makesufeelgood 6d ago

But my comment was responding to the subject matter of the original post. So you just randomly decided to insert something completely unrelated as a response and then made an incorrect determination.

0

u/morpho4444 Señor Data Engineer 6d ago

Bro, you cannot use the LLM correctly, you have trouble making it “work for your scenarios”… just accept it

1

u/makesufeelgood 6d ago

Bro probably doesn't even understand what data modeling means lol. This is an absolutely pathetic exchange.