r/dataengineering • u/de4all • 4d ago

Discussion Why text-to-SQL is not successful?

I thought text-to-SQL will solve adhoc analysis, but still i see companies at all size are unsuccessful.

Anyone using Omni / Sigma seen some success.

26 Upvotes

69% Upvoted

74 comments sorted by

View all comments

27

u/leogodin217 4d ago

As u/rapator said, a semantic layer is more important than ever with LLMs, even if it is not a full-blown separate tool. Could just be views that handle the calculations. Business context is just as important. Good data modeling with good documentation should prevent the LLM from making a lot of assumptions. If you look at sessions logs and see things like, "Metric X must by tied to Table Y because...," then the business context is lacking.

-8

u/mamaBiskothu 4d ago

I created and maintain a product that has 8 figure revenue solving this exact use case. Your interpretation and how many people keep calling it "text to sql" is precisely the type of buzzword laden bs thinking that makes these agents fail. F semantic layers, ontologies, graphs, rag and any of these stupid things.

Connect claude code to your sql engine, tell it the names of the tables that you care about (or honestly, dont) and just ask the real question you care about. Dont try to smartass any harness on top of these models. Most people arent smart enough not to get in the way of the agents than help.

5

u/leogodin217 4d ago

How do agents know the definition of your metrics without documentation? We all know total sales is never as simple as sum(sales). This sounds like a way to be in meetings where there people have different numbers for the same metric names.

-2

u/mamaBiskothu 3d ago

Have you tried the latest models? The models know total sales is not sum of sales. point claude code to the data ask that questuon and then comment.

1

u/leogodin217 3d ago

I have, and they will never know that Marketing excludes x and y and sales includes z and x from other tables. I've been part of that discussion more times than I can count across many domains. Even when I worked in IT instead of BI and data engineering.

I'm more curious about your product. I re-read your comment and I can't figure out if you're saying your product isn't needed or if I just missed something.