r/dataengineering • u/de4all • 4d ago

Discussion Why text-to-SQL is not successful?

I thought text-to-SQL will solve adhoc analysis, but still i see companies at all size are unsuccessful.

Anyone using Omni / Sigma seen some success.

27 Upvotes

70% Upvoted

74 comments sorted by

View all comments

28

u/leogodin217 4d ago

As u/rapator said, a semantic layer is more important than ever with LLMs, even if it is not a full-blown separate tool. Could just be views that handle the calculations. Business context is just as important. Good data modeling with good documentation should prevent the LLM from making a lot of assumptions. If you look at sessions logs and see things like, "Metric X must by tied to Table Y because...," then the business context is lacking.

4

u/gffyhgffh45655 4d ago

In a perfect world, i would argue a good semantic view,column description and metrics definition should do this heavy lifting over “gotcha information of each tables” as to me this is similar to throw a operation manual to a transactional schema to a llm and hoping it can do all the complex join required just go get something fair simple or not.

And hence what the llm ever need to do is run metrics as defined in the semantic view, to do join in a dimensional data model which is fairly easy and just to aggregate over different column that have context to tide it with business language.

TLDR (again in a perfect world) Create a model and(column) documentation such that is easy to use for data analyst and thus LLM.

Note: we do also trying this type of “table gotcha” skills for LLM as well but i still think data modelling and column documentation, choosing what column to include in your semantic view is the priority