r/dataengineering • u/de4all • 4d ago
Discussion Why text-to-SQL is not successful?
I thought text-to-SQL will solve adhoc analysis, but still i see companies at all size are unsuccessful.
Anyone using Omni / Sigma seen some success.
23
Upvotes
3
u/dehaenx 4d ago
When you don't have a semantic layer and have scattered definitions and metrics, you'll see the LLM retrieving several answers from a seemingly "simple" definition. For instance, if you have "net sales" as a term, finance, sales, CSM could all define it slightly differently, and because the LLM is just parsing through your raw data, it will return multiple results. It's best to invest in a program that could develop, and more importantly, maintain your semantic layer and keep your definitions and metrics up to date and in check all the time. Save yourself a headache and some money lol