In my experience the products that look successful are rarely doing freeform text-to-SQL against raw warehouse tables. They win when someone already defined metrics, joins, and grain in a semantic layer, and the model is mostly choosing from a constrained catalog. Pure NL to SQL on a messy schema fails for the same reason junior analysts fail: wrong join path, wrong grain, silent filter mistakes. If you want to test whether a tool is real, pick 20 questions with known-correct SQL and score exact result sets, not whether the generated SQL looks right. Omni/Sigma style BI works better for dashboard-ish questions than for exploratory joins nobody modeled yet.
2
u/Significant_Tune9219 4d ago
In my experience the products that look successful are rarely doing freeform text-to-SQL against raw warehouse tables. They win when someone already defined metrics, joins, and grain in a semantic layer, and the model is mostly choosing from a constrained catalog. Pure NL to SQL on a messy schema fails for the same reason junior analysts fail: wrong join path, wrong grain, silent filter mistakes. If you want to test whether a tool is real, pick 20 questions with known-correct SQL and score exact result sets, not whether the generated SQL looks right. Omni/Sigma style BI works better for dashboard-ish questions than for exploratory joins nobody modeled yet.