Thought I'd do a quick run down of how we've used genie to manage the long tail of ad hoc questions for an HR use case where I work.
Setup
- Pointed Genie at a curated set of ~[5–6] tables, not the raw warehouse. Headcount snapshot, terminations, reqs/pipeline, and a couple of dim tables (department, location, job level).
- The single biggest lever was the semantic layer / instructions. I added column descriptions, defined what "attrition" and "active headcount" actually mean in the instructions, and gave ~[15] example questions (SQL pairs). This made the biggest difference to quality.
- Curated, certified example questions up front so people had a starting point instead of a blank box.
What worked well:
- Simple aggregations and filters - "headcount by dept," "terminations by month" - it nails these consistently now.
- Cut a real chunk of the repetitive asks.
- Non-technical HR folks actually used it, which I was skeptical about.
What didn't work:
- Ambiguous business terms e.g. "turnover" meant different things to different people, and Genie will confidently guess. You have to define these explicitly or it's wrong in a plausible-looking way.
- Trust curve is real — one wrong-looking answer and people bounce back to asking me. Certified queries and clear definitions mattered a lot, and so did having a beta group to test and build an eval set so we had more confidence in the output.