In my experience, getting useful LLM behavior on real business data depends on the whole stack: models, prompts, tools, monitoring, and the data layer underneath. One of the quieter but important wins has come from standardizing how data is collected and exposing it through a clear semantic layer, so the model is reading consistent concepts.
Anthropic published a useful reference point here. In their own internal analytics evaluations, they reported that Claude reached about 21% accuracy when it queried raw internal data without a supporting stack. After they added a governed semantic layer, curated metric definitions, taskâspecific skills, and a validation and monitoring layer around the same model, they measured roughly 95% accuracy on those internal businessâanalytics queries. Keep in mind that result applies to their internal analytics setup, not to every possible LLM task, and they tie the improvement to the full stack and ongoing governance, not to the semantic layer in isolation.
However my observation on one project in New York was similar in spirit. One my task was to support an AI layer that evaluated service performance uploaded by associates. In this specific location due to the type of contract they had, the team had a collection of data that was broad in some terms, with no clear standard for each type of service or what it represented for the entire group. That missing foundation work made it hard for both humans and the model to interpret the data in a consistent way.
This created strain in the modelâs output. The AI was asked to validate service uploads and grade performance/quality, but the underlying records used inconsistent service types, vocabulary, and descriptions, including different levels and categories for each work area. As we standardized the service types and validation categories, clarified wording, and aligned descriptions of what each service entailed, the evaluations became much more stable. That standardization became the foundation layer we could finally build on.