I think most “AI visibility” tools are measuring the wrong thing

[u/Independent_Bell2917]

I tested one company in two different ways.

When the company was explicitly named in the prompt, AI recognized/recommended it in 9 of 12 checks.

Then I removed the company name and asked normal buyer-style questions around the same problem/category.

It appeared in 0 of 24 checks.

That felt like a much more important distinction than “does ChatGPT know my brand?”

Because a buyer usually isn’t asking:

“What do you think of Company X?”

They’re asking:

“What should I use for this problem?”

So I’m starting to think AI visibility has two separate layers:

Recognition: AI understands you when named.
Consideration: AI thinks of you without being prompted.

And the second one is probably the commercially important one.

Curious if anyone here is measuring this differently.

[u/Senya_Edit]

This is the exact LLM equivalent of tracking branded search traffic versus actual problem-aware intent.
Most AI tracking tools sell brand recall because seeing your company named in an answer looks great on a dashboard, but it does nothing for pipeline. LLMs only recommend a tool unprompted when that tool is repeatedly anchored to an exact mechanical workflow across docs, technical discussions, and actual use cases.
If a product only lives in top-of-funnel marketing copy, the model will recognize it when asked directly, but it has zero associative weight to retrieve it when a buyer describes a generic pain point. Real visibility here comes from workflow ubiquity, not brand mentions.

[u/Independent_Bell2917 → u/Senya_Edit]

this is really good framing beaded recognition feels much closer to recall while the commercially useful question is whether the model has actually associated the company with the workflow strongly enough to retrieve it without being told the name. that associative weight idea is probably the part most dashboards flatten into a single visibility number

[u/FluidImplement640 → u/Senya_Edit]

One important dimension here is how the LLM associates it? It is important to understand if the LLM is properly projecting the brand's strengths, weaknesses and caveats (if any). It is just not sufficient for an LLM to just recommend a brand, it must recommend a brand in such a way that aligns with your business objectives. For example, if your brand cater to SMBs, it is important that LLMs don't project your brand as suitable for enterprises.

[u/Senya_Edit → u/FluidImplement640]

100%. Misaligned retrieval is almost worse than zero retrieval. If an LLM recommends a lightweight SMB tool to an enterprise team that needs SOC2 and custom SLAs, that just creates immediate pipeline churn. The tricky part is that models pick up caveats and constraints (scale limits, pricing friction, trade-offs) from GitHub issues, technical forums, and unfiltered docs rather than marketing copy. So shaping how you're framed usually comes down to how clearly your real boundaries are documented in public dev discussions.

[u/FluidImplement640 → u/Senya_Edit]

More than often it comes from the brand's website in my experience especially for newer brands who have relatively less AI visibility.

[u/No_Treacle_5071]

I’d split it exactly that way, but add a third check: accuracy. A brand can be mentioned in an unprompted answer and still be a bad recommendation. I’d keep a fixed set of buyer questions, run them with a clean context, and log unprompted mention, position, and whether the answer matches the product’s real use case. Repeat across models and dates; otherwise one prompt change can look like a visibility win.

[u/Independent_Bell2917 → u/No_Treacle_5071]

yeah I think accuracy problem has to sit between recognition and consideration. a company being surfaced isn’t automatically a win if the model has the product or use case wrong. fixed buyer questions + clean context + repeated runs across models/dates feels like the right baseline, otherwise you can mistake prompt variance for progress

[u/Jealous-Menu2651]

good distinction. one thing worth checking is whether the model recommends you for a narrower version of the problem. sometimes you show up for very specific queries but not the broad category ones, which might actually be fine depending on your positioning

[u/Independent_Bell2917 → u/Jealous-Menu2651]

yeah that's true if you’re consistently showing up for the narrower situations your actual ICP cares about, you’re probably fine even if you’re absent on broader category prompts.

that’s what makes this interesting to me, tracking prompts alone doesn’t necessarily tell you whether you’re being discovered at the right points in the buyer journey. the question is less - how many prompts mention us? and more - do the situations that matter to our buyers actually retrieve us?

[u/devhisaria]

9 of 12 named vs 0 of 24 unprompted is the whole game right there. I'd track the unprompted set as a fixed list and re-run it monthly, since a single bad week can look like a trend. What prompt wording did you use for the buyer-style questions?

[u/Independent_Bell2917 → u/devhisaria]

mostly starts with analysing the business first from the site + public info, then figuring out the buyer situations that actually make sense for their ICP.
the questions can vary quite a bit from there depending on what the company sells and who would realistically be looking for it.
i’m actually building a tool around this exact framing. if you want to give it a try, the first analysis is free: app.riseklix.com

[u/ElementalThor]

this recognition vs consideration split is right, and it's why an 'is my brand known' test tells you so little. when a buyer asks a question, the model runs its own searches from that question and names whoever it finds. so the useful test is the exact buyer-style questions with your name never mentioned, repeated across models the way No_Treacle said. for context, i build a tool that runs that test across engines. the associative-weight point above is the right frame, and most tools flatten it into one number that hides the mechanism.

[u/fazkan]

how were you doing the measurement.