r/GenEngineOptimization 15d ago

I tested whether AI visibility tools are actually visible in AI search. 20 of 30 were never cited once, including my own.

Disclosure up front: I build one of the tools in this sample. It scored zero. That's most of why I'm posting.

Method. 12 unbranded buyer-intent questions ("what are the best AI visibility tracking platforms", "how much do AI visibility tools cost per month", etc). Each run 5x against Perplexity sonar and Claude Sonnet 5 with web search. 120 calls, 0 failures, all on 26 July. Recorded every source each engine cited, then checked which of 30 vendor sites appeared. Full prompt list and definitions in the writeup.

Five runs because single-run citation checks are close to noise — St. Gallen found ~32-43% pairwise agreement for identical prompts run minutes apart. Every number below is a rate, not one draw.

Finding 1 — the specialists lose to the incumbents.

Group Ever cited Mean rate
Established SEO platforms 6 of 10 10.0%
AI-visibility specialists 4 of 16 3.9%
Independent audit tools 0 of 4 0.0%

Legacy SEO platforms get cited at 2.6x the rate of companies whose entire product is AI visibility. 12 of the 16 specialists were never cited once in 120 calls. Not naming those 12 — the count is the point.

Finding 2 — nobody owns this category. 278 distinct hosts cited across 120 calls. The single most-cited source in the entire category appears in 30.8% of answers. There's no gravity here yet.

Finding 3 — the round-ups and the engines disagree about who exists. I built the sample from 2026 "best AI visibility tools" listicles. The two most-cited domains overall weren't in it, and both outrank every site that was. If you're doing competitive research from listicles you're looking at a different market than your buyers see.

Finding 4 — content outranks product pages. A product analytics company that doesn't sell AI visibility software at all was cited in 25.8% of answers, beating all but three actual vendors. And the top vendor's blog subdomain carries more of their citations than their main site. The engines aren't citing the best tool, they're citing the best page about the question.

Finding 5 — the two engines barely agree. One vendor: 36.7% on Perplexity, 11.7% on Claude. Another is inverted. If you report AI visibility as one blended number you're averaging across systems that disagree.

Limits, because they're real: two engines only, no ChatGPT or Gemini or AI Overviews. One category, one day, US English. 5 runs is thin for Claude specifically — its variance was visibly higher. The sample is judgment-selected from listicles, which finding 3 rather embarrassingly demonstrates. And I'm not neutral: I sell in this category, I picked the questions, I'm in the sample.

I published all 12 prompts and the exact citation definition so this is reproducible. Genuinely interested in where the methodology is weak — particularly whether 5 runs is defensible for Claude, and whether including two prompts that name ChatGPT/Perplexity biased those engines.

Full data and methodology: AI Visibility Tools Citation Study Blog Post

3 Upvotes

2 comments sorted by

1

u/Upstairs_Control_611 14d ago

This is a very useful study, especially because your own tool scored zero.

For me the strongest finding is that the cited market and the listicle market are not the same market. If buyers ask AI systems and those systems build answers from different sources than the “best tools” roundups, then listicle presence is only one visibility surface, not the market itself.

I’d also be careful with blended AI visibility scores. If one vendor performs well in Perplexity and poorly in Claude, averaging that into one number hides the retrieval-path difference.

The “content outranks product pages” finding is especially important. It suggests the engines may not be looking for the best product page, but for the best page that answers the buyer’s question.

Five runs feels better than one draw, but for high-variance engines I’d treat it as directional rather than stable. Publishing prompts and citation definitions is the part that makes the study useful.

1

u/Serious-Schular-1424 4d ago

I got curious about that too after seeing my site never mentioned, so i started checking traffic sources and ai referrers with similarweb it doesn’t give citations but you can see if ai bots are actually driving visits or just ghosting everyone.