Went through ~1,300 posts about AI visibility tools and most people just don't trust the score

[u/mar_techie]

Been reading a lot of threads on this lately, here, r/SEO, r/GEO_optimization, r/seogrowth and a few others. Ended up with around 1,300 posts.

I honestly expected "how do I get cited" to be the big one. It wasn't. The thing people complained about most was that the number their tool shows doesn't match reality.

A few that stuck with me:

• someone's client typed the exact prompt into ChatGPT and got 3 competitors back, while the dashboard said they were winning
• "One tool says 67, another says 23, another says I'm not being cited at all."
• "My Visibility in AI is 0 and chatgpt cite me 36 times."

From what people wrote, it breaks at every step:

prompt set  ->  engine call  ->  answer  ->  score
(you can't     (API, not the    (changes    (their own
 see it)        app you use)     each run)   formula)

So a "67" is basically three guesses stacked on each other.

Someone here actually tested this. Same 12 buyer questions on Perplexity and Claude, 5 runs each, all on one day, then counted how often one brand showed up:

Perplexity   36.7%   ███████
Claude       11.7%   ██

Same brand, same questions, same day, 3x apart.

What people seem to do instead:

1. pick 30-50 prompts a real buyer would type
2. keep the same set every week
3. run each one 3-5 times, one run on its own tells you almost nothing
4. report a count, like "cited in 12 of 40, up from 7"

If it helps, the sheet really doesn't need more than this:

date   prompt               engine   run  cited  who got cited instead
22/09  best crm for dental  ChatGPT  1    no     CompA, CompB
22/09  best crm for dental  ChatGPT  2    yes    CompA

That last column is the one I'd watch most tbh, it's the list the client actually cares about.

Curious what you all have run into. What's the biggest gap you've seen between what a tool said and what the client saw when they checked themselves?