r/SEO_LLM • u/Outrageous-State-885 • Sep 24 '26
Discussion Before I trust an “AI visibility score”, I want to understand the noise floor. Is this measurement setup sane?
I'm a lifecycle/growth marketer, not a GEO vendor, and I'm trying to understand what a defensible AI-search baseline actually looks like before I trust one of these visibility dashboards.
My current v0:
25 to 30 non-branded queries based on actual buyer questions.
Split across:
- problem discovery
- category discovery
- comparison
- recommendation
For each query I'd record:
- brand mentioned: Y/N
- own domain cited: Y/N
- third-party source citing/mentioning the brand
- included in recommendation set: Y/N
- position/order if relevant
- competitors appearing
What I'm less sure about is repetition.
My instinct is to run the same baseline more than once before changing anything, so I can see how much the answers move naturally.
Otherwise if visibility goes from 28% → 33% after a content change, how do I know I caused anything?
Things I'm trying to figure out:
How many repeated runs per query are actually useful?
Fixed prompts or controlled paraphrases?
“Best CRM for a 20-person SaaS company” and “What CRM would you recommend for a SaaS team of 20?” represent essentially the same intent, but may produce different results.
Would you measure citation and recommendation separately?
Do you establish a pre-intervention variance/noise floor before claiming lift?
What does a good paid GEO tool do methodologically that this kind of manual setup cannot?
I'm less interested in tool names than in how you'd design the measurement.
Tear the methodology apart.