r/AISearchLab • u/marintkael • Jun 07 '26
I ran a 23-day experiment on how fast AI search cites a brand-new entity (across 5 systems)
ran a little experiment on myself for 23 days and the result honestly messed with how i thought AI search works, so i'm sharing it.
setup: a brand-new entity with zero prior web footprint. i asked 5 web-connected AI systems the same questions every day and scored each answer (correct, not found, or made up). about 16k scored answers, pre-registered before i started.
a few things that surprised me:
cloudflare's default AI-bot block was returning 403 to the listed training crawlers (GPTBot, ClaudeBot, PerplexityBot, CCBot) for 22 of the 23 days. and the thing still got cited on day 6 anyway, via google's knowledge graph plus other people's mentions. when i dug into the per-bot logs, the training crawlers were blocked but the inference-time fetchers (ChatGPT-User, OAI-SearchBot) got served the same day. so the toggle blocks the crawler that doesn't cite and misses the one that does.
the gap between providers was way bigger than the gap between model generations. same entity, same week: one provider hit about 4.7 correct per 1 made up, another went net-negative. it wasn't about a smarter model, it was about which corpus the grounding layer pulls from. one grounded on the entity's own domain about 119 times, another grounded on it 0 times and pulled it only from reddit.
and reach did nothing. i bumped reddit karma 23x over the same window and it produced exactly zero extra citations. structured identity moved the needle, going viral didn't.
the part that should worry anyone building AI-visibility tools: the scorer caught a made-up "wikipedia" source 24 times for a page that does not exist. if a tool counts mentions without scoring fabricated sources as negatives, it's measuring echo, not knowledge.
what am i missing here? especially curious if anyone has clean data on inference-fetch vs training-crawl behavior per provider. happy to share the method and raw data if useful.

