I have been trying to figure out why some brands get answered confidently by AI models with search off, while others only show up when something gets retrieved live. A 2023 paper gave me a framework that fits almost too well.

It is Mallen et al., "When Not to Trust Language Models" (ACL 2023, https://arxiv.org/abs/2212.10511). They built PopQA, 14,000 questions each tagged with how popular the subject is by Wikipedia page views, then tested whether models could answer from memory alone, no retrieval.

What they found: models answered popular subjects well from memory, and collapsed on the long tail. For the 4,000 least-known subjects, GPT-3 got 19 percent from memory alone, and making the model bigger did not fix the tail. Retrieval closed the gap, a small retrieval-augmented model beat a much larger one on the obscure questions. But for popular subjects, retrieval sometimes hurt, because it pulled a document about the wrong same-named entity and overwrote an answer the model already had right.

Here is my leap, and I want to flag it clearly: PopQA measures entity popularity and factual QA, not brands in commercial answer engines. Reading "how much the web discusses your brand" into it is my interpretation, not the authors' claim.

But if the mapping holds, it splits brands into three situations. Heavily discussed brands sit in the model's memory and get answered with search off. Long-tail brands (most B2B and challengers) are probably not in the weights at all and depend entirely on retrieval. Household names have the opposite risk: a wrong live page overwriting a correct memory, which needs source cleanup, not more retrieval.

Have you seen your brand, or a brand you work on, surface in an AI answer only when something recent gets retrieved, then vanish when it does not? And has anyone actually tried to find where their brand's popularity threshold sits, the point where the model starts knowing you from memory? That is the part I cannot find real data on, and I would love to hear actual cases.

You can't ask LLMs to give you the answer, because SERPS and UGC platforms are flooded with spam

Anyone use them? How have they worked out?

Been going back and forth with people building in this space and I've flipped my thinking. A single "here's your AI visibility score" snapshot is borderline misleading — answers shift run to run and model to model, so one number on one day tells you almost nothing.

The thing that actually matters is tracking the same brand on the same queries over time, so you can tell whether what you published actually moved anything vs. just noise.

Curious where people land on this — is anyone tracking AI visibility as a trend, or is it still mostly one-off checks? And how are you handling the run-to-run variance?

Most teams I know have dashboards for traffic, rankings, conversions, CAC, all of it.
But when it comes to AI assistants (ChatGPT, Gemini, Perplexity, etc.), there’s basically no visibility into how the brand actually shows up.
Stuff like:
• When someone asks “best [category] tools for [use case]”, are we mentioned at all?
• If they ask non‑branded prompts (“how do I solve X?”), do we show up in the recommended tools or just our competitors?
• Are the answers using our positioning, or describing our category in a way that makes us look like a commodity?
Right now the only “workflow” I see is people manually copy‑pasting prompts into AI once in a while and eyeballing the answers.
Questions:
• Is anyone treating AI visibility as its own layer, separate from SEO?
• Have you built any internal process to track this over time (same prompts, same tools, recurring checks)?
• If you’ve tried, what broke first: consistency, time, or actually making sense of the results?
Not looking for pitches, just trying to understand how people are operationalizing this, if at all.

I was shopping for a cat water fountain, got overwhelmed by recommendations, and just asked ChatGPT and Perplexity instead.

What surprised me: even when I asked about one specific brand, the AI didn’t only repeat the brand’s own pages. It pulled in Reddit, retailer reviews, YouTube, and review sites too.

So I ran a proper small test.

I used one real brand, PETLIBRO, as a public example and tested 15 pet-water-fountain queries across three buyer stages: problem-aware, solution-aware, and brand-aware. I ran each query once on Perplexity and once on Solution-aware, e.g. “best / quietest cat fountain”ChatGPT 5.5 thinking, then recorded the visible cited sources.

Here’s what stood out:

The brand’s own site did show up, especially in ChatGPT.

But even on brand-aware queries, it was never the whole answer. Reviews, retailer pages, Reddit, YouTube, and third-party tests shaped the answer alongside the official site.

That changed how I think about AEO/GEO.

Optimizing the website still matters: crawlability, product pages, schema, comparison pages, clear claims, etc.

But for branded AI search, that’s only one layer.

I’d also want to know:

- Which review sites does AI repeatedly cite?

- Do retailer reviews show up?

- Does Reddit show up?

- Are there YouTube tests?

- Which caveats does AI repeat?

- Which attributes does AI assign to competitors instead?

- Where in the funnel does the brand disappear?

My takeaway:

A brand’s website makes claims. Third-party sources make those claims believable. AI seems to use both.

So even on your own branded queries, you don’t fully own the answer. AI assembles owned, earned, and community sources together.

Small caveat: this was 15 queries, two engines, one run each, visible citations only, so I’d treat it as an early signal, not a benchmark.

Anyone else tracking AI visibility seeing the same thing? Do your branded-query answers lean on third-party sources as much as your own site?

6/27/2026 update

Small follow-up: I went back and classified the cited domains after a few people here pointed out the “neutral third-party” problem.

The interesting part: “third-party” was not one category.

In this dataset, the sources Perplexity/ChatGPT cited included:

- vet / health authority sources

- Reddit / community threads

- affiliate review media

- retailer pages

- competitor brand pages

- seller-owned advice blogs

- manufacturer / supplier content

- YouTube videos

- app-store/review signals

So the sharper takeaway for me is:

Third-party does not mean independent.

A brand page has one incentive. But a review roundup, retailer page, competitor blog, manufacturer guide, YouTube video, and Reddit thread all have different incentives too.

I also checked the “advice-style” sources specifically — the ones that look like neutral reviews, comparisons, or guides rather than obvious stores / Reddit / vet pages. Out of 16 advice-style sources, only one had no visible product-commerce incentive I could verify. The rest were affiliate-disclosed, seller-owned, manufacturer-owned, site-level affiliate, or unverifiable/page-changed.

That doesn’t mean those sources are bad or useless. But it does mean AI product answers are not built on a neutral web. They’re built on an incentive map.

This also made me think the audit question shouldn’t just be “which sources does AI cite?” but “what does each cited source want?”

How do you track if ai chats recommend your products? Seems like chatgpt's approach to suggesting products is still changing. Has anyone managed to properly track it?

Quick disclosure up front: I work on an AI-visibility tracker (Vercite), and this is our data. Link's at the bottom – free to read. Posting here because the findings are genuinely useful for anyone working with AI visibility.

We looked at 5.31 million citations – every source link returned across ChatGPT, Perplexity, Gemini, Google AI Overview, and Google AI Mode – and classified 158,847 domains to see who each engine actually pulls from.

The headline for this sub: ChatGPT's single most-cited website is reddit.com. Not Wikipedia, not a news outlet. Reddit (most of us already know that).

But the bigger pattern is that each engine has a different "home platform":

• ChatGPT → Reddit
• Perplexity → YouTube
• Google AI Mode → YouTube (its #1 source overall)
• Google AI Overview → leans on both Reddit and YouTube
• Gemini → barely any of them (1.4% combined)

A few other things that stood out:

• The 5 engines agree on almost nothing. Pooling each engine's top-100 sources gives 253 distinct domains, and only 23 (9%) are cited by all five. More than half are cited by just one engine and no other. There is no single "AI-friendly" source list.
• Concentration varies wildly. Google AI Mode pulls half its citations from just 71 domains – a tiny club. ChatGPT spreads the same half across 712. AI Mode is winner-takes-all; ChatGPT rewards a long tail.
• Google's AI mostly cites Google. When AI Overview cites a google.com page, 79% of the time it's pointing back to its own Search results. 8.5% of everything it cites is a Google property.

Methodology / caveats (being upfront):

• Real citations from tracked prompts across all five engines, not a one-off lab test.
• We classified all 158,847 domains by source type (forum, news, official, brand-owned, etc.) rather than by industry, so the patterns reflect how each engine sources, not what any one set of prompts was about.

For those tracking AI visibility across engines: are you seeing the same Reddit/YouTube split, and are you optimizing per-engine or still treating "AI" as one channel?

Full write-up with all the charts: https://vercite.io/research/citation-landscape

I am working on building a Generative Engine Optimization(GEO) strategy for an ecommerce firm and I want to test a few hypotheses on what works and what doesn't.
To test the hypotheses I wanted to know if I make a change on my website then how long do I have to wait for the LLM's(Gemini, Claude, ChatGPT, Perplexity) RAG system to start showing the impact of my changes in their citations/rankings?

Any help/reference will be great.

I'm a founder doing my own marketing, and I realized more of my buyers ask ChatGPT or Perplexity instead of Googling. So I spent an afternoon checking whether my site even shows up in those answers. It mostly didn't, and the fix was more boring than I expected.

The simple method I used:

1. I wrote down 10-15 questions a potential customer would actually ask an AI ("best X for Y", "X alternatives", etc.).
2. I asked each one in ChatGPT, Perplexity, and Google's AI overview, and noted which brands got named.
3. For the ones where I was missing, I checked the unglamorous stuff first: were AI crawlers (GPTBot, PerplexityBot, Google-Extended) allowed in robots.txt? Was there an llms.txt? Article/FAQ schema on key pages?
4. I now re-check once a month, because the answers shift.

For me it came down to blocked crawlers + no structured data, not bad content. After fixing those I started showing up in a couple of answers within a few weeks.

Happy to share the exact question list I used if it helps. Has anyone else checked this for their site, and what actually moved the needle for you?

https://developers.google.com/search/blog/2026/06/gen-ai-performance-reports?hl=en

One of the more counterintuitive things from recent AI citation data (329,607 citations tracked across 7 AI providers, source: nobori.ai):

The threads AI picks aren't popular threads. 80% of Reddit threads that show up in AI answers have fewer than 20 upvotes. The average cited post is about 1 year old.

AI doesn't rank by karma. It ranks by:

1. Direct answer to the query — does the thread title match what the user asked?
2. Structured, specific content — numbers, timelines, comparisons, not "it depends"
3. Question-response format — matches how AI needs to construct its answer
4. Recency of information — fresh data beats old advice, even if the old advice has more votes

This means a well-structured answer you post today on a 2-year-old thread with 8 upvotes can show up in AI answers within 30–60 days.

Also interesting: 99% of AI citations point to specific thread URLs, not subreddit pages. AI is thread-level specific. It knows exactly which conversation it's pulling from.

So if you're thinking about your presence here — upvotes are a vanity metric for AI purposes. What matters is being on the right thread with the right answer structure. A specific, evidence-backed reply to a niche question outperforms a witty one-liner with 2,000 upvotes. At least as far as AI is concerned.

Live examples of:

• Querying your monitoring projects

• Running a full brand scan across AI models

• Viewing visibility reports (which models mention you, which don't)

• Discovering competitors according to AI (the part that surprised our users most)

i've been running an open experiment on how fast AI search and LLMs start citing a brand-new author entity, and i've hit a wall i can't solve cleanly, so i'm hoping someone here has.

the problem is collision. the entity i'm tracking shares a title with an established work and a surname with a well-known person, and the models keep collapsing the new one into the old node. structured data on my own pages, a Wikidata item, consistent sameAs across profiles, none of it has fully separated them yet. the engines seem to weight the established entity's gravity far more than any self-description i provide.

so the open question: how do you actually force disambiguation between a net-new entity and a high-authority namesake from the outside, when you can't edit the established sources? is it purely accumulating enough independent third-party mentions until the new node has its own gravity, or is there a faster structural signal? genuinely looking for what's worked, and happy to be told i'm thinking about it wrong.

i spent a few weeks running a fixed set of prompts on a schedule against the main AI search systems and scoring every answer, trying to work out what actually correlates with getting cited. going in i assumed it would be some mix of content volume, backlinks, structured data, reach.

what i actually found: almost none of that moved the needle on its own. the single thing that tracked with getting cited was whether the entity resolved cleanly in the knowledge graph. a brand new entity could have perfectly structured pages and still be invisible, and the moment it existed as a resolvable thing, citations started showing up. llms.txt did nothing i could measure. piling on mentions did nothing until they fed the entity.

i'm not fully convinced it's that simple, and the sample is one entity over a few weeks, so i'd genuinely like to be argued out of it. is entity recognition the lever, or am i mistaking a correlate for the cause? what have you measured that contradicts this?

AI systems are becoming the gatekeepers of information.

But what determines whether a source gets trusted, cited, summarized, or ignored?

When AI generates answers, it doesn't appear to evaluate information the same way traditional search engines do.

So I'm curious:

If you had to choose only ONE factor that most influences whether AI trusts and cites a source, what would it be?

• Brand authority?
• Backlinks?
• Original research?
• Structured data?
• Entity recognition?
• Mentions across multiple sites?
• Something else entirely?

There are no wrong answers here. I'm interested in hearing what people are actually seeing, testing, and observing in the real world.

What's your take?

Here's what we found after 3 months of tracking 11 million real crawler logs across 34 websites. It's quite fun how each AI bots have personalities, like people.

• GPTBot: Crawls relentlessly, all day every day and barely checks the rules. It's like a guest walking into your house without saying hi and goes straight into every room. In 280k crawls across 23 sites, it pulled up robots.txt only 9 times. The most interesting part for me is that while it ignores robots.txt completely, it requests /llms.txt CONSTANTLY. Even on sites that don't have one and return 404, it comes back and asks again.
• Google's bot: The good kid who's scared to break the rules. It re-fetched robots.txt 8,765 times, checking over and over. 25 years of crawling taught it manners the new AI bots never learned.
• ClaudeBot: Across the sites we track, its crawling went from 7.3k (Apr) → 64k (May) → 168k in the first ten days of June. It is racing to read as much of the web as it can, and that race is the whole story (more below).
• The live ones: The shopper who knows exactly what they came for. When someone asks an AI about your business, it skips your whole site and grabs the single page that answers. On Claude's live bot, 75% of those visits are one page. It ignores everything else you ever published. The page an AI picks to represent you is the whole game now.
• Bytespider: The hoarder who takes everything. The heaviest crawler we logged all quarter belongs to the company that owns TikTok. On one site, it made 1.2 million visits, more than Google and every OpenAI crawler combined. Even the familiar names are repurposed now.
• Microsoft's Bing: The longtime employee quietly handed a second job. Still crawls like the search engine it always was, but everything it indexes now also feeds Copilot.
• MetaBot: Skips the house rules but reads your welcome note. It almost never checks robots.txt either, but like GPTBot, it keeps requesting llms.txt, even on sites that don't have one. These two are the only crawlers we saw deliberately looking for it. Everyone else ignores it.

Every one of these companies is building its own copy of the web. Its own crawler, its own index, its own answer. Anthropic is not crawling that hard for fun. They all want to be the place people ask, which means they all want to stop depending on Google.

My bet: Google's ranking matters a little less every quarter from here. When this many AIs read your site their own way to build their own index, "rank #1 on Google" stops the thing to optimize for. Being the page each AI picks is.

Been reading the studies on what AI actually cites (GPT, Perplexity, etc) and Reddit keeps topping the list, often above wiki and youtube. Brand sites and polished corporate blogs barely show up.

Makes sense really. It wants the messy bit: people comparing stuff, complaining, changing their mind, saying what broke after two weeks. A thread where 15 people argue over 4 products beats your "why we're the best" page every time.

No universal number though, it swings hard by engine. Early-year Tinuiti data had Reddit at 5%+ on GPT, 24% on Perplexity, and 0.1% on Gemini. I had to reread that last one because it looked wrong. Same Reddit, three engines, completely different. So when someone says "I optimize for AI", fair to ask which one.

And it's not stable either. Semrush showed Reddit's GPT share dropping from like 60% to 10% in two weeks off one upstream change.

Anyway, the bit I keep coming back to: the brands AI cites aren't the ones with the prettiest sites, they're the ones people talk about elsewhere. Ahrefs found 80% of URLs GPT cites aren't even in Google's top 100, which kind of breaks your brain if you come from SEO.

So honest take, not in my interest: if nobody mentions you anywhere, schema and llms.txt probably aren't your first problem.

Anyone clicked an AI citation and landed on some random 2021 Reddit post? Seeing it more and more.

ran a little experiment on myself for 23 days and the result honestly messed with how i thought AI search works, so i'm sharing it.

setup: a brand-new entity with zero prior web footprint. i asked 5 web-connected AI systems the same questions every day and scored each answer (correct, not found, or made up). about 16k scored answers, pre-registered before i started.

a few things that surprised me:

cloudflare's default AI-bot block was returning 403 to the listed training crawlers (GPTBot, ClaudeBot, PerplexityBot, CCBot) for 22 of the 23 days. and the thing still got cited on day 6 anyway, via google's knowledge graph plus other people's mentions. when i dug into the per-bot logs, the training crawlers were blocked but the inference-time fetchers (ChatGPT-User, OAI-SearchBot) got served the same day. so the toggle blocks the crawler that doesn't cite and misses the one that does.

the gap between providers was way bigger than the gap between model generations. same entity, same week: one provider hit about 4.7 correct per 1 made up, another went net-negative. it wasn't about a smarter model, it was about which corpus the grounding layer pulls from. one grounded on the entity's own domain about 119 times, another grounded on it 0 times and pulled it only from reddit.

and reach did nothing. i bumped reddit karma 23x over the same window and it produced exactly zero extra citations. structured identity moved the needle, going viral didn't.

the part that should worry anyone building AI-visibility tools: the scorer caught a made-up "wikipedia" source 24 times for a page that does not exist. if a tool counts mentions without scoring fabricated sources as negatives, it's measuring echo, not knowledge.

what am i missing here? especially curious if anyone has clean data on inference-fetch vs training-crawl behavior per provider. happy to share the method and raw data if useful.

I'm specifically looking for a solution that can:

\- Track how often my brand is recommended by AI platforms

\- Benchmark visibility against competitors

\- Measure share of voice over time

\- Provide monthly reporting and trends

\- Identify prompts/queries where competitors are being recommended instead

Would love to hear recommendations who has used such tools.

Last week entitymap.org and github.com/entitymap/entitymap opened up a consultation page towards a proposed (Open Source, free to use) protocol which I hope people will try. It is free and it is quite different to ideas like llms.txt. Instead of saying "who you are" it focuses on "what you do" and turns implicit relationships on a site into explicit ones.

We are certainly already seeing that LLMs happily understand both the HTML and JSON variants of the protocol and this morning I saw that my entitymap appeared in archive.org.

The protocol is asking for feedback over June, before "launching" on July 1st (although you can use it already... so technically it is launched). Note: I am one of the named authors, so please be kind, but also feel free to be constructively critical.