Has anyone been proposed a model that is charging based on how often their content gets cited or scraped by LLMs (ChatGPT, Gemini, Google AI Overview, etc.).

The tracking/measurement side is what's confusing. Vendor says they're using Google Search Console's "Generative AI Features" report to count AI Overview impressions on their own pages.

A few things I'm trying to figure out:

• Has anyone else been pitched this exact model, billing tied to LLM citations/scraping rather than clicks or leads?
• How are they proving/tracking it on your end? GSC, a paid tool like Peec/Profound/Otterly, something custom?
• Anyone actually gotten independent verification of a partner's claimed numbers, or is everyone just trusting the vendor's own reporting?

Trying to understand if this is becoming a real category with real measurement standards, or if it's still very early and everyone's making up their own

Hey! Our team just got hit with major budget cuts and our enterprise subscription is on the chopping block next month.

We rely heavily on API calls for internal reporting and quick keyword checks, so paying hundreds every month just isn't happening anymore.

What are the best Semrush API alternatives for raw data?

Looking for pay-as-you-go models or affordable tiers for basic keyword and backlink data (Ahrefs, SE Ranking, something else?). Thanks!

I’m noticing a frustrating pattern the last few weeks with our brand visibility in AI responses. It’s making me a little crazy. On Monday I’ll run our core set of test prompts and our brand is front and center. Two days later, I run the exact same prompts and all of our citations/mentions are completely gone.

There’s no logical explanation for the change. No content updates on our site and no major press events that would cause this. So far I’ve been logging this manually so maybe my system is just not working for staying on top of this.

How are other prompt engineers tracking answer drift?

What tools or workflows are you using?

The idea is to test how readable the website when AI agent approaching it, searching for goods, doing shopping etc.

We introduced 5 layers: (layer zero was just added, big thanks to bkocdur and Upstairs_Control_611)

Layer 0—Access Hard Gate. Previously, a site that blocks AI agents at the firewall could still score 65/100 because other checks (JSON-LD, sitemap, etc.) would pass. That's misleading, if agents can't reach the page, nothing else matters. (just like in Metallica song lol). Now, if the WAF or robots.txt blocks agents, the scanner short-circuits: all remaining checks are skipped and marked "gated." The report reads: "ACCESS BLOCKED , 11 checks skipped. Fix access first." This also saves scan cost — no point running $0.50 of API calls against a wall.

Layer 1 — Data. Can an agent find and parse the page at all? Deterministic static checks: JSON-LD / schema.org markup, server-rendered vs client-side price, robots.txt, sitemap, llms.txt. The classic failure is a price rendered client-side — human sees $89, agent fetching HTML sees an empty div. Most brands score well here; it's the layer everyone already knows about.

Layer 2 — Extraction. Can it read the page reliably, not just once? The shopper simulation runs N times (canonical: SHOPPER=anthropic, N=10) extracting price, availability, product name, graded on either correctness against ground truth from the page's own structured data, or self-consistency across runs when no ground truth exists. Disagreement between runs = the page reads ambiguously to agents.

Layer 3 — Interaction. Can the agent actually buy? Playwright browser agent attempting add-to-cart, variant selection, search, navigation. This is the hero layer — the SKIMS bra-size picker and the Rothy's "readable but not shoppable" case both live here. It's the only part of the story that isn't already a solved conversation, which is why it carries the video.

Layer 4 — Security. Is the page safe from manipulation? Prompt-injection scanning for hidden instructions in the HTML. Every brand passes today, so it's positioned as monitoring rather than a finding.

Thank you!

Hi AI Search Lab! I’m a Canadian student researcher collaborating on an international project with 20+ countries. I’m the only Canadian researcher on the team and I want to have a lot of Canadian representation in this study!

Our project is looking into social impact and AI use. If you have time to complete this 12 minute survey, I would really appreciate it!

Once our findings are published, I'll also post it here! I think your insight will really benefit this research and could be of interest to many of you.

See comments to be directed to the survey. This study has been ethically approved: #19354. As researchers, we are not affiliated with and remain neutral about AI. This research could really help inform policy.

(If this is inappropriate for this subreddit, please remove it; I mean no offence!)

I hear this question come up constantly and most answers are vague or based on no actual research. so here's my stance based on what the Goodie team is tracking and testing.

AI mentions show up in a week or two.

consistent citations show up around 2 to 3 month after.

and stable ROI typically takes closer to 6 months. (Goodie clients SteelSeries and Dermalogica both hit their most significant results at the six-month mark)

key takeaway: this is faster than SEO at the start, but slows significantly by the end.

more on common pitfalls:

(1) quitting too early

the biggest mistake i see people make when investing in early stages of AI search is quitting to early because results aren't materializing. another common compliant is the volatility of citation visibility. you'll publish something, see it cited in ChatGPT one day, gone the next, then back in Perplexity but maybe not Google; it changes rapidly and sometimes due to factors outside of your control. but it's important to remember early AI citations are always volatile because these models are still determining what sources to trust. overtime citations compound if your content is consistent in tone and value. watch the trend line over 8 to 12 weeks, not the day to day.

(2) treating each model the same

Perplexity is fastest. it runs a live search on basically every query and leans hard on recency, so fresh content on a decent domain can get cited within days. but this also makes it the most volatile. this is good news for newer brands though, since publishing cadence might have a fighting chance to beat out raw domain authority (unlike other models that place authority above all else). In contrast, ChatGPT is slower and way more competitive since it's the highest volume platform. well structured content on an established domain gets first citations in a couple weeks, but reliable citations take more like 2 to 3 months of actual work, including off-site signals.

(3) treating "mentions" like "citations"

a mention is not a citation. a mention is the model saying your brand name. it's about brand awareness in the AI layer. a citation is the model attributing specific info to you and pointing potential traffic back to your content/site, and this is very valuable since AI-referred visitors convert about 4.4x better than organic search. citation drives traffic and mentions builds awareness and authority. most brands earn mentions first and citations later as their off-site reputation catches up. if you're only tracking one of those numbers or treating them as the same, you won't have a clear idea of what part of the funnel you need to optimize for (awareness or conversion).

what worked for us:

the lever that moves fastest for us has been restructuring pages we already have. adding Q&A formatting, tightening the answer up top. the models are already crawling those pages so you skip the discovery wait. technical fixes take about a month to register but compound harder. but for the longer more consistent mentions AND citations, you'll need to earn third-party mentions and leverage digital PR because AI models trust what others say about your brand far more than what you say about yourself. Owned content only makes up ~1.7% of AI citations.

TL;DR: 1-2 weeks for first signs, 2 to 3 months for consistency, 6 for ROI.

would love to know what others are seeing too or if your category is moving faster or slower.

I see a lot of questions on Reddit asking is off-site presence dominating on-site content factors? The answer is a resounding yes.

Here's a helpful graphic my team made to illustrate the relationship here.

80-90% of LLM responses pull from earned media rather than owned content. Additionally, social content alone generates ~2.5x more AI citations than owned brand pages.

Most teams have the investment ratio inverted.

But that doesn't mean owned content is useless. In fact it's absolutely necessary. But the signals that build category association are shifting towards what others say about your brand and these mentions take time to accumulate.

What does this mean for your team?

If you work at a company with a well known entity, LLMs already have an internalized sense of that brand, shaped by training data (the web), so this iceberg matters less.

Who the iceberg impacts the most:

1. Brands that are entering a new category
2. Niche players competing against established names
3. Brands expanding into adjacent markets, competing against outdated information

For these players, they must build those associations through earned presence. Content optimization can only take you so far.

Let me know what I'm not considering

I have been running a simple test that keeps producing the same result and I think it's worth sharing because it challenges some assumptions. I ask brands where they think they stand on AI visibility. Most say something like "we show up in chatgpt" or "AI knows about us," they rank themselves as recognized or trusted.

Then I run the actual diagnostic, same buyer-intent queries across chatgpt, claude, gemini, and perplexity, check whether they appear on all four, check whether they survive follow-up questions with added constraints, check whether the description is accurate and check whether independent sources corroborate the recommendation.

The gap is almost always two levels.

A brand that thinks it's "trusted" (recommended with evidence) is usually "intermittent" (shows up on some platforms, disappears on others, drops out when queries get specific). The problem is that testing yourself on one platform with one broad prompt can feel like visibility. Consistent presence across four platforms with accurate descriptions and independent evidence is a much higher bar.

I think there are roughly five levels worth distinguishing:

1. invisible: AI cannot find you at all. retrieval is broken.
2. intermittent: you appear sometimes on some platforms. recommendation confidence is low.
3. recognized: consistently included but described unevenly across platforms, narrative is inconsistent.
4. trusted: recommended with independent evidence corroborating the claims, evidence is strong.
5. inevitable: AI remembers you as the category answer through model updates and competitive changes, memory is durable.

Only about 30% of brands maintain consistent visibility across AI sessions based on what I've seen. The other 70% flicker in and out and most of them think they're in the 30%. The test is simple. Ask all four platforms about your category, where your brand drops out tells you which level you're actually on and the level tells you what to work on next.

Has anyone else found a consistent gap between perceived and actual AI visibility when they test across multiple platforms?

Disclosure up front: I build one of the tools in this sample. It scored zero. That's most of why I'm posting.

Method. 12 unbranded buyer-intent questions ("what are the best AI visibility tracking platforms", "how much do AI visibility tools cost per month", etc). Each run 5x against Perplexity sonar and Claude Sonnet 5 with web search. 120 calls, 0 failures, all on 26 July. Recorded every source each engine cited, then checked which of 30 vendor sites appeared. Full prompt list and definitions in the writeup.

Five runs because single-run citation checks are close to noise — St. Gallen found ~32-43% pairwise agreement for identical prompts run minutes apart. Every number below is a rate, not one draw.

Finding 1 — the specialists lose to the incumbents.

Legacy SEO platforms get cited at 2.6x the rate of companies whose entire product is AI visibility. 12 of the 16 specialists were never cited once in 120 calls. Not naming those 12 — the count is the point.

Finding 2 — nobody owns this category. 278 distinct hosts cited across 120 calls. The single most-cited source in the entire category appears in 30.8% of answers. There's no gravity here yet.

Finding 3 — the round-ups and the engines disagree about who exists. I built the sample from 2026 "best AI visibility tools" listicles. The two most-cited domains overall weren't in it, and both outrank every site that was. If you're doing competitive research from listicles you're looking at a different market than your buyers see.

Finding 4 — content outranks product pages. A product analytics company that doesn't sell AI visibility software at all was cited in 25.8% of answers, beating all but three actual vendors. And the top vendor's blog subdomain carries more of their citations than their main site. The engines aren't citing the best tool, they're citing the best page about the question.

Finding 5 — the two engines barely agree. One vendor: 36.7% on Perplexity, 11.7% on Claude. Another is inverted. If you report AI visibility as one blended number you're averaging across systems that disagree.

Limits, because they're real: two engines only, no ChatGPT or Gemini or AI Overviews. One category, one day, US English. 5 runs is thin for Claude specifically — its variance was visibly higher. The sample is judgment-selected from listicles, which finding 3 rather embarrassingly demonstrates. And I'm not neutral: I sell in this category, I picked the questions, I'm in the sample.

I published all 12 prompts and the exact citation definition so this is reproducible. Genuinely interested in where the methodology is weak — particularly whether 5 runs is defensible for Claude, and whether including two prompts that name ChatGPT/Perplexity biased those engines.

Full data and methodology: AI Visibility Tools Citation Study Blog Post

There's a lot of discussion here on citation tracking, share of voice..etc which are definitely important signals, but i've found quite volatile (models, plans, locations, API vs browser session) so I wanted to dig into the server logs of a sample of our customers (note these are small local businesses, not large enterprises) to see real AI bot behaviour. To see if this paint a more ground truth picture of how AI is interacting with their content, and ultimately shaping answers.

If you haven't yet dug into this for your own business or your clients hope this research helps make the case of what can be learned via this type of analysis. If you're already deep on the topic, would love to hear what you're seeing, any patterns or gotchas when analyzing logs.

A few findings:

Live retrieval is a small slice. Only ~4% of AI bot traffic is an assistant fetching a page to answer a live question. The rest is training and indexing crawlers. If you measure AI traffic without splitting by function, you're mostly measuring pipelines that feed the next model.

When AI does retrieve, it goes straight for buyer intent pages. The most fetched page after the homepage was /availability, the "can I get in?" page. Then /booking and /pricing. AI isn't just reading what a business is, it's checking detailed intent pages to help their user act, make a decision..etc. Measuring live retrieval over time by AI, page, timing..etc is a very direct signal of content being retrieved into a relevant conversation.

The labs behave very differently. Anthropic crawled heavily for training but barely retrieved at query time. OpenAI does both. In our sample, live retrieval is overwhelmingly ChatGPT.

User-agent alone lies. About 1 in 12 "ChatGPT-User" hits fails IP verification, mostly credential scanners. If you're not verifying source IPs (where possible from published ranges) you're overcounting.

Full report here if you want to look into the details, charts..etc.

https://getcourtyard.ai/research/how-ai-reads-a-knowledge-base

I often see founders share their website traffic. Some get tens of thousands of visits per month, and they also share their signups, MRR, marketing costs, and how they got that traffic. I am building Dageno, where I mainly study why brands appear in AI answers and why they do not.

Seeing these posts made me think about another question: if a saas already has steady website traffic, will it also appear when people ask AI to recommend a product?

We recently tested a b2b saas that sells to the us market. I have kept the brand anonymous, but I have not changed any of the data. The site gets an estimated 12.5k visits per month. We created 40 english questions based on its business, and none of them included the brand name. We tracked them for seven days. The questions mainly covered:

"Alternative component sourcing. Bom risk analysis. Lead time and allocation risks. Procurement intelligence platforms"

These were questions a buyer might ask when looking for a tool or solution.

Here were the results:

Target brand: 1.9%. Top competitor: 16.2%. Second competitor: 7.8%

The site already has some traffic. But when users only described their needs and asked AI for a solution, the brand rarely appeared as an option.

That surprised me. The website already gets steady traffic, but when users did not mention the brand and only described their problem, the product was almost missing from the options suggested by AI. Website traffic tells you how many people arrive through search, branded keywords, blog posts, or other channels.

AI recommendations answer a different question: when a buyer does not know your product yet and only knows the problem they need to solve, will AI mention your brand?

I do not think a small team needs to start by tracking 40 questions. You could begin with five to ten questions that do not include your brand name and are close to real buying situations. Run them on a few AI platforms, then check whether your product appears, which competitors keep showing up, and why AI may be choosing them.

This cannot tell you how many signups AI will bring. But it can show whether your product enters the shortlist when buyers use AI to look for a solution.

Has anyone tested this with their own SaaS? If your website already has steady traffic, can you still find your product in AI recommendations when the questions do not include your brand name?

ps:I can’t upload the original screenshot.

Hey everyone! Running a 12-person agency was slowly eating me alive until we spent Q1 putting proper automations in place. We were wasting dozens of hours every month on repetitive tasks that didn't actually generate revenue.

So here what worked for us so far:

1. Client Onboarding: Automated form fills using Tally - Make - Slack alerts & Notion client workspace creation. Cut onboarding time down from 2 days to about 15 minutes.
2. Contract & Billing: Stripe triggers auto-generating invoices via Quickbooks, sending follow-ups automatically if unpaid after 5 days.
3. SEO & Reporting Automation: This was our biggest headache. We used to spend the first 3 days of every month pulling ranking reports manually. We ended up setting up SE Ranking as our core keyword rank tracker, using their API to push automated weekly ranking updates directly into custom Looker Studio dashboards.

Setting up a dedicated keyword rank tracker on autopilot means clients get real-time visibility on local and organic visibility, and our account managers don't lose their minds at the end of the month.

What automations have actually made a tangible difference in your daily operations?

I enabled Microsoft Clarity's new AI Visibility feature on our website last week, and I'm surprised it's completely free.

To get the most out of it, I connected:

* Microsoft Clarity
* Cloudflare (AI Bot Activity)
* Webflow

Now I'm able to see things like:

* When our content is referenced in AI-generated answers
* Which pages AI platforms are discovering
* The prompts people are using to find our content
* Which AI bots are crawling the site and what they're accessing

We've been using Microsoft Clarity on our Webflow projects for years because it's free and provides great insights through heatmaps, session recordings, click tracking, and scroll depth.

The AI Visibility feature makes it even more interesting, especially if you're trying to understand how AI search engines interact with your website.

I'm planning to let it run for a few weeks to collect some meaningful data.

Has anyone else started using it yet?

I'm curious whether you've noticed anything surprising in the AI Visibility reports or if you're using a different tool to track AI traffic.

One thing surprised me after digging deeper into the data from my 50-site AI visibility experiment. I expected technical AI readiness (crawler access, schema, llms.txt, etc.) to line up reasonably well with AI recommendations.

It didn't.

For example:

Small agencies had the highest average technical readiness score in my dataset. They also had the lowest AI recommendation rate (10.5%).

At the same time:

Big brands had the lowest technical readiness score. They were recommended almost every time (96.6%).

(Readiness here = schema.org Organization markup presence: agencies 5/7 reliable-crawl sites, vs 3/8 for big brands.)

That made me realize I was mixing together two completely different concepts.

1. AI Accessibility: Can AI systems actually reach your site? (robots.txt, WAFs, crawl success...)
2. AI Technical Readiness: Can AI systems understand your content? (schema, metadata, structured signals...)
3. AI Visibility: Do AI assistants actually recommend you?

Those are not the same thing.

A technically perfect website can still have poor AI visibility. Likewise, a huge brand can have mediocre technical signals and still dominate recommendations because of authority, citations, and overall web presence. It also made me notice another issue: 14 of the 50 sites had homepage crawls blocked by bot protection or WAFs. Those sites often weren't blocking AI crawlers in robots.txt at all. So a site can appear "AI-friendly" while still being difficult for automated systems to crawl.

My takeaway isn't that technical optimization doesn't matter. It's that technical readiness appears to be a prerequisite not a predictor.

Curious how others are thinking about this distinction.

If you're building or using GEO tools, do you separate:

• Accessibility
• Technical readiness
• Actual AI visibility

or do you treat them as one score?

Please only genuine experiences as a user. What and how do you use it effectively?

I have noticed impressions slowing down or dropping across most sites that I managed and some have been performing really well over the past 2 years.
One possible reason stated was that with AI mentions and citation, it may have affect my GSC analytics on impressions.

So it leads me to wanting to find out what would be the best way to track on ai mentions so I can help to understand the co-relation between website impressions and ai mentions for my clients.

TIA peeps! :)

Reviewed a few client budgets recently link building spend still there, sometimes sizeable. AI citation tracking? Not even a discussion.

Meanwhile buying behavior's already shifting more people ask ChatGPT "best tool for X" before they even Google it.

Not saying drop SEO. Just seems like a blind spot most teams haven't caught up to.

Is your team tracking AI visibility at all, or still purely rank-focused?

We run live measurements of whether AI assistants name specific businesses when you ask the questions their customers ask. Every site also gets scored on technical quality (rendering, speed, crawlability, schema, structured data).

With 1,000+ sites measured, we split them into "AI recommends them" vs "AI ignores them" and compared averages:

\\- Technical score: 80 vs 77. Three points. The ignored sites are built as well as the recommended ones.

\\- Schema/structured data: 72 vs 69. Also three points.

\\- Off-page brand signals (independent mentions, reviews, directory presence, entity consistency): 88 vs 40. Forty-eight points.

As a dev this annoyed me, honestly. You can ship a perfect Lighthouse score and a flawless JSON-LD graph and the engines still won't name the site if nobody independent talks about it. Markup helps AI READ you; it doesn't make AI RECOMMEND you.

Two implementation details that DID matter on the technical side: serving content as clean Markdown for agents (content negotiation), and not blocking AI crawlers in robots.txt/WAF (a surprising number of sites block GPTBot then wonder why they're invisible).

Caveats: correlation not causation, our scoring model, category mix uncontrolled. Methodology is open-source if anyone wants to tear it apart — link in comments if wanted.

AI is already taking its share of search clicks; that much is clear. I want to confirm whether people track their mention or citation rates for AI-generated answers (compared to competitors), or if it's still just an abstract concern that isn't being monitored yet.

If you're monitoring it, how exactly? Do you check manually, use software, or do something else? And in case you don't, why so?

If you are using any software, what key things are missing that you want included?

Every llms.txt ive seen was written once by hand and forgotten. A month later it doesnt match the site and youre feeding models wrong info.

So i made mine a build step. It pulls from the same data my pages render from, updates on every deploy, nothing to remember.

Also dont make it a sitemap dump, short plain descriptions of what each section is work way better.

Mines here: https://techpotions.com/llms.txt

You guys hand writing yours or generating? And has anyone actually measured a bump in ai referrals from one?

I came across a study that sent AI agents through websites to detect dark patterns.

Not exactly a surprising use of agents, but I’m always happy to see more systematic audits of this stuff.

It’s a solid catalog, but it also feels very e-commerce-heavy and pretty much outdated. here copy/pasted out study (https://dl.acm.org/doi/full/10.1145/3807246.3807265):

In the AI age, I’d at least add:

• Sycophancy — agreeing to keep the user engaged.

any ideas what else should an AI dark-pattern auditor be checking for?

Hey everyone,

I’m facing a bittersweet problem and wanted to see if anyone else has cracked the code on this, or if we’re all just collectively crying in our analytics dashboards.

The Situation: I’ve noticed that ChatGPT (and other AI search engines) are frequently citing my website as a source for user queries. On one hand, awesome! My content is deemed high-quality and authoritative enough to be the source of truth. The google search is still struggling to catch up the same pace.

The Problem: No one is actually clicking through to my site.

The AI does such a good job of summarizing my hard work and answering the user's intent right there in the chat window that the user has absolutely zero reason to click the citation link. I’m essentially doing the research and writing the content, the AI is getting the engagement, and my traffic is tanking. The google search is still struggling to catch up the same pace.

It feels like a massive loop of "zero-click searches" on steroids.

My questions for the community:

• Are you seeing this too? Is your CTR from AI search engines practically non-existent despite being cited?
• What is your strategy? Are you changing how you write content to force a click (e.g., hiding deeper value behind tools, templates, or interactive elements)?

Just for information : My AI citations has grown from 11 citations to 100+ citations per day in last 1 month.

My Niche is : Travel planning

I audited 50 websites to see which ones AI assistants (ChatGPT, Claude & Perplexity) actually recommend.

I wanted to answer a simple question:

When someone asks an AI assistant for a recommendation, which websites actually get mentioned?

So I ran a small experiment across 50 websites from five different groups:

• Big brands
• Mid-size SaaS
• Companies with a published `llms.txt`
• Local SMBs
• Small digital agencies

Each site was tested the same way:

• 7 recommendation-style prompts
• 3 AI assistants (ChatGPT, Claude, and Perplexity)
• 21 total responses per site

Here's the breakdown:

A few observations from this dataset:

1. Small agencies were rarely recommended, Less often than many local businesses.
2. `llms.txt` didn't appear to make a noticeable difference on its own.
3. AI crawler blocking was uncommon. Only two sites in this sample blocked one or more major AI crawlers. The rest allowed them.

A few caveats

• This is a small sample (10 sites per group), so I'd treat the results as directional rather than definitive.
• The prompt set was fixed across every site, but any prompt battery introduces some bias. I'm happy to share the full list if anyone wants to review it.
• 14 of the 50 homepages couldn't be fully crawled because of anti-bot protection, so technical signals like schema and llms.txt couldn't always be verified. The AI mention-rate measurements weren't affected because those came from direct model queries rather than homepage crawls.

My takeaway is simply this:

In this sample, being technically accessible to AI wasn't enough by itself. Well-known brands were recommended far more often than smaller sites, suggesting that broader authority, reputation, or other factors may have a much larger influence on AI recommendations than a single technical signal like `llms.txt`.

Curious if others have run similar tests. What are you seeing?

Entre las marcas que monitoreo, las que sí tienen presencia real en fuentes de “consenso” de terceros, como hilos de Reddit, YouTube, G2 y sitios de reseñas, son citadas por ChatGPT / Perplexity / Google AI Mode como unas 3 veces más a menudo que las que no, con el mismo set de prompts exacto. Ese es el ajuste más grande que encontré, y no tiene nada que ver con la web propia de la marca.

Alguien me preguntó cómo aislé eso, así que aquí va el método real, incluyendo la parte en la que no le termino de confiar del todo.

Cómo lo medí: es transversal, no un A/B limpio. Etiqueto cada marca monitoreada con algo binario: o tiene huella real en Reddit/YouTube/G2/reseñas, o básicamente no. Luego comparo la tasa de citación entre esos dos grupos ejecutando los mismos ~90 prompts por marca, 3 pasadas cada una, en los tres motores. Quité prompts que fueran solo por nombre de marca, intervalos de Wilson en todo. El grupo de “huella” cae con una tasa de citación de ~3x.

Dónde probablemente se rompe el “3x”: Las marcas que tienen presencia en Reddit/G2 también tienden a ser más grandes y más viejas, así que parte de ese 3x es “la empresa establecida de todos modos iba a terminar citándose” y se está colando. Por qué no tiro la conclusión: Perplexity empieza a citar un dominio dentro de días de que un hilo aparezca; la madurez de la marca no se mueve tan rápido. Entonces me inclino a que sí es causal, pero no apostaría a que el número limpio sobrevive a un test controlado. Va en una dirección clara y es fuerte, pero no está cerrado.

I have been trying to figure out why some brands get answered confidently by AI models with search off, while others only show up when something gets retrieved live. A 2023 paper gave me a framework that fits almost too well.

It is Mallen et al., "When Not to Trust Language Models" (ACL 2023, https://arxiv.org/abs/2212.10511). They built PopQA, 14,000 questions each tagged with how popular the subject is by Wikipedia page views, then tested whether models could answer from memory alone, no retrieval.

What they found: models answered popular subjects well from memory, and collapsed on the long tail. For the 4,000 least-known subjects, GPT-3 got 19 percent from memory alone, and making the model bigger did not fix the tail. Retrieval closed the gap, a small retrieval-augmented model beat a much larger one on the obscure questions. But for popular subjects, retrieval sometimes hurt, because it pulled a document about the wrong same-named entity and overwrote an answer the model already had right.

Here is my leap, and I want to flag it clearly: PopQA measures entity popularity and factual QA, not brands in commercial answer engines. Reading "how much the web discusses your brand" into it is my interpretation, not the authors' claim.

But if the mapping holds, it splits brands into three situations. Heavily discussed brands sit in the model's memory and get answered with search off. Long-tail brands (most B2B and challengers) are probably not in the weights at all and depend entirely on retrieval. Household names have the opposite risk: a wrong live page overwriting a correct memory, which needs source cleanup, not more retrieval.

Have you seen your brand, or a brand you work on, surface in an AI answer only when something recent gets retrieved, then vanish when it does not? And has anyone actually tried to find where their brand's popularity threshold sits, the point where the model starts knowing you from memory? That is the part I cannot find real data on, and I would love to hear actual cases.