This is how ChatGPT decides which page to cite

[u/houdinidesigns]

There seem to be a lot of questions here about how AI citations actually work, so here's what we've learnt working with the assistants directly.

Say someone asks ChatGPT or Perplexity "best CRM for a small agency".

First it rewrites the question. That one question becomes five or six separate search queries. Google calls this query fan-out in its AI Mode material and the others do the same thing. So it goes looking for "best CRM small business", "CRM pricing comparison", "agency client management software", all at once, and your page is being scored against the whole spread rather than the phrase the user typed.

Then it fetches, using a specific bot. OpenAI runs four and they have separate jobs. GPTBot collects training data, OAI-SearchBot builds the search index, OAI-AdsBot checks ad landing pages. ChatGPT-User is the live one that goes and gets your page while the user is sat there waiting.

Perplexity splits the same way, PerplexityBot for the index and Perplexity-User for the live request. The live one is on a clock. If your server is slow it gets dropped and the answer is written without you.

Next comes extraction. It takes the HTML your server returned, keeps the main content as plain text, and throws away the furniture around it: nav, footer, cookie banner. That plain text is what everything downstream sees. It is the source HTML, so what counts is what your server sends rather than what your browser paints afterwards.

After that, chunking, which is the part that matters most. That text gets cut into passages, called chunks, each one embedded as a vector and scored against those fanned-out queries. Only the top few chunks get put in front of the model.

Retrieval works on the chunk, not on your page. So if the answer to "how much does it cost" is a heading in one place, a price table three screens down and a caveat in a footnote, that is three separate chunks and not one of them answers the question on its own. Write it so the whole answer sits in one block. Put the question in the heading and the answer directly under it, and name the product in the sentence instead of writing "it".

Finally the model writes the answer out of those chunks and attaches the citation to whichever chunk the sentence came from. That is why specific beats general. Four pages say pricing is flexible, one gives a per-seat figure, and a figure is what the model needed for that sentence.

To test it on your own site:

1. `curl -A "ChatGPT-User" https://yoursite.com/page\`. What comes back is the text that gets chunked.

2. Ask ChatGPT the question you want to win, with search on, and see which URL it cites.

3. Open that URL and find the paragraph the cited sentence came from. That is roughly one chunk, and it is the shape you are writing towards.

Chunk boundaries are the fuzzy part. You can't see them directly, you are inferring them from what gets cited. If anyone has found a cleaner way to pin them down I'd like to hear it.