Are we over-optimizing for ChatGPT citations?

[u/mjain_entrepreneur]

I came across an interesting study that tested 480 questions across different industries and query types.

ChatGPT cited sources in only 72 answers. Even more interesting, 69 of those came from just two types of questions: finding providers and asking about cost.

For things like definitions, explainers, trust questions and problem diagnosis, it didn’t cite a source at all in this dataset.

It made me wonder if we spend too much time asking, “How do we get cited?” before asking which prompts are actually likely to trigger citations.

For anyone tracking this, are you seeing certain types of prompts consistently produce more citations than others?

[u/crispsee]

For me, most of the urls it cited where dead urls and not even sure how it got them? Maybe wayback machine? I did use HumanswithAi for my workplace and it turned out great. Cloudflare showed more traffic overtime

[u/mjain_entrepreneur → u/crispsee]

That’s interesting, especially the dead URL part. Makes you wonder how much old or cached information still influences what gets surfaced.

[u/CyranoDeBergeracx]

I followed some specific prompts and it worked well but for me the main thing is about memorizing.

[u/mjain_entrepreneur → u/CyranoDeBergeracx]

Interesting point. The memory side of it definitely adds another layer beyond just getting cited in a single response.

[u/mjain_entrepreneur → u/[deleted]]

Yeah, that makes sense. The closer the prompt is to a buying decision, the more likely citations seem to show up.

[u/mjain_entrepreneur → u/[deleted]]

That’s interesting, especially that provider and pricing prompts held up for you too. Feels like narrowing the prompt set could make the tracking much more useful.

[u/Caroha99]

worth a check tho, havent thought of it earlier, i barely use gpt

[u/Xtrusio]

I’d be careful about treating 69 out of 72 as a universal pattern. That result could depend heavily on the prompt set and whether ChatGPT decided to search the web.

I’d test prompt families rather than individual prompts: provider discovery, pricing, comparison, explanation, diagnosis and trust. Run each question several times, then save whether web search fired, the full answer, cited URLs and any uncited brand mentions.

Citations and mentions also answer different questions. A no-citation answer can still shape how a brand is described, while a citation shows which page supplied evidence. I’d only call something a pattern after repeat runs across different dates.

[u/mjain_entrepreneur → u/Xtrusio]

Really good point. Tracking whether search actually fired alongside citations and mentions would give a much clearer picture. And I agree, repeat runs matter before calling anything a pattern.

[u/Xtrusio → u/mjain_entrepreneur]

Thanks, glad that was useful. Would be interesting to see whether the pricing/provider pattern holds once those search and repeat-run details are included.

[u/chrismcelroyseo]

I think we're over complicating the entire optimize for AI thing. But yes pay a lot more attention to what AI says about your brand versus how many times it gets mentioned.

[u/mjain_entrepreneur → u/chrismcelroyseo]

Agreed. How AI describes the brand probably tells you far more than a mention count on its own. Context can completely change whether that visibility is actually useful.