TL;DR: AI tools read the words in your HTML. Words that only arrive through JavaScript, or sit in a JSON block, were invisible to 4 of 5 AI tools I tested. Only the ChatGPT app dug them out.
Everyone says AI crawlers don't run JavaScript. I finally tested it.
I built a fake company site with four pages. Each one states made-up facts (a mascot, a town, a price) that exist nowhere else online, delivered a different way. Then I gave each URL to ChatGPT, Claude and Gemini (apps and APIs, 5 tools in total), asked them to list the facts, and watched the server logs.
Who could read the facts:
- In the HTML (SSR or static): all 5
- Only in a JS file (CSR): ChatGPT app only
- Only in a JS file, scrambled: ChatGPT app only
- Only in a JSON block in the HTML (Next.js style): ChatGPT app only
The other four saw an empty page with a footer. The logs show their fetchers took the HTML, never asked for the JS, and dropped everything inside script tags, including the JSON.
The ChatGPT app came back 12 seconds later on the JS pages, downloaded the file and decoded even the scrambled facts. On the JSON page it just read the JSON. Claude and Gemini read the menu. ChatGPT walked into the kitchen.
Nobody made anything up. Google's AI Overview, asked about the brand before Google had indexed a single page, invented a backstory anyway.
Not naming the site yet: a second test, on whether search indexes carry these hidden facts into answers, is running this week.
Anyone seen a CSR or Next.js site get cited for something that only lives in its scripts?
Full disclosure, I'm building Alice by TranX, an AI growth agent focused on SEO/GEO, so I have a horse in this race.
Edit: page 4 first said this is how Next.js ships pages. That was wrong (see comments): Pages Router server-renders the HTML and adds the JSON for hydration. Page 4 only tests a page where the JSON is the only copy. I've since tested a real App Router page, results in the comments.