The machine writing about your landing page cannot run your landing page

Claude spent a bigger share of its fetches on JavaScript than Googlebot, and ran none of it

Vercel names the bots, which is the part worth copying out: OpenAI's OAI-SearchBot, ChatGPT-User and GPTBot, Anthropic's ClaudeBot, Meta-ExternalAgent, ByteDance's Bytespider and Perplexity's PerplexityBot. It publishes a JavaScript share for only three of them. Claude spent 23.84 per cent of its requests on JavaScript files and ChatGPT 11.50 per cent, and neither executed a line of it. Googlebot's share was 15.25 per cent, in the same window, on the same network, and Googlebot runs what it downloads.

The content mix says what each one is there for. ChatGPT's fetches are 57.70 per cent HTML. Claude's are 35.17 per cent images. Googlebot spreads itself over 31.00 per cent HTML, 29.34 per cent JSON and 20.77 per cent plain text, which is a crawler that has learned where a modern site keeps its data. The answer engines are reading the document, and the document is whatever came back in the first response.

Where that claim comes from matters, because the AI search genre usually attributes it to the wrong place. Anthropic's crawler page names ClaudeBot, Claude-User and Claude-SearchBot, says its bots honour robots.txt and publishes an IP list so you can verify them. OpenAI's page names GPTBot for training, OAI-SearchBot for search inside ChatGPT, and ChatGPT-User for a page visited because somebody asked a question. Neither page says whether its bots run JavaScript. The vendors do not document the behaviour. It was measured in logs, by one host, in December 2024.

The same genre will tell you Google-Extended is Gemini's crawler and that it renders JavaScript because it inherits Googlebot's plumbing. Google's own crawler page, updated 14 July 2026, says Google-Extended has no separate HTTP request user agent string, that crawling is done with existing Google user agent strings, and that the robots.txt token is used in a control capacity. It is a permission switch over content already crawled. It fetches nothing, so there is nothing for it to render.

Nearly a quarter of Claude's requests were JavaScript files it had no intention of running.

A third of those fetches came back with nothing, and for these platforms the fetch is the visit

The miss rate should worry an operator more than the rendering does. 34.82 per cent of ChatGPT's fetches hit a 404 and another 14.36 per cent hit a redirect. Claude: 34.16 per cent on 404s. Googlebot over the same period: 8.22 per cent and 1.49 per cent. Four times the 404 rate and nearly ten times the redirect rate, the two failure modes that decide whether a URL yields anything at all.

What the number does not license is the leap to a third of your pages being missing from ChatGPT. Vercel names one cause and gives it no arithmetic: these crawlers frequently attempt to fetch outdated assets from the static folder. How much of the third that accounts for is not in the study. What survives is narrower and still bad: a third of these crawlers' fetches returned nothing.

The other half of the 404 story sits on Google's own troubleshooting page, updated 18 December 2025: a single page app handling its errors in client script often reports a 200 status code instead of the appropriate one. A crawler that cannot run that script receives a 200 and an empty shell. Nothing looks broken in either log.

Cloudflare puts the commercial number on it twice, and the two numbers disagree. In the week of 19 to 26 June 2025, Anthropic made nearly 71,000 HTML page requests for every HTML page referral it sent back, a ratio Cloudflare states as 70,900 to 1. Measured again for the week of 1 August 2025, the same ratio is 50,000 to 1. Both weeks carry Cloudflare's caveat that native app traffic sends no referral header, so both may be overstated by an unknown margin. Take the magnitude: the fetch is the visit, and no human arrives later to see the rendered version.

Googlebot rendered every HTML page in the sample, and Google still tells you to pre-render

The split is not that search engines cannot handle JavaScript. Vercel and MERJ instrumented over 100,000 Googlebot fetches on nextjs.org, with monogram.io and basement.io alongside, between 1 and 30 April 2024, and matched over 37,000 rendered pages against server beacons. 100 per cent of HTML pages resulted in full page renders, including pages with complex JavaScript interactions, and links were discovered in fully rendered pages regardless of the rendering method.

Rendering everything is not rendering promptly. The same sample puts the median render delay at 10 seconds, the 75th percentile at 26 seconds, the 90th at roughly 3 hours and the 99th at roughly 18 hours. Pages carrying a ref parameter waited longer still. If a change to your page exists only in the rendered copy, that queue is your publishing schedule.

The JavaScript SEO basics page, last updated 4 March 2026, still carries the sentence the whole pre-rendering argument rests on: server side or pre-rendering is still a great idea because it makes your website faster for users and crawlers, and not all bots can run JavaScript. That is the one crawler that does render, telling you to serve rendered HTML, and naming the other bots as the reason.

One correction to the old playbook. Sniffing the user agent and serving a rendered copy to bots, the trick the JavaScript SEO era ran on, is disavowed by Google in the past tense on a page updated 10 December 2025: dynamic rendering was a workaround and not a recommended solution, because it creates additional complexities and resource requirements. The alternatives Google names instead are server side rendering, static rendering and hydration.

The only crawler that runs your JavaScript is the one telling you not to depend on it.

The median page loses a sixth of its words to a non-rendering fetch, not all of them

The HTTP Archive Web Almanac's 2024 SEO chapter counted the words on a page twice, once in the raw HTML and once in the rendered DOM. A median home page carries 330 words raw and 400 rendered on desktop, 311 and 364 on mobile. The chapter calls that a surprisingly small discrepancy: 17.5 per cent on desktop, 13.6 per cent on mobile. Inner pages are tighter still, 284 raw against 333 rendered. So the catastrophic version of this argument, the one where the machine sees nothing, does not hold at the median.

A page that assembles its copy in the browser loses all of it, not a sixth, and no median will tell you which of the two you own. The Almanac also shows what does not move: rendering changed the canonical tag on 0.6 per cent of desktop pages and 0.5 per cent of mobile ones, and a robots directive on 0.4 and 0.3 per cent. The plumbing survives the trip. The copy is what goes missing.

One limit on those medians, stated because it changes how hard you can lean on them: the chapter does not say how many pages are behind its numbers. They are medians without a published denominator, which is a different weakness from Vercel's, whose sample is weighted to one site on one host's network and dated to a single December. Two imperfect measurements pointing the same way is what the evidence here amounts to, and quoting the tail as the middle is how this subject gets sold.

The markup Google blesses is markup a non-rendering crawler cannot read

Google's page on generating structured data with JavaScript, updated 10 December 2025, says Google Search can understand and process structured data available in the DOM when it renders the page, and it treats Google Tag Manager as a supported way to inject JSON-LD. The only warning it attaches is about frequency, for Product markup: dynamically generated markup can make Shopping crawls less frequent and less reliable, which matters for fast changing content like availability and price.

A practice Google documents and supports is a practice that hides your structured data from every AI crawler in the measurement. The earlier post here on video indexing lists what a VideoObject has to carry. Inject that block client side and every property in it exists for exactly one reader.

The test takes a minute. Fetch your own URL with curl, which runs no scripts, and read what comes back. For the Googlebot half, Google's advice is the URL Inspection tool or the Rich Results Test, to confirm the rendered HTML holds all the content you expect. For everything else there is no tool, because there is no rendering step to inspect. The first response is the whole document, and if the argument for your product is not in it, no answer engine has ever read it.

What to do about it

1. Fetch your ten most important URLs without a browser and read what arrives. If the argument for the product is not in that response, no answer engine has it.
2. Move the JSON-LD out of the tag manager and into the server response, and keep it there even though Google reads it either way.
3. Make missing routes answer with a real status code from the server, because a client rendered 200 reports success to a crawler that cannot see the error.