How often does the rendered HTML differ? Over 200 domains show it

Question and method

Every page has two versions. The source code is the HTML the server delivers. The rendered HTML is created after the browser has run JavaScript. Our experiments with Google Tag Manager also suggested that Google indexes based on the rendered HTML. If the two versions differ, a crawler without rendering sees a different page from Google.

For the study, we crawled over 200 domains. For each domain, we included the homepage and all URLs linked from the homepage. For every page, we compared source code and rendered HTML in the SEO-relevant areas, that is text, internal links, title and meta tags such as canonical and robots.

The results in numbers

Almost every domain was affected, but by no means every page. On just over a third of the domains, the homepage looked the same in both versions. If you only check the homepage, you miss these cases.

Internal links

Almost all crawled pages had additional links in the rendered HTML. On some important pages, almost 3,000 links per subpage were added. Here, presumably the entire main navigation exists only in the rendered HTML. If Google ignored these links, large parts of the website would be missing from the index. According to the documentation, every page with status 200 goes into the render queue, unless there is a noindex in the source code or in the HTTP header.

We noticed three patterns in which links get lost for Google or send an unclear signal.

Whether these structures were intended could not be told from the outside. For optimization, what counts is that you know about them. So check links in the rendered HTML before any interaction, or use a crawler that renders.

Additional content

Many domains loaded additional text in the rendered HTML. Often this was the consent banner. On a weather portal, it added a large amount of privacy text to the page. If Google does not cleanly recognize something like this as boilerplate, it can dilute the actual content. In the cases we looked at more closely, relevant content loaded later was indexed by Google.

Title, canonical and robots

A food discounter optimized its titles via JavaScript. Google showed the title from the rendered HTML in the search result, Bing the one from the source code. If you set titles via JavaScript, you should therefore expect other search engines to use the source code. On very large websites, Google may also not render every subpage.

For canonical and robots, there were differences on only a few domains. A second electronics retailer stood out. Its privacy page carried a noindex in the rendered HTML and no robots tag at all in the source code. The page was still in Google's index. Google apparently did not adopt the noindex from the rendered HTML here.

Care is also needed when JavaScript changes the head or closes it too early. A canonical that then ends up in the body does not count if it only appears in the rendered HTML.

Conclusion of the study

Hardly any website gets by without complex JavaScript, and that changes how Google sees it. Rendering makes technical SEO more demanding. Especially with internal linking, many problems only show up on close inspection. Title, canonical and robots belong in the comparison too. Used correctly, however, JavaScript can also help, as the title example shows.

What has changed since?

The figures date from 2020. We have not yet run a new crawl with the same method. The examples have probably changed in the meantime. Google's JavaScript SEO documentation now describes several findings of the study itself. According to Google, inserting links into the DOM via JavaScript is fine as long as they are <a> elements with href. Title and meta description may be set via JavaScript. If a noindex is already in the source code, Google may skip rendering. There are also clear rules for canonicals via JavaScript. They are covered in the guide Canonical via JavaScript.

The main new question since 2020 is about AI crawlers. According to an analysis by Vercel from December 2024, GPTBot, ClaudeBot, PerplexityBot and other AI crawlers do not run JavaScript. They do not see content and links that only exist in the rendered HTML. Google itself writes that not all bots can run JavaScript. The difference between source code and rendered HTML has therefore become more important. More on this in the guide AI crawlers.

How to check your own website

1. Pick a few URLs per page type, such as homepage, category, product or article. The study shows that the homepage alone is not enough.
2. Open the page and run the following script in the console before you click anything.
3. Compare the number of links, title, canonical, robots and hreflang in both versions.
4. Use the live test in Search Console's URL Inspection to check whether Google renders the page the same way as your browser.
5. Check whether robots.txt blocks scripts or Ajax sources needed for navigation or content.

Common mistakes

• Only the homepage is checked, although the differences are on the subpages.
• Links in the DOM are only checked once the menu is already open.
• robots.txt blocks a resource that loads the navigation.
• A crawler without rendering serves as the only source for internal linking.
• noindex is set or removed via JavaScript without checking what Google adopts.

Frequently asked questions