Guides

Guides for SEO and GEO

How-tos on technical SEO, AI crawlers, data and website migrations.

AI & GEO

UserAgentShopTaskDataCheckoutsearch · compare · buy
AI & GEOAgentic commerce, when AI agents shop for your customers

Which protocols exist, where they already run today and what your shop can prepare technically.

EmailOrderTreetextbox“Email”button“Order”
AI & GEOAgentic Engine Optimization for websites that AI agents can use

How agents read pages, where they fail and how to fix the most common stumbling blocks.

SEOGEORankingsMentionsContentreachable · crawlable · renderable
AI & GEOWhat stays the same in GEO and SEO, and what is new?

What Generative Engine Optimization means, why it builds on technical SEO and how to measure mentions in AI answers.

AI browserChrome+ AgentCloudBrowser
AI & GEOAI browsers such as Comet, Claude in Chrome and Gemini compared

Which agents exist, how they are built, what has changed since August 2026 and why you barely see them in your data.

TrainingSearchFetch
AI & GEOWhich AI crawlers should you allow?

Which bots train, which crawl for AI search and what a block in robots.txt really does.

1 Searchtitle and URL only2 Fetchno JavaScript3 Markdownno head, script4 Summary5 Main model
AI & GEOWhat does the Claude Code leak reveal about AI grounding and GEO?

How Claude Code searches, converts and summarises web pages, and what of that reaches the model.

public dataset = small sample
AI & GEOWhat do prompt data show, and what not?

Why public datasets of real AI prompts are small and skewed, and what to use for demand instead.

# llms.txt> Summary## Docs## Optional
AI & GEOllms.txt explained, with structure, example and limits

What belongs in the file, who reads it and whether it helps you in ChatGPT and similar tools.

document.modelContextproduct_searchadd_to_basketcheck_delivery_timeAgent
AI & GEOHow AI agents use your website with WebMCP

How forms and functions become tools for browser agents, what Chrome supports today and how to measure agent calls.

HTMLDOMGPTBotGooglebot
AI & GEOAI crawlers and JavaScript rendering

Which AI crawlers execute JavaScript and what that means for your content, with an overview of 39 bots.

Accessibility treeForm with labelsCookie banner closableBot protection aligned
AI & GEOOptimize your website for AI browsers

Where AI browsers get stuck on your site and how you adjust structure, forms, banners and bot protection.

Technical SEO

/old/interim/new302301/old/new301 direct
Technical SEO301 or 302, which redirect fits when?

What the status codes mean for Google, how to set them up and how to avoid chains during a migration.

/table?colour=red/table?utm=mail/TABLE//tablecanonical
Technical SEOSetting canonical tags correctly and finding errors

How Google picks the canonical URL and how to avoid conflicts with sitemaps, JavaScript and hreflang.

GooglebotGPTBotVisitorsCloudflareServerNo blocks logged
Technical SEOCloudflare sits in front of your server and has a say in SEO and GEO.

Cloudflare decides which crawlers see your pages. Here is how to check blocks, logs and robots.txt.

LCPINPCLS2.5 s4 s200 ms500 ms0.10.25
Technical SEOThe Core Web Vitals LCP, INP and CLS explained

Thresholds, field and lab data, and the most common causes of poor scores.

CrawlingRenderingIndexrobots.txtnoindex
Technical SEOWhy are pages missing from Google's index, and what helps?

This guide explains how Google crawls, renders and indexes pages, and how to read the “Page indexing” report.

de-DEde-ATenx-default
Technical SEOThe hreflang guide for international websites

Codes, x-default and the three ways to implement it, with examples and the most common mistakes.

Page /en/blog/ENDEGTM<head>hreflang enhreflang derendered
Technical SEOImplementing hreflang with Google Tag Manager in three steps

How to generate hreflang from the language switcher without development work, and where the method reaches its limits.

/de/tisch/canonical = self/en/table/canonical = selfhreflang/de/tisch/?farbe=rotcanonicalno hreflangon variants
Technical SEOHow do hreflang and canonical fit together?

Which URL gets which signal and how to avoid conflicting signals on international websites.

/en-eu/one URLen-gben-ieen-nlen-been-ch/en-us/ = en-us
Technical SEOWhat does Google say about multiple hreflang values on one URL?

A URL gets several hreflang values when it is both x-default and a language version, or serves several countries.

Technical SEOFrom SEO audit to priority list, step by step

What a technical SEO audit checks, how to prepare it and how thousands of findings become a short list.

index.xmlproductscategoriesguides
Technical SEOStructure, limits and typical mistakes of XML sitemaps

How to split sitemaps for large websites, what Google reads and how to find errors early.

HumanGooglebotverifiedVPN userScraperChallenge?throttled
Technical SEOBot protection and SEO

How bot protection locks out customers, Googlebot and AI bots, and how you verify bots and respond in stages.

staging401Go-livelive
Technical SEOStaging environment and SEO

Why Disallow: / and noindex on staging are risky, and how password protection and IP allowlisting keep the test environment safe and testable.

Search ConsoleCrawl stats5xxPage indexing404URL Inspection200 OK
Technical SEOCrawl errors in Search Console

Where the errors from the old report live today and which ones are really worth your time.

200 OK404
Technical SEOFinding, assessing and fixing 404 errors

How to use GA4, Search Console and a crawl to find the 404 errors that cost clicks.

Data & monitoring

RolloutBeforeAfter+7 d
Data & monitoringHas a Google core update hit you?

All core updates since 2024 and how to check in Search Console whether a loss comes from the update.

ProductsCategoriesFilter URLs
Data & monitoringLog file analysis shows what bots really crawl

How to find Googlebot and AI bots in your logs, separate real from fake ones and when crawl budget matters.

SearchConsoledailysearchconsolesite_impressionurl_impressionExportLog
Data & monitoringAll Search Console data in BigQuery every day

How to set up the bulk data export and what it shows beyond the interface and the API.

PositionClicks
Data & monitoringWhat belongs in an SEO ranking report?

Which metrics count and how to bring Rank Tracking and Search Console together in one report.

Search ConsoleLinksLatest links≤ 100,000 rowsSample links≤ 100,000 rowsCSV · Google Sheets
Data & monitoringExport backlinks from Search Console

How to get the links out of the Links report, where the sample ends and why there is no API for it.

InputPage Path/en/guides/Table/help/ › Help/guides/ › Guides/pricing/ › Pricingelse › OtherOutputGuides
Data & monitoringLookup tables in Google Tag Manager

Lookup and RegEx tables with GA4 examples, from campaigns via URL fragments to one measurement ID per domain.

GA4 · ExplorationPage title contains “404”Page pathPage referrerViews/old-table/garden-furniture312/sale-2024/newsletter87/contact.html/imprint23
Data & monitoring404 errors in Google Analytics 4

Which URLs lead nowhere and which link brings visitors there, step by step in GA4.

Relaunch & migration

BeforeStagingGo-liveAfter301 ✓Redirects at least 1 year
Relaunch & migrationWebsite migration checklist with every SEO step until after go-live

What to secure before the migration, check on staging and watch after go-live.

Go-liveSEO crawlerother bots
Relaunch & migrationWhy does the SEO crawler come under suspicion after a migration?

What really slows things down after go-live and how to find the actual cause with your infrastructure team.

example.atexample.deexample.co.uk301example.com/at//de//gb/
Relaunch & migrationWhat does moving from country domains to one gTLD achieve?

A case study with three markets shows when visibility grows after the merger and when it drops first.

old URLs301301Target OK?
Relaunch & migrationTesting redirects before and after go-live

How to check redirects on staging, avoid chains and keep an eye on old URLs after the migration.

Staging401Live✓ robots.txt✓ index✓ canonical✓ hreflang
Technical SEOFinding errors on staging with an SEO check before go-live

How to protect your test environment properly and check every release before it goes live.

Go-live
Relaunch & migrationMigration traffic forecast

Three scenarios show what is at stake for SEO in a website migration and what SEO in the project is worth spending.

90% of clicks
Relaunch & migrationImportant content in a website migration

Which pages, PDFs and images bring traffic today, and how do you carry them over to the new website?

Relaunch & migrationSEO risks in a website migration

Nine causes of ranking losses after a website migration and how to check them before and after the go-live.

Experiments and studies

FeaturesVolumeCPCTitle 0/0.5/1ContentURL, DomainPosition#1
Experiment · 2017Predicting rankings with machine learning, our experiment from 2017

How we used SEMrush data, on-page features and a neural network to find keywords with ranking potential.

Source codehreflang="de"hreflang="en"RenderedGoogle reads the right side
Experiment · 2017What does Google do with hreflang that is only in the source code?

An experiment on searchviu.com shows which version of a page Google reads for hreflang.

GTM changeuntil visibleTitle + DescriptionnoindexCanonicalinternal linkURL in JSON-LD8 days
Experiment · 2017Five tests of SEO changes via Tag Manager

We tested on searchviu.com what Google takes from the rendered HTML and how long that takes.

BlogoverviewPage APage BPage CPage D10 d23 d24 d34 dCanonical only in the rendered HTML
Experiment · 2018Canonical via JavaScript, tested with four URLs

We tested whether Google picks up canonicals from the rendered HTML and how long that takes.

anonymousbasketbypassCacheCloudflareHTML 30mServerLoad time 4.49 s → 3.25 s
Case study · 2019HTML caching with Cloudflare made an online shop 28 percent faster

Three rules and a cookie cut the shop's load time from 4.49 to 3.25 seconds. Here is how we did it.

Differences source ↔ renderedDomains96 %URLs56 %over 200 domains, as of 2020
Study · 2020Over 200 domains show how often the rendered HTML differs

We show where source code and rendered HTML differ and what Google and AI crawlers see of it.

Regular visitorsw.js activeGooglebotRejectedCacheServer
Experiment · 2020What does Googlebot see of service workers?

Googlebot rejects service workers. Here is how to check that your pages arrive complete without them.

5,99 €<script ld+json>"price": "8.99"AI
Experiment · 2025Do AI systems really read schema markup?

Our test with eight price sources and five AI systems shows what arrives when a page is fetched.