Cloudflare SEO log analysis

Cloudflare sees every bot, and searchVIU analyzes the logs.

searchVIU collects your Cloudflare logs every day. You see how Googlebot and AI bots fetch your pages.

Request a demoGo to log file analysis
What is analyzed

Four reports from your Cloudflare logs

The logs from your Cloudflare zone land in your searchVIU project. There they sit alongside the crawl, Search Console and alerts.

Bot Dynamicsall verified bots over time, with search engines and AI bots shown separately
Crawl Budget & WasteFetches of redirects, parameter URLs and error pages
Coverage & HealthStatus codes your pages serve to bots
Performance & CacheResponse times for bots and the share served from the cache
More about log file analysis
AI bots

Which AI reads your pages, and what for?

Fetches by ChatGPT-User show when an AI reads your page for a specific answer. You see training bots separately. Requests to llms.txt and MCP endpoints come on top.

  • Fetches on user requestFetches for a specific question
  • Agent signalsllms.txt, .well-known, MCP
More about AI crawler monitoring
AI bots in the logsexample-shop.com · last 7 days
OAI-SearchBot5,410
ChatGPT-User3,720
GPTBot2,960
ClaudeBot1,870
PerplexityBot760
Requests to /llms.txtfrom AI bots and agents184
Fake bots and firewall

The WAF protects your site, and searchVIU shows who it locks out.

Cloudflare rules sometimes block genuine bots, too. The firewall events show you which bots are affected. At the same time, searchVIU checks whether a Googlebot really comes from Google.

  • Firewall eventsblocked requests per bot
  • Verificationgenuine and fake bots kept apart
Firewall eventsexample-shop.com · week 41
Bingbot blocked1,240 requests · rule “Rate Limit Shop”
High
OAI-SearchBot blocked310 requests · rule “Bot protection”
Medium
Fake Googlebot blocked4,130 requests · IP not from Google
Info
Setup

Once set up, the logs arrive every day.

We do the setup together with you. You create a Logpush job or an API token in Cloudflare. The permissions it needs are listed in the questions below.

1 · Connect the zoneA Logpush job or API token gives searchVIU access to the logs of your Cloudflare zone.
2 · Daily importsearchVIU collects the logs every day and verifies every bot.
3 · AnalyzeThe reports sit in your project, next to the crawl and Search Console.
Firewall and import

The firewall lets the right bots in.

  • No genuine bots blockedGooglebot, Bingbot and AI bots
  • Googlebot fetches within the usual rangeover the past few weeks
  • Logs fully importedevery day
Week 37 · no genuine bots blocked

Frequently asked questions about Cloudflare logs

How often do the logs arrive in searchVIU?
searchVIU imports the Cloudflare logs every day.
Which Cloudflare plan and which permissions are needed?
Any plan works, including Free. If your plan includes Logpush, we prefer to set up a Logpush job with you that sends your zone's logs to searchVIU. For this, the API token needs the “Logs Write” permission for your zone. Cloudflare bills Logpush by volume, with 25 GB per month included. On Free and other plans without Logpush, searchVIU reads the data via the Cloudflare API, using a token that may only read “Account Analytics”.
We don't use Cloudflare. What then?
Log file analysis also works with logs by upload, via FTP or from Amazon S3. We connect other sources. Go to log file analysis
Where is the data stored?
Your project data is stored in Google Cloud in the EU. searchVIU is certified to ISO/IEC 27001:2022. No AI is trained on your data. More about security
Do the logs contain personal data?
Yes, server logs contain IP addresses. However, we only receive the lines with bot user agents and use the IP addresses to identify genuine bots. The data processing agreement governs the processing.
How much does the analysis of Cloudflare logs cost?
The analysis runs via log file analysis. It is an add-on to SEO Monitoring, from €199 per month for all log sources. Only bot requests count toward the price. You can find all prices on the pricing page.
Demo

Let's talk about your website.

In the demo, Michael shows you how searchVIU works. We use your website and a demo project for this.

  1. Request in three short steps
  2. Quick alignment on crawler access, NDA and Search Console
  3. Demo with your website and the demo project
  4. Start planned and supported together
  1. Website
  2. Needs
  3. Contact

Which website is it about?