AI log file analysis searchVIU reads the traces AI bots leave in your logs.

searchVIU analyzes your server logs, bot by bot. That way you see which pages ChatGPT and other AI services really fetch.

Request a demo
Genuine botskept apart from fake onesFetches on user requestkept apart from training and searchAdd-onto SEO Monitoring
AI bots in the logsexample-shop.com · last 7 days
Fetches by verified AI bots
ChatGPT-User4,820
OAI-SearchBot3,110
ClaudeBot2,430
GPTBot1,980
PerplexityBot1,270

Plus 12,900 fetches that claim to be Googlebot but do not come from Google.

Server logs

Every fetch is in the log, including ChatGPT's.

The robots.txt controls which bots may fetch your pages. What they actually fetch, and how often, is only in your server logs.

robots.txt

Sets which bots may fetch. searchVIU checks it every 15 minutes, with all known AI bots.

Go to AI crawler monitoring

Server logs

Record every fetch, with bot, URL, time and your server's response.

Fetches on user request

Someone asks ChatGPT, and ChatGPT fetches your page.

Bots such as ChatGPT-User fetch pages live when someone asks a question. These fetches show you early on which of your pages AI services use for answers. searchVIU shows them separately from training and search.

    More about AI Visibility Monitoring
    Fetches on user requestexample-shop.com · week 41
    • /tents/2-person/ChatGPT-User, Claude-User
      214 fetches
    • /guides/waterproofing/ChatGPT-User
      167 fetches
    • /women/hiking-boots/ChatGPT-User, MistralAI-User
      98 fetches
    Verification

    Which bots are genuine, and which only use the name?

    Anyone can claim to be Googlebot or GPTBot in the user agent. searchVIU checks whether a fetch really comes from the stated bot. Fake bots distort your figures and add server load.

    User agentVerifiedFetches
    Googlebot182,400
    Googlebotdoes not come from Google12,900
    ChatGPT-User4,820
    ClaudeBot2,430
    GPTBotdoes not come from OpenAI2,150

    Demo project, last 7 days.

    Agent signals

    What are AI agents looking for on your website?

    searchVIU counts fetches of llms.txt, of files under .well-known and of MCP endpoints, per bot. That way you see who specifically targets your website for AI offerings.

    Agent Readiness & WebMCP Audit

      More about llms.txt Monitoring
      Agent signalsexample-shop.com · last 7 days
      • /llms.txtGPTBot, ClaudeBot, PerplexityBot
        38 fetches
      • /llms-full.txtClaudeBot
        6 fetches
      • /.well-known/various agents
        11 fetches
      • /mcpunknown agent
        3 fetches
      Bot protection

      The logs show who your bot protection blocks.

      With Cloudflare logs you also see the firewall events. That way you notice when stricter bot protection locks out Googlebot or AI bots that you have allowed in the robots.txt.

      How the logs come in

      Your logs sit in the same project as your crawl.

      searchVIU collects the logs wherever you keep them. They are analyzed alongside the crawl, robots.txt and Search Console. Your project data is stored in the EU.

      1Log source

      Cloudflare, upload, FTP or Amazon S3

      2Import

      Cloudflare logs arrive daily

      3Bots

      recognized, verified and assigned to a class

      4Reports

      alongside the crawl, robots.txt and Search Console

      More about Cloudflare log file analysis
      Last week's logs

      Nothing out of the ordinary in the logs

      • ChatGPT-User is fetchingas often as in previous weeks
      • Googlebot verifiedno fake fetches noticed
      • Firewall lets them throughGooglebot and AI search bots not blocked
      Week 40 · logs unremarkable

      Frequently asked questions about AI log file analysis

      Which log sources can searchVIU analyze?

      Cloudflare, upload, FTP and Amazon S3. searchVIU connects further sources on request.

      When will I see data?

      From the start of log file analysis. How far back older logs go, we clarify during setup.

      Are our logs allowed to leave the company?

      Your project data is stored in the EU, with logs kept separately per project. We only receive the lines with bot user agents. We use the IP addresses in them to check whether a fetch really comes from the stated bot. The data processing agreement, which is part of our terms and conditions, governs the processing. searchVIU is certified to ISO/IEC 27001:2022, and no AI is trained on your data. More about security

      How is this different from log file analysis?

      It is the same add-on. This page shows the view of AI bots. Log file analysis also shows Googlebot, crawl budget and the coverage of your pages. Go to log file analysis

      How much does AI log file analysis cost?

      Log file analysis is an add-on to SEO Monitoring, from €199 per month, for all log sources. You can find all prices on the pricing page. Go to pricing

      Demo

      Let's talk about your website.

      In the demo, Michael shows you how searchVIU works. We use your website and a demo project for this.

      1. Request in three short steps
      2. Quick alignment on crawler access, NDA and Search Console
      3. Demo with your website and the demo project
      4. Start planned and supported together
      1. Website
      2. Needs
      3. Contact

      Which website is it about?