Log file analysis tool Bots leave traces, and searchVIU analyzes them.

searchVIU imports your server logs and separates genuine bots from fake ones. You see which pages Googlebot and AI bots fetch and where crawl budget is lost.

Request a demo
Googlebot and AI botsverified, over timeYour log sourceCloudflare, upload, FTP, Amazon S3In the same projectas the crawl and Search Console
Bot Dynamicsexample-shop.com · last 7 days
Googlebot61,240
Bingbot13,580
OAI-SearchBot5,410
ChatGPT-User3,720
GPTBot2,960
ClaudeBot1,870
Fake bots, filtered outUser agent claims to be Googlebot4,130
Your log source

searchVIU collects your logs wherever they are.

searchVIU fetches logs from Cloudflare, accepts files by upload or reads them via FTP or from Amazon S3. If you use a different source, we connect it.

    More about Cloudflare SEO log analysis
    • CloudflareLogs from your zone, imported daily
      connected directly
    • UploadLog files from your hosting
      File
    • FTPFetch logs from the server
      Server
    • Amazon S3Logs from your bucket
      Cloud
    • Other sourcefor example a CDN or load balancer
      we connect it
    Crawl Budget & Waste

    Where does Googlebot spend its time?

    The Crawl Budget & Waste report shows how many fetches go to redirects, parameter URLs and error pages. Your important pages miss out on these fetches.

    Googlebot fetches by targetDemo project · 30 days
    Indexable pages68 %
    Parameter URLs14 %
    Redirects11 %
    4xx and 5xx7 %
    Coverage & HealthStatus codes per bot. Is Googlebot getting errors your browser never sees?
    Performance & CacheResponse times for bots and how much is served from the cache
    Alongside the crawlThe full crawl runs in the same project as the logs.
    More about the SEO crawler
    Verification & Firewall

    searchVIU checks which Googlebot is genuine.

    Many requests claim to be Googlebot but come from third-party servers. searchVIU verifies every bot and only counts the genuine ones. The firewall events also show you whether your WAF is blocking genuine bots.

    • Clean numbersMetrics with genuine bots only
    • WAF in viewblocked requests per bot
    More about SEO alerts
    Verification & Securityexample-shop.com · week 41
    GooglebotIP belongs to Googleverified
    GooglebotIP does not belong to Googlefake
    Bingbot1,240 requests blocked by a firewall ruleHigh
    OAI-SearchBotIP belongs to OpenAIverified
    AI bots in your logs

    Your logs show which AI bots really visit.

    Bot Dynamics separates search engines from AI bots. You see fetches for a specific AI answer, for example from ChatGPT-User, separately from training. Requests to llms.txt and MCP endpoints come on top.

    Search engines

    Googlebot, Bingbot and other crawlers for search.

    Crawl budget
    AI training

    Bots that collect content for training models.

    for example GPTBot
    Fetches on user request

    Fetches when an AI reads your page to answer a user's question.

    for example ChatGPT-User
    Agents

    Requests to llms.txt, .well-known and MCP endpoints.

    Agent signals
    More about AI log file analysis
    This week's logs

    Googlebot is visiting as usual.

    • Fetches within the usual rangeGooglebot, Bingbot and AI bots
    • No firewall blocksfor verified bots
    • Status codes for bots unremarkableno new error pages
    Week 38 · bots as usual

    Frequently asked questions about log file analysis

    Which log sources can searchVIU analyze?

    Cloudflare, log files by upload, FTP and Amazon S3. We connect other sources on request.

    We don't use Cloudflare. Does it still work?

    Yes. Cloudflare is one of several sources. The logs then come via upload, FTP or from Amazon S3.

    How is this different from the crawl stats in Search Console?

    Search Console shows Googlebot from Google's point of view and only in aggregate. In the logs you see every fetch per URL, including from Bingbot and AI bots. Crawl, clicks and logs sit in the same project.

    Where is the data stored?

    Your project data is stored in Google Cloud in the EU. searchVIU is certified to ISO/IEC 27001:2022. No AI is trained on your data. More about security

    Do the logs contain personal data?

    Yes, server logs contain IP addresses. However, we only receive the lines with bot user agents and use the IP addresses to identify genuine bots. The data processing agreement governs the processing.

    How much does log file analysis cost?

    Log file analysis is an add-on to SEO Monitoring, from €199 per month for all log sources. Only bot requests count towards the price. You can find all prices on the pricing page.

    Demo

    Let's talk about your website.

    In the demo, Michael shows you how searchVIU works. We use your website and a demo project for this.

    1. Request in three short steps
    2. Quick alignment on crawler access, NDA and Search Console
    3. Demo with your website and the demo project
    4. Start planned and supported together
    1. Website
    2. Needs
    3. Contact

    Which website is it about?