{"id":677,"date":"2023-02-04T18:41:13","date_gmt":"2023-02-04T18:41:13","guid":{"rendered":"https:\/\/www.viu-data.com\/?page_id=677"},"modified":"2025-04-30T07:32:41","modified_gmt":"2025-04-30T07:32:41","slug":"robots-txt-pre-check","status":"publish","type":"page","link":"https:\/\/www.searchviu.com\/de\/robots-txt-vorabprufung\/","title":{"rendered":"Robots.txt Vorabpr\u00fcfung"},"content":{"rendered":"<div data-elementor-type=\"wp-page\" data-elementor-id=\"677\" class=\"elementor elementor-677\" data-elementor-post-type=\"page\">\n\t\t\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-6213c95 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"6213c95\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;background_background&quot;:&quot;classic&quot;,&quot;shape_divider_bottom&quot;:&quot;split&quot;,&quot;shape_divider_bottom_negative&quot;:&quot;yes&quot;,&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t<div class=\"elementor-shape elementor-shape-bottom\" aria-hidden=\"true\" data-negative=\"true\">\n\t\t\t<svg xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewbox=\"0 0 1000 20\" preserveaspectratio=\"none\">\n\t<path class=\"elementor-shape-fill\" d=\"M519.8,0.2c-11,0-19.8,8.5-19.8,19c0-10.4-8.8-19-19.8-19L0,0v20h1000V0.2H519.8z\"\/>\n<\/svg>\t\t<\/div>\n\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-a56c83c\" data-id=\"a56c83c\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-a264cb8 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"a264cb8\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-16a4ca5\" data-id=\"16a4ca5\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-265bf4a elementor-widget elementor-widget-heading\" data-id=\"265bf4a\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h5 class=\"elementor-heading-title elementor-size-default\">Anwendungsfall<\/h5>\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-a2322e7\" data-id=\"a2322e7\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-c828a85 elementor-widget elementor-widget-text-editor\" data-id=\"c828a85\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<div class=\"wpb_text_column wpb_content_element vc_custom_1602597132967\"><div class=\"wpb_wrapper\"><div id=\"custom_post_widget-4525\" class=\"content_block\"><div class=\"wpb_text_column wpb_content_element vc_custom_1602599897715\"><div class=\"wpb_wrapper\"><div id=\"custom_post_widget-4523\" class=\"content_block\"><h4>Stelle sicher, dass \u00c4nderungen an der robots.txt-Datei zum gew\u00fcnschten Ergebnis f\u00fchren und dass kein Traffic oder Umsatz verloren geht.<\/h4><\/div><\/div><\/div><div class=\"wpb_text_column wpb_content_element\"><div class=\"wpb_wrapper\"><p>Vor allem bei gro\u00dfen Websites oder Websites mit vielen dynamischen Inhalten gibt es oft eine extrem hohe Anzahl von URLs. Das bedeutet, dass Google nicht mehr alle URLs crawlt und wichtige Unterseiten nicht indizieren kann. Um das \u201cCrawl-Budget\u201d optimal zu nutzen, ist es in solchen F\u00e4llen oft sinnvoll, Seiten \u00fcber die robots.txt auszuschlie\u00dfen.<\/p><p>Es sollte sichergestellt werden, dass keine relevanten Seiten ausgeschlossen werden.<\/p><\/div><\/div><\/div><\/div><\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<section class=\"elementor-section elementor-top-section elementor-element elementor-element-9166818 elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"9166818\" data-element_type=\"section\" data-e-type=\"section\" data-settings=\"{&quot;ekit_has_onepagescroll_dot&quot;:&quot;yes&quot;}\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-2adf7c0\" data-id=\"2adf7c0\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-f0211c9 elementor-widget elementor-widget-heading\" data-id=\"f0211c9\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h5 class=\"elementor-heading-title elementor-size-default\">So k\u00f6nnen wir helfen<\/h5>\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t<div class=\"elementor-column elementor-col-50 elementor-top-column elementor-element elementor-element-cda01a2\" data-id=\"cda01a2\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-6b1622a elementor-widget elementor-widget-text-editor\" data-id=\"6b1622a\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;ekit_we_effect_on&quot;:&quot;none&quot;}\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<div class=\"wpb_text_column wpb_content_element vc_custom_1602597132967\"><div class=\"wpb_wrapper\"><div id=\"custom_post_widget-4525\" class=\"content_block\"><p>Bevor wir die robots.txt tats\u00e4chlich \u00e4ndern, crawlen wir die Website mit der bei uns gespeicherten neuen Version und vergleichen den Crawl mit dem letzten Live-Crawl der Website. So k\u00f6nnen wir u.a. die folgenden strukturellen Faktoren \u00fcberpr\u00fcfen:<\/p><ul><li>K\u00f6nnen Inhalte mit Klicks oder Impressionen weiter indexiert werden - wenn nicht, welche?<\/li><li>Wurden wichtige Vertriebsseiten ausgelassen und sind Inhalte mit Klicks oder Impressionen intern nicht mehr verlinkt?<\/li><li>Gibt es extern verlinkte Seiten, die f\u00fcr den Googlebot nicht mehr crawlbar sind?<\/li><li>Sind relevante Seiten in der internen Linkstruktur zur\u00fcckgefallen?<\/li><\/ul><\/div><\/div><\/div>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>","protected":false},"excerpt":{"rendered":"<p>Use case Ensure that changes to the robots.txt file lead to the desired outcome and that no traffic or revenue are lost. There is often an extremely high number of URLs, especially with large websites or websites with a lot of dynamic content. This means that Google no longer crawls all URLs and cannot index [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"elementor_header_footer","meta":{"_lmt_disableupdate":"","_lmt_disable":"","footnotes":""},"class_list":["post-677","page","type-page","status-publish","hentry"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.5 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Robots.txt pre-check - SEO Data Warehouse And Monitoring<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.searchviu.com\/de\/robots-txt-vorabprufung\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Robots.txt pre-check - SEO Data Warehouse And Monitoring\" \/>\n<meta property=\"og:description\" content=\"Use case Ensure that changes to the robots.txt file lead to the desired outcome and that no traffic or revenue are lost. There is often an extremely high number of URLs, especially with large websites or websites with a lot of dynamic content. This means that Google no longer crawls all URLs and cannot index [&hellip;]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.searchviu.com\/de\/robots-txt-vorabprufung\/\" \/>\n<meta property=\"og:site_name\" content=\"SEO Data Warehouse And Monitoring\" \/>\n<meta property=\"article:modified_time\" content=\"2025-04-30T07:32:41+00:00\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@searchviu\" \/>\n<meta name=\"twitter:label1\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data1\" content=\"1\u00a0Minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/en\\\/robots-txt-pre-check\\\/\",\"url\":\"https:\\\/\\\/www.searchviu.com\\\/en\\\/robots-txt-pre-check\\\/\",\"name\":\"Robots.txt pre-check - SEO Data Warehouse And Monitoring\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/#website\"},\"datePublished\":\"2023-02-04T18:41:13+00:00\",\"dateModified\":\"2025-04-30T07:32:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/en\\\/robots-txt-pre-check\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.searchviu.com\\\/en\\\/robots-txt-pre-check\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/en\\\/robots-txt-pre-check\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.searchviu.com\\\/en\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Robots.txt pre-check\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/#website\",\"url\":\"https:\\\/\\\/www.searchviu.com\\\/\",\"name\":\"SEO Data Warehouse And Monitoring\",\"description\":\"SEO Solutions For Teams And Agencies\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.searchviu.com\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/#organization\",\"name\":\"searchVIU\",\"url\":\"https:\\\/\\\/www.searchviu.com\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.searchviu.com\\\/wp-content\\\/uploads\\\/2023\\\/02\\\/d3babe41-searchviu-logo2x.png\",\"contentUrl\":\"https:\\\/\\\/www.searchviu.com\\\/wp-content\\\/uploads\\\/2023\\\/02\\\/d3babe41-searchviu-logo2x.png\",\"width\":340,\"height\":340,\"caption\":\"searchVIU\"},\"image\":{\"@id\":\"https:\\\/\\\/www.searchviu.com\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/searchviu\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/searchviu\\\/\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Robots.txt Vorabpr\u00fcfung - SEO Data Warehouse und \u00dcberwachung","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.searchviu.com\/de\/robots-txt-vorabprufung\/","og_locale":"de_DE","og_type":"article","og_title":"Robots.txt pre-check - SEO Data Warehouse And Monitoring","og_description":"Use case Ensure that changes to the robots.txt file lead to the desired outcome and that no traffic or revenue are lost. There is often an extremely high number of URLs, especially with large websites or websites with a lot of dynamic content. This means that Google no longer crawls all URLs and cannot index [&hellip;]","og_url":"https:\/\/www.searchviu.com\/de\/robots-txt-vorabprufung\/","og_site_name":"SEO Data Warehouse And Monitoring","article_modified_time":"2025-04-30T07:32:41+00:00","twitter_card":"summary_large_image","twitter_site":"@searchviu","twitter_misc":{"Gesch\u00e4tzte Lesezeit":"1\u00a0Minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.searchviu.com\/en\/robots-txt-pre-check\/","url":"https:\/\/www.searchviu.com\/en\/robots-txt-pre-check\/","name":"Robots.txt Vorabpr\u00fcfung - SEO Data Warehouse und \u00dcberwachung","isPartOf":{"@id":"https:\/\/www.searchviu.com\/#website"},"datePublished":"2023-02-04T18:41:13+00:00","dateModified":"2025-04-30T07:32:41+00:00","breadcrumb":{"@id":"https:\/\/www.searchviu.com\/en\/robots-txt-pre-check\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.searchviu.com\/en\/robots-txt-pre-check\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.searchviu.com\/en\/robots-txt-pre-check\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.searchviu.com\/en\/"},{"@type":"ListItem","position":2,"name":"Robots.txt pre-check"}]},{"@type":"WebSite","@id":"https:\/\/www.searchviu.com\/#website","url":"https:\/\/www.searchviu.com\/","name":"SEO Data Warehouse und Monitoring","description":"SEO-L\u00f6sungen f\u00fcr Teams und Agenturen","publisher":{"@id":"https:\/\/www.searchviu.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.searchviu.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"https:\/\/www.searchviu.com\/#organization","name":"searchVIU","url":"https:\/\/www.searchviu.com\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/www.searchviu.com\/#\/schema\/logo\/image\/","url":"https:\/\/www.searchviu.com\/wp-content\/uploads\/2023\/02\/d3babe41-searchviu-logo2x.png","contentUrl":"https:\/\/www.searchviu.com\/wp-content\/uploads\/2023\/02\/d3babe41-searchviu-logo2x.png","width":340,"height":340,"caption":"searchVIU"},"image":{"@id":"https:\/\/www.searchviu.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/searchviu","https:\/\/www.linkedin.com\/company\/searchviu\/"]}]}},"_links":{"self":[{"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/pages\/677","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/comments?post=677"}],"version-history":[{"count":7,"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/pages\/677\/revisions"}],"predecessor-version":[{"id":6508,"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/pages\/677\/revisions\/6508"}],"wp:attachment":[{"href":"https:\/\/www.searchviu.com\/de\/wp-json\/wp\/v2\/media?parent=677"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}