{"id":15549,"date":"2019-07-17T09:44:44","date_gmt":"2019-07-16T20:44:44","guid":{"rendered":"https:\/\/www.pureseo.co.nz\/?p=15549"},"modified":"2019-08-15T09:01:27","modified_gmt":"2019-08-14T21:01:27","slug":"google-ends-support-robotstxt-noindex","status":"publish","type":"post","link":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex","title":{"rendered":"Google Announces End of Support for Robots.txt Noindex Directives"},"content":{"rendered":"\n<p>Google announced early this month that they will be discontinuing their unofficial support of noindex directives within robots.txt files. As of 1 September 2019, publishers who are currently relying on robots.txt crawl-delay, nofollow, and noindex directives will need to find another way to instruct search engine robots how to crawl their sites&#8217; pages.<\/p>\n\n\n\n<p>These policy updates will mean rapid changes are necessary for many web publishers in preserving their approach to <a href=\"https:\/\/www.pureseo.co.nz\/ \">search engine optimisation<\/a>. Are you one of them? Continue reading to learn more about Google&#8217;s major policy update and what it will mean moving forward.<\/p>\n\n\n\n<br>\n\n\n\n<h2 class=\"wp-block-heading\">What are robots.txt files?<\/h2>\n\n\n\n<p>Webmasters create <a href=\"https:\/\/moz.com\/learn\/seo\/robotstxt\" target=\"_blank\" rel=\"nofollow noreferrer noopener\" aria-label=\" (opens in a new tab)\">robots.txt text files<\/a> to direct user agents (web crawlers) how to crawl the pages on their websites. These text files either allow or disallow web robots such as search engine crawlers to engage in specified behaviour.<\/p>\n\n\n\n<p>Robots.txt files can be as short as two lines long. The first line specifies a user agent to which the directive applies. The second provides specific instructions to that user agent, such as allow or disallow. Specific web crawlers will disregard robots.txt files that are not directed at them, and some will ignore the files that are. Google will soon count themselves among the latter group.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" data-src=\"https:\/\/www.pureseo.co.nz\/wp-content\/uploads\/2019\/07\/robots.txt-2-2-1.png\" alt=\"\" class=\"wp-image-15593 lazyload\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" \/><figcaption>An example of noindex robots.txt directives, which Google will soon stop acknowledging.<br><\/figcaption><\/figure>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" data-src=\"https:\/\/www.pureseo.co.nz\/wp-content\/uploads\/2019\/07\/robots.txt-2-3.png\" alt=\"\" class=\"wp-image-15594 lazyload\" src=\"data:image\/svg+xml;base64,PHN2ZyB3aWR0aD0iMSIgaGVpZ2h0PSIxIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPjwvc3ZnPg==\" \/><figcaption>An example of disallow robots.txt directives, which will still function.<\/figcaption><\/figure>\n\n\n\n<p><a href=\"https:\/\/www.seroundtable.com\/google-noindex-in-robots-txt-dead-27824.html\" target=\"_blank\" rel=\"nofollow noreferrer noopener\" aria-label=\" (opens in a new tab)\">Google has long discouraged<\/a> publishers from using crawl-delay, nofollow, and noindex directives within robots.txt files, but have followed most directives in spite of having no standardised policy toward them.<\/p>\n\n\n\n<br>\n\n\n\n<h2 class=\"wp-block-heading\">Why is Google ditching robots.txt now?<\/h2>\n\n\n\n<p>Google has spent years trying to standardise their robot exclusion protocol so that they can move ahead of this change. This is why Google has also long encouraged publishers to find alternatives to robots.txt directives.<\/p>\n\n\n\n<p>In their <a href=\"https:\/\/webmasters.googleblog.com\/2019\/07\/repp-oss.html\">announcem<\/a><a href=\"https:\/\/webmasters.googleblog.com\/2019\/07\/repp-oss.html\" target=\"_blank\" rel=\"nofollow noreferrer noopener\" aria-label=\"e (opens in a new tab)\">e<\/a><a href=\"https:\/\/webmasters.googleblog.com\/2019\/07\/repp-oss.html\">nt<\/a>, Google said they were making the Robots Exclusion Protocol (REP) an internet standard. To do so, they have open-sourced the C++ library they used to parse and match roles in robots.txt files. The 20-plus-year-old library, along with a testing tool offered by Google, can help developers create the parsing tools of the future.<\/p>\n\n\n\n<br>\n\n\n\n<h2 class=\"wp-block-heading\">How can you keep controlling crawling on your site?<\/h2>\n\n\n\n<p>Disregarding robots.txt noindex does not leave publishers without means to control crawling on their sites.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Use robots meta tags to noindex<\/strong><\/h3>\n\n\n\n<p>Robots meta tags are supported in HTTP response headers, as well as HTML.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Use HTTP status codes 404 and 410<\/strong><\/h3>\n\n\n\n<p>These status codes tell crawlers that the page does not exist, thus dropping it from Google&#8217;s index once they&#8217;ve been crawled and processed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Hide content behind password protections<\/strong><\/h3>\n\n\n\n<p>Unless you&#8217;ve signaled subscription or paywall content with markup, content that is concealed behind login pages will often remove it from Google&#8217;s index.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Disallow in robots.txt<\/strong><\/h3>\n\n\n\n<p>If search engines are disallowed from crawling a page, that content cannot be indexed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>Use the Search Console Remove URL tool<\/strong><\/h3>\n\n\n\n<p>The <a href=\"https:\/\/support.google.com\/webmasters\/answer\/1663419\">Search Console Remove URL tool<\/a> allows for the temporary removal of a URL from Google&#8217;s search results.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Adapting your noindex approach for the future<\/h3>\n\n\n\n<p>If your sites have relied on robots.txt noindex directives to avoid search engine indexing, you have until 1 September to make the necessary changes. If you&#8217;re not sure whether your site uses noindex directives, it would be wise to double check. Indexing certain pages you want concealed can cost your website in search rankings.<\/p>\n\n\n\n<p>For help auditing your site&#8217;s indexing and search performance, continue reading the <a href=\"\/?cat_ID=2\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">Pure SEO blog<\/a> for more information, or <a href=\"https:\/\/www.pureseo.co.nz\/contact-us\/\">contact us today<\/a> for the SEO support you need.<\/p>\n\n\n\n<br>\n","protected":false},"excerpt":{"rendered":"<p>Google announced early this month that they will be discontinuing their unofficial support of noindex directives within robots.txt files. As&#8230;<\/p>\n","protected":false},"author":9,"featured_media":22428,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"ep_exclude_from_search":false,"footnotes":""},"categories":[2,301],"tags":[],"ai":[],"class_list":["post-15549","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog","category-seo-news","entry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.5 (Yoast SEO v27.5) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Google Announces End of Support for Robots.txt Noindex Directives - Pure SEO<\/title>\n<meta name=\"description\" content=\"Google announced that they will be discontinuing their support of noindex directives within robots.txt files. Find out if these changes affect you.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Google Announces End of Support for Robots.txt Noindex Directives\" \/>\n<meta property=\"og:url\" content=\"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex\" \/>\n<meta property=\"og:site_name\" content=\"Pure SEO\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/pureseo\" \/>\n<meta property=\"article:published_time\" content=\"2019-07-16T20:44:44+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-08-14T21:01:27+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/07\/google-announces-end-of-support-for-robots-txt.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1920\" \/>\n\t<meta property=\"og:image:height\" content=\"1080\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Rollan Schott\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Pure_SEO\" \/>\n<meta name=\"twitter:site\" content=\"@Pure_SEO\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Rollan Schott\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex\"},\"author\":{\"name\":\"Rollan Schott\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn#\\\/schema\\\/person\\\/411cce715efca3feeccb65335a1fd370\"},\"headline\":\"Google Announces End of Support for Robots.txt Noindex Directives\",\"datePublished\":\"2019-07-16T20:44:44+00:00\",\"dateModified\":\"2019-08-14T21:01:27+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex\"},\"wordCount\":614,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/pureseo.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/google-announces-end-of-support-for-robots-txt.jpg\",\"articleSection\":[\"Blog\",\"News\"],\"inLanguage\":\"en-CN\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#respond\"]}],\"copyrightYear\":\"2019\",\"copyrightHolder\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex\",\"url\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex\",\"name\":\"Google Announces End of Support for Robots.txt Noindex Directives - Pure SEO\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/pureseo.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/google-announces-end-of-support-for-robots-txt.jpg\",\"datePublished\":\"2019-07-16T20:44:44+00:00\",\"dateModified\":\"2019-08-14T21:01:27+00:00\",\"description\":\"Google announced that they will be discontinuing their support of noindex directives within robots.txt files. Find out if these changes affect you.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#breadcrumb\"},\"inLanguage\":\"en-CN\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-CN\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#primaryimage\",\"url\":\"https:\\\/\\\/pureseo.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/google-announces-end-of-support-for-robots-txt.jpg\",\"contentUrl\":\"https:\\\/\\\/pureseo.com\\\/wp-content\\\/uploads\\\/2019\\\/07\\\/google-announces-end-of-support-for-robots-txt.jpg\",\"width\":1920,\"height\":1080,\"caption\":\"Robots\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Digital Marketing Agency\",\"item\":\"https:\\\/\\\/pureseo.com\\\/cn\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Blog\",\"item\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Google Announces End of Support for Robots.txt Noindex Directives\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn#website\",\"url\":\"https:\\\/\\\/pureseo.com\\\/cn\",\"name\":\"Pure SEO\",\"description\":\"Digital Marketing Agency\",\"publisher\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/pureseo.com\\\/cn?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-CN\"},{\"@type\":[\"Organization\",\"Place\"],\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn#organization\",\"name\":\"Pure SEO\",\"url\":\"https:\\\/\\\/pureseo.com\\\/cn\",\"logo\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#local-main-organization-logo\"},\"image\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#local-main-organization-logo\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/pureseo\",\"https:\\\/\\\/x.com\\\/Pure_SEO\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/pureseo\\\/\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UCinrMPUg7xZncotAkegPvEA\"],\"address\":{\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#local-main-place-address\"},\"telephone\":[],\"openingHoursSpecification\":[{\"@type\":\"OpeningHoursSpecification\",\"dayOfWeek\":[\"Monday\",\"Tuesday\",\"Wednesday\",\"Thursday\",\"Friday\",\"Saturday\",\"Sunday\"],\"opens\":\"09:00\",\"closes\":\"17:00\"}]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn#\\\/schema\\\/person\\\/411cce715efca3feeccb65335a1fd370\",\"name\":\"Rollan Schott\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-CN\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d995014ad341cf72f26d46d922532cac9c3fc7d6d8b577bfda98c798a0075820?s=96&d=mm&r=g\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d995014ad341cf72f26d46d922532cac9c3fc7d6d8b577bfda98c798a0075820?s=96&d=mm&r=g\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/d995014ad341cf72f26d46d922532cac9c3fc7d6d8b577bfda98c798a0075820?s=96&d=mm&r=g\",\"caption\":\"Rollan Schott\"},\"description\":\"Rollan Schott is a copywriter with Pure SEO. Rollan was born and raised in the United States, having moved to New Zealand after 4 years teaching and writing in Asia. When he's not churning out quality content at breakneck speed, Rollan is probably busy writing the next great American novel. He may also be idly watching true crime documentaries in his Auckland Central apartment with his wife, Lauren. The latter is more likely than the former.\",\"sameAs\":[\"https:\\\/\\\/www.linkedin.com\\\/in\\\/rollanschott\\\/\"],\"url\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/author\\\/rollan\"},{\"@type\":\"PostalAddress\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#local-main-place-address\",\"streetAddress\":\"35 Kawana Street, Northcote\",\"addressLocality\":\"Auckland\",\"postalCode\":\"0627\",\"addressRegion\":\"Auckland\",\"addressCountry\":\"NZ\"},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-CN\",\"@id\":\"https:\\\/\\\/pureseo.com\\\/cn\\\/blog\\\/google-ends-support-robotstxt-noindex#local-main-organization-logo\",\"url\":\"https:\\\/\\\/pureseo.com\\\/wp-content\\\/uploads\\\/2019\\\/08\\\/Digital-Marketing-Company-NZ-Pure-SEO.png\",\"contentUrl\":\"https:\\\/\\\/pureseo.com\\\/wp-content\\\/uploads\\\/2019\\\/08\\\/Digital-Marketing-Company-NZ-Pure-SEO.png\",\"width\":4159,\"height\":2055,\"caption\":\"Pure SEO\"}]}<\/script>\n<meta name=\"geo.placename\" content=\"Auckland\" \/>\n<meta name=\"geo.region\" content=\"New Zealand\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Google Announces End of Support for Robots.txt Noindex Directives - Pure SEO","description":"Google announced that they will be discontinuing their support of noindex directives within robots.txt files. Find out if these changes affect you.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Google Announces End of Support for Robots.txt Noindex Directives","og_url":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex","og_site_name":"Pure SEO","article_publisher":"https:\/\/www.facebook.com\/pureseo","article_published_time":"2019-07-16T20:44:44+00:00","article_modified_time":"2019-08-14T21:01:27+00:00","og_image":[{"width":1920,"height":1080,"url":"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/07\/google-announces-end-of-support-for-robots-txt.jpg","type":"image\/jpeg"}],"author":"Rollan Schott","twitter_card":"summary_large_image","twitter_creator":"@Pure_SEO","twitter_site":"@Pure_SEO","twitter_misc":{"Written by":"Rollan Schott","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#article","isPartOf":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex"},"author":{"name":"Rollan Schott","@id":"https:\/\/pureseo.com\/cn#\/schema\/person\/411cce715efca3feeccb65335a1fd370"},"headline":"Google Announces End of Support for Robots.txt Noindex Directives","datePublished":"2019-07-16T20:44:44+00:00","dateModified":"2019-08-14T21:01:27+00:00","mainEntityOfPage":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex"},"wordCount":614,"commentCount":0,"publisher":{"@id":"https:\/\/pureseo.com\/cn#organization"},"image":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#primaryimage"},"thumbnailUrl":"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/07\/google-announces-end-of-support-for-robots-txt.jpg","articleSection":["Blog","News"],"inLanguage":"en-CN","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#respond"]}],"copyrightYear":"2019","copyrightHolder":{"@id":"https:\/\/pureseo.com\/#organization"}},{"@type":"WebPage","@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex","url":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex","name":"Google Announces End of Support for Robots.txt Noindex Directives - Pure SEO","isPartOf":{"@id":"https:\/\/pureseo.com\/cn#website"},"primaryImageOfPage":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#primaryimage"},"image":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#primaryimage"},"thumbnailUrl":"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/07\/google-announces-end-of-support-for-robots-txt.jpg","datePublished":"2019-07-16T20:44:44+00:00","dateModified":"2019-08-14T21:01:27+00:00","description":"Google announced that they will be discontinuing their support of noindex directives within robots.txt files. Find out if these changes affect you.","breadcrumb":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#breadcrumb"},"inLanguage":"en-CN","potentialAction":[{"@type":"ReadAction","target":["https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex"]}]},{"@type":"ImageObject","inLanguage":"en-CN","@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#primaryimage","url":"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/07\/google-announces-end-of-support-for-robots-txt.jpg","contentUrl":"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/07\/google-announces-end-of-support-for-robots-txt.jpg","width":1920,"height":1080,"caption":"Robots"},{"@type":"BreadcrumbList","@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Digital Marketing Agency","item":"https:\/\/pureseo.com\/cn"},{"@type":"ListItem","position":2,"name":"Blog","item":"https:\/\/pureseo.com\/cn\/blog"},{"@type":"ListItem","position":3,"name":"Google Announces End of Support for Robots.txt Noindex Directives"}]},{"@type":"WebSite","@id":"https:\/\/pureseo.com\/cn#website","url":"https:\/\/pureseo.com\/cn","name":"Pure SEO","description":"Digital Marketing Agency","publisher":{"@id":"https:\/\/pureseo.com\/cn#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/pureseo.com\/cn?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-CN"},{"@type":["Organization","Place"],"@id":"https:\/\/pureseo.com\/cn#organization","name":"Pure SEO","url":"https:\/\/pureseo.com\/cn","logo":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#local-main-organization-logo"},"image":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#local-main-organization-logo"},"sameAs":["https:\/\/www.facebook.com\/pureseo","https:\/\/x.com\/Pure_SEO","https:\/\/www.linkedin.com\/company\/pureseo\/","https:\/\/www.youtube.com\/channel\/UCinrMPUg7xZncotAkegPvEA"],"address":{"@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#local-main-place-address"},"telephone":[],"openingHoursSpecification":[{"@type":"OpeningHoursSpecification","dayOfWeek":["Monday","Tuesday","Wednesday","Thursday","Friday","Saturday","Sunday"],"opens":"09:00","closes":"17:00"}]},{"@type":"Person","@id":"https:\/\/pureseo.com\/cn#\/schema\/person\/411cce715efca3feeccb65335a1fd370","name":"Rollan Schott","image":{"@type":"ImageObject","inLanguage":"en-CN","@id":"https:\/\/secure.gravatar.com\/avatar\/d995014ad341cf72f26d46d922532cac9c3fc7d6d8b577bfda98c798a0075820?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/d995014ad341cf72f26d46d922532cac9c3fc7d6d8b577bfda98c798a0075820?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/d995014ad341cf72f26d46d922532cac9c3fc7d6d8b577bfda98c798a0075820?s=96&d=mm&r=g","caption":"Rollan Schott"},"description":"Rollan Schott is a copywriter with Pure SEO. Rollan was born and raised in the United States, having moved to New Zealand after 4 years teaching and writing in Asia. When he's not churning out quality content at breakneck speed, Rollan is probably busy writing the next great American novel. He may also be idly watching true crime documentaries in his Auckland Central apartment with his wife, Lauren. The latter is more likely than the former.","sameAs":["https:\/\/www.linkedin.com\/in\/rollanschott\/"],"url":"https:\/\/pureseo.com\/cn\/author\/rollan"},{"@type":"PostalAddress","@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#local-main-place-address","streetAddress":"35 Kawana Street, Northcote","addressLocality":"Auckland","postalCode":"0627","addressRegion":"Auckland","addressCountry":"NZ"},{"@type":"ImageObject","inLanguage":"en-CN","@id":"https:\/\/pureseo.com\/cn\/blog\/google-ends-support-robotstxt-noindex#local-main-organization-logo","url":"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/08\/Digital-Marketing-Company-NZ-Pure-SEO.png","contentUrl":"https:\/\/pureseo.com\/wp-content\/uploads\/2019\/08\/Digital-Marketing-Company-NZ-Pure-SEO.png","width":4159,"height":2055,"caption":"Pure SEO"}]},"geo.placename":"Auckland","geo.region":"New Zealand"},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/posts\/15549","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/comments?post=15549"}],"version-history":[{"count":0,"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/posts\/15549\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/media\/22428"}],"wp:attachment":[{"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/media?parent=15549"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/categories?post=15549"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/tags?post=15549"},{"taxonomy":"ai","embeddable":true,"href":"https:\/\/pureseo.com\/cn\/wp-json\/wp\/v2\/ai?post=15549"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}