Google dropping supporting for noindex, nofollow and crawl-delay directives


Posts: 13,947   +1,770

In short, if you mention crawl-delay, nofollow, and noindex in your robots.txt file - Google on September 1, 2019 will stop honoring it. They currently do honor some of those implementations, even though they are "unsupported and unpublished rules" but will stop doing so on September 1, 2019.

Google may send out notifications via Google Search Console if you are using these unsupported commands in your robots.txt files.