noindex

noindex is a rule that tells search engines not to show a page in their results. How to set it, why the page must stay crawlable, and when to use it.

noindex is a rule that tells search engines not to include a page in their results. When Googlebot crawls a page and finds the rule, Google drops the page from its search results entirely, even if other sites link to it.

There are two ways to set it, with the same effect: a robots meta tag in the page's head, such as a meta element named robots with the content noindex, or an X-Robots-Tag HTTP response header, which also works for files such as PDFs. Google doesn't support noindex inside robots.txt.

One mistake undoes it: blocking the same page in robots.txt. Google's documentation is clear that for noindex to work, the page must not be blocked by robots.txt and must be accessible to the crawler. If Google can't fetch the page, it never sees the rule, and the page can still appear in results when other pages link to it.

Typical uses are pages that should exist for visitors but not in search: thank-you pages after a form, internal search results, thin tag or archive pages, and drafts. For anything private, use a password instead. After you remove a noindex rule, Google has to crawl the page again before it can be indexed.

Related terms

Sources

  1. Block Search indexing with noindex
  2. Robots meta tag, data-nosnippet, and X-Robots-Tag specifications

All terms