Auspex.

Tools / Sitemap check

Does your sitemap offer what the index would keep?

A sitemap is a list of the URLs you would like kept. Every entry that redirects, returns an error, or carries a noindex or a canonical pointing elsewhere is a contradiction: the site offers a page and then takes it back. This check finds the sitemap your site names, reads what it declares, and fetches ten declared URLs as Googlebot to see whether each is as declared.

Sends up to sixteen requests to the site: robots.txt, the sitemap, up to four files of a sitemap index, and ten declared URLs, each fetched once as Googlebot with the first 256 KB read. Gzip sitemaps are decompressed. Redirects to another host are reported, not followed. Four checks a minute per visitor, three a minute per site.

What as declared means

The address returned 200 with a page, carried no noindex in its HTML or its headers, and either named itself as the canonical or declared none. That is what a sitemap entry asserts. Google's sitemap documentation asks for canonical URLs only, and treats lastmod as a signal it reads only while the dates prove accurate, so a sitemap that stamps every URL with today's date has told the engine to ignore the field.

As declared is not indexed. Submitted, crawled and indexed are three stages and three filters. This check reads the first: whether the site's own list is true. Whether the index holds a URL is Search Console's reading, reconciled against a crawl, and the audit does that for every declared URL.

The other checks

The crawl check asks what your site returns to each named crawler, and the identity check asks whether the crawler in your logs was the crawler at all. The index check reads one URL the way this check reads ten, in more detail.