Conflicting index signals: a page that says index and do not index

What conflicting index signals mean

A sitemap entry says “index this page”. A noindex directive, a canonical pointing at another URL and a robots.txt block each say the opposite. When a page carries the first and two or more of the others, the signals contradict each other and a search engine has to pick one. Which one it picks is not something you control.

The individual rules each report one symptom and each suggests a fix that pulls in a different direction: drop the noindex, drop the canonical, drop the sitemap entry. This rule reports the combination once, so you can decide first whether the page should be in search and then fix every signal to match.

What squirrel checks

The rule is site-wide. It collects every <loc> from every discovered sitemap, then walks the crawled pages that answered with a 2xx status. A page is reported only when it is listed in a sitemap and at least two of these “do not index” signals apply:

• noindex in the robots meta tag.
• noindex in the X-Robots-Tag response header.
• A canonical (meta tag or header) that points at a different URL. The comparison ignores the fragment and a trailing slash.
• A robots.txt Disallow that matches the page URL.

These are not counted: a self-canonical, dofollow internal links and hreflang. Nearly every page has the first two, so counting them would turn every single mismatch into a conflict, and hreflang needs data from other pages. A page with only one “do not index” signal is left to crawl/noindex-in-sitemap or core/canonical, so one mismatch is never reported twice.

Results:

• Skipped when no sitemap was discovered. Message: No sitemap discovered, so no page can be listed in one.
• Skipped when no page data is available. Message: Page data not available.
• Warn when at least one page conflicts. Message: N page(s) say both "index me" and "do not index me". Decide whether each page should be indexed. followed by the two fixes: keep the page out of search (remove it from the sitemap, keep noindex) or put it in search (remove noindex and any robots.txt block, point the canonical at the page itself, keep it in the sitemap). Each page is listed with the signals found, up to 50 pages, and the finding names the related rules. Severity warning, weight 4.
• Info check site-wide-noindex when the whole site is closed: at least 80% of at least three crawled pages carry a noindex or a robots.txt block. Message: N of M crawled pages are kept out of the index (noindex or robots.txt): this looks like a site-wide block (a staging site or a pre-launch site), so per-page conflicts are not reported. The site is reported once and no page is listed. See crawl/all-noindex-pages.
• Pass otherwise. Message: No page sends contradictory indexing signals.

crawl/noindex-in-sitemap keeps reporting every noindexed page in a sitemap. For the pages both rules find, its finding carries alsoReportedBy and alsoReportedPages details pointing here, so a report can group them.

How to fix it

Decide first, then make every signal agree.

To keep the page out of search:

<!-- remove the URL from sitemap.xml, keep the noindex -->
<meta name="robots" content="noindex, follow">

To put the page in search, remove the noindex meta tag and the X-Robots-Tag header, remove the robots.txt Disallow that matches it, point the canonical at the page itself, and keep it in the sitemap:

<link rel="canonical" href="https://example.com/this-page">

A robots.txt block also stops crawlers reading the noindex on the page, so removing the block is the first step when a page must drop out of the index.

Enable / disable

Disable this rule

[rules]
disable = ["crawl/conflicting-index-signals"]

Disable all Crawlability rules

[rules]
disable = ["crawl/*"]

Enable only this rule

[rules]
enable = ["crawl/conflicting-index-signals"]

enable replaces the default ["*"], so this is the only rule selected. Leave disable out: a rule that matches any disable pattern is skipped even when enable names it. Only an enabled setting under a rule’s own [rule_options] table overrides both lists.

Check names by outcome

Each check name this rule can emit, by what it means to a tool comparing two audits. A problem check is a finding. A clean check says the rule ran over what it covers and found nothing; it clears the problem checks listed beside it, so a finding is fixed when the rule ran over the same scope and a clean check took its place. A not checked check says the rule did not evaluate, and never clears a finding.

Related rules

• crawl/noindex-in-sitemap: the noindex half of the contradiction on its own.
• core/canonical: the canonical half, checked per page.
• social/og-url-match: an og:url that disagrees with the canonical.
• schema/coverage-outlier: a page that lacks the structured data its siblings carry.
• crawl/robots-meta-conflict: noindex on a page robots.txt already blocks, so nobody reads it.
• crawl/all-noindex-pages: the full list of blocked pages.

Crawlability findings ship in every audit next to the SEO, performance and agent experience rules. See The Screaming Frog alternative for agents for how an agent works through a report.

References

• Block search indexing with noindex, Google Search Central
• Consolidate duplicate URLs, Google Search Central

Check your site

Run squirrel audit https://example.com and open the Crawlability section of the report. Every page with conflicting signals is listed with its address and the signals found. Signed-out local audits need no account and cost nothing. Signed-in audits spend credits.