Soft 404: error content served with an HTTP 200 status

What a soft 404 is

A soft 404 is a page that tells a human the content is missing while telling a machine that everything is fine. The body says “page not found” and the response status is 200 rather than 404 or 410.

Crawlers have no reason to stop requesting a URL that answers 200, so soft 404s consume crawl budget indefinitely and can be indexed as thin or duplicate content. Search engines try to detect them heuristically, which means you lose control of whether the URL is dropped and when. Single-page applications produce them easily, because the shell renders successfully and only the client-side route resolves to nothing.

What squirrel checks

The signal itself is computed during parsing from the status code and the page content. The rule surfaces it and emits a single check named soft-404:

• Pass when the page is not a soft 404. Message: Page does not serve 404 content with a success status.
• Warn when it is, with the wording chosen by an end-of-crawl confirmation re-fetch. Severity warning, weight 5.

Every warn message ends with the same sentence: This check reads the server's HTML response and can differ from what a browser shows. The part before it depends on the confirmation verdict:

• Confirmed: Page serves 404 content with HTTP 200.
• Intermittent, meaning the re-fetch returned real content: Page intermittently serves 404 error content with HTTP 200.
• Unconfirmed: Page serves 404 content with HTTP 200 (based on a single crawl observation).
• Unconfirmed after a rendered observation: Page serves 404 content with HTTP 200 (based on the rendered crawl observation; cannot verify without JS rendering).

The value is the status code as HTTP 200, and each detection signal is listed as an item with its own detail. The intermittent case matters on sites where a CDN or incremental regeneration occasionally serves the error shell for a real page.

A page is picked for the confirmation re-fetch the same way in every audit mode, including the large-crawl mode that releases each page’s document as it goes: the error-shell marker on the document root is recorded while the page is parsed, so it still counts after the document is gone.

How to fix it

export async function loader({ params }) {
  const post = await getPost(params.slug);
  if (!post) throw new Response("Not Found", { status: 404 });
  return post;
}

Return the real status from the server. For a client-rendered app, that means the server route has to know the resource is missing, which usually means resolving it during server rendering rather than after hydration.

If the URL is valid and the finding is wrong, the page is emitting error-shell markup: a title or heading saying not found, or an error template that renders alongside the content.

Enable / disable

Disable this rule

[rules]
disable = ["crawl/soft-404"]

Disable all Crawlability rules

[rules]
disable = ["crawl/*"]

Enable only this rule

[rules]
enable = ["crawl/soft-404"]

enable replaces the default ["*"], so this is the only rule selected. Leave disable out: a rule that matches any disable pattern is skipped even when enable names it. Only an enabled setting under a rule’s own [rule_options] table overrides both lists.

Related rules

• crawl/indexability: whether the same page is also blocked from indexing.
• links/broken-links: internal links pointing at pages that fail outright.
• crawl/sitemap-4xx: sitemap entries returning a real 4XX status.
• content/word-count: the thin-content side of the same page.

Crawlability findings ship in every audit next to the SEO, performance and agent experience rules. See The Screaming Frog alternative for agents for how an agent works through a report.

References

• HTTP status codes and Google Search
• Managing crawl budget for large sites, Google Search Central

Check your site

Run squirrel audit https://example.com and open the Crawlability section of the report. Every soft 404 is listed with the signals that identified it. Signed-out local audits need no account and cost nothing. Signed-in audits spend credits.