Broken internal links: 404s inside your own site

What a broken internal link is

A broken internal link is an <a href> on your site pointing at a URL on your site that answers with an error. The visitor hits a 404, and a crawler spends a request learning the same thing.

Internal links break quietly. A page is renamed, a section is restructured, a product is retired, and the links that pointed at it are left behind in body copy, in old posts and in hard-coded navigation. Nothing warns you, because the link still looks like a link.

What squirrel checks

The rule is site-wide and compares link targets against pages the crawl actually fetched. It collects every internal link from every crawled page, normalises both sides, and looks up the recorded status code. Targets the crawl never visited are skipped rather than guessed at.

• Skipped when there is no site data. Message: Site data not available for broken link analysis.

• broken-links is skipped when the crawl did not follow links and some link targets were fetched neither by the crawl nor by the internal link check, with skip reason link-targets-not-fetched. Message: N of M internal link target(s) were not fetched because this audit did not follow links and <reason>, so they were not checked for broken links. The reason names what left them out:

  • internal link checks were off.
  • internal_links.max_targets is 0.
  • its internal link checks stop at <limit> targets, most-linked first: more targets were eligible than the limit.
  • its internal link checks never request targets outside the crawl scope, disallowed by robots.txt, or that look like actions (log out, unsubscribe) or carry a token.
  • the internal link checks could not be trusted, as the site answers status-only (HEAD) requests with errors on pages it serves.
  • its internal link checks got no answer.
  • With more than one cause, its internal link checks left them out (<counts>). <counts> lists, separated by ; , one count for each cause that applies: <count> past the limit of <limit> targets (or <count> because internal_links.max_targets is 0), <count> skipped as outside the crawl scope, disallowed by robots.txt, or action or token links (an action or token link for one), <count> not trusted, because the site answers HEAD requests with errors on pages it serves, and <count> with no answer.

  details.notFetched carries the same counts (off, overLimit, skipped, untrusted, noAnswer, each present only when not zero), and they add up to N. It replaces the pass, and sits next to the failure when a target the crawl did fetch is broken. The report shows it as Not checked, and the score does not count it. Its advice suggests turning internal link checks back on only when they were off, and raising internal_links.max_targets only when the limit left targets out. When every unfetched target was skipped, it says that a Surface or Full audit, which follows links, can reach the ones inside the crawl scope and allowed by robots.txt.

• broken-links fails when a target returned 400 or above. Message: 7 broken internal link(s) (4xx/5xx). Each item reads https://example.com/old-page (404) and lists every page that links to it. Severity error, weight 7.

• broken-links passes otherwise. Message: No broken internal links detected.

• Cloudflare email-protection links are skipped. With Cloudflare’s Email Address Obfuscation on, every mailto: link and every email address in a page’s text becomes a link to /cdn-cgi/l/email-protection, and a script in the browser puts the address back. Requested directly, that path answers an error, but it is an email address, not a page. So it is never a link target: the crawl and the internal link check never request it, and the rule never reports it as broken. squirrel reads it as the mailto: link it stands for instead, so rules that look for contact links still see it. Other /cdn-cgi/ links, such as Cloudflare image resizing URLs, are checked as usual.

• rate-limited-links reports info for targets that returned a rate-limit status, to the crawl or to the internal link check. Message: N internal link(s) rate-limited - status unverifiable.

• refused-links reports info for targets whose status cannot be verified: the internal link check got a 401 or 403, or the crawl got a bot-wall challenge. Message: N internal link(s) <reason> - status unverifiable, where the reason is refused the link check (401/403), answered the crawl with a bot-wall challenge, or both joined with or. An item from a challenge carries wafChallenge: true.

Every broken-links row, whether it fails, passes or is not checked, carries the same details about what the audit covered:

• linkTargets: distinct internal link targets on the audited pages.
• probedTargets: targets the internal link check got a status for.
• unfetchedTargets: targets whose status the rule does not know: neither the crawl nor the internal link check gave one. A linked page the crawl fetched that is not HTML, such as a PDF, keeps the status the crawl got, so a PDF that loaded is not broken and one that answered 404 is. A rate-limited target is listed in rate-limited-links, and a bot-wall challenge in refused-links, instead of counting here.
• emailProtectionLinks: Cloudflare email-protection links on the audited pages, which the rule left out, counted once for each link on each page rather than once per address. Present only when there were any. A page carries at most 32 of these links, so a page with more counts 32.
• notFetched: why those targets were left out, as counts that add up to unfetchedTargets. off means internal link checks were off. overLimit means past internal_links.max_targets. skipped means never requested by design: outside the crawl scope, disallowed by robots.txt, or an action or token link. untrusted means the site answers HEAD requests with errors on pages it serves. noAnswer means no status came back, or none the rule could read. Each count is present only when it is not zero, so a row with nothing unfetched has an empty notFetched.

Read these before comparing two audits’ broken links. A link missing from this audit’s failure was fixed only if this audit got a status for it. A target that was broken last time and unfetched this time is unknown, not fixed, and so is one listed in rate-limited-links or refused-links. Targets counted as skipped are left out the same way on every audit with the same settings, the same robots.txt and the same links. These counts are kept on every broken-links row a published report carries. A published report over its size limit can leave a row out, and then the coverage is unknown.

Normalisation strips the query, the fragment and the trailing slash and lowercases the scheme and host, but not the path, so a case difference in the path is treated as a different URL. A throttled response means the host limited the crawl, not that the page is missing, so those are separated out before the 400 test.

Which audits can find a broken link

Every audit level can. The rule reads a target’s status from the crawl when the crawl fetched it, and otherwise from the internal link check: a status-only HEAD request for each internal link target the crawl did not fetch, up to 200 per audit, most-linked first. No render, no rules, and no credits. It stays inside the crawl scope ([crawler] include and exclude) and, when the crawl respects it, robots.txt, uses the crawl’s user agent and per-host concurrency, sends no cookies, credentials or custom headers, and never requests a link that looks like an action or carries a token: a logout, unsubscribe, confirm or add-to-cart path, a query such as ?action=delete or ?token=, or a rel="nofollow" link whose text is “Log out” or “Delete”. See What the internal link check never requests. A target that answers this check with 401 or 403 is reported in a refused-links info check as unverifiable, never as broken. That is what lets Quick, which does not follow links, find a 404 that only a link points at, and what catches a broken page past a Surface or Full page budget. A target found broken this way carries probed: true in its item.

With --no-internal-links or [internal_links] enabled = false, or past the 200, a crawl that did not follow links (Quick, the Sitemap crawl strategy) reports the targets it could not see as Not checked instead of passing, and an open broken link finding stays open until an audit that can see them runs. Set [internal_links] max_targets to change the limit. A site with no usable sitemap has its links followed even at Quick.

How to fix it

Update the link to the current URL, or redirect the old URL permanently and leave the link alone. Prefer fixing the link where you can: a redirect works but costs a hop on every click.

Fix the source, not just the page. The report lists which pages carry each broken link, and a link repeated in a template needs one change rather than twenty.

Enable / disable

Disable this rule

[rules]
disable = ["links/broken-links"]

Disable all Links rules

[rules]
disable = ["links/*"]

Enable only this rule

[rules]
enable = ["links/broken-links"]

enable replaces the default ["*"], so this is the only rule selected. Leave disable out: a rule that matches any disable pattern is skipped even when enable names it. Only an enabled setting under a rule’s own [rule_options] table overrides both lists.

Related rules

• links/broken-external-links: the same check for links leaving your site.
• links/redirect-chains: internal links that resolve, but through a redirect.
• crawl/soft-404: pages that answer 200 while showing an error.
• crawl/sitemap-4xx: dead URLs you are also submitting in a sitemap.

Links findings ship in every audit next to the SEO, performance and agent experience rules. See Fix SEO issues with AI for how an agent works through a report.

References

• HTTP status codes and Google Search
• Make your links crawlable, Google Search Central

Check your site

Run squirrel audit https://example.com and open the Links section of the report. Every broken link is listed with its status code and the pages that link to it. Signed-out local audits need no account and cost nothing. Signed-in audits spend credits.