Broken external links: link rot on pages you cannot fix

What link rot is

An external link points at a site you do not control, so it can break at any time and you will not be told. Studies of link rot put the half-life of a web citation in the low single-digit years.

Not every failed check is a dead link. A 403 from a web application firewall means the automated request was refused; whether a person would reach the page is unverified either way. A 429, or a 503 carrying Retry-After, is treated as throttling, though a 503 can equally be scheduled maintenance. squirrel classifies both as unverifiable rather than broken, because reporting them as broken sends you chasing links that work. The same goes for any wall a link check meets from your own machine: a sign-in or bot wall (401, 403, 405, 406, LinkedIn’s 999), a challenge page served as a 503, or a site that never answers in time.

What squirrel checks

The rule is site-wide and reads the results of the external link check. It emits these checks:

• broken-external-links is skipped when external link checking did not run for the audit, with skip reason external-links-not-checked. Message: External link checking was off for this audit, so 12 external link(s) on the audited pages were not checked, counting the distinct external links on the audited pages. It is not a pass: the report shows it as Not checked, and the score does not count it.
• broken-external-links is skipped the same way when checking was on but left no result to read: No external link check results for the 3 external link(s) on the audited pages, so they were not checked when there are no results at all (for example when squirrel analyze re-runs the rules on a crawl whose link check never ran), and None of the 5 external link(s) could be checked this audit when every link is still unresolved.
• broken-external-links passes when checking ran and the audited pages have no external links. Message: No external links found to check.
• broken-external-links warns when links failed. Message: 4 broken external link(s): 3 with 404, 1 failed, grouped by status code with failed covering fetch errors. Each item reads https://other.example/page (404), or (Error: …) with the reason a fetch failed (a domain that does not exist, a refused connection), with the pages that link to it. Severity warning, weight 5.
• broken-external-links passes otherwise. Message: All 128 external link(s) are working.
• waf-blocked-external-links reports info for links that could not be verified either way. Message: {count} external link(s) could not be verified - status unverifiable. Each item reads {url} (unverified), or {url} (unverified: {vendor}) when the checker named a vendor, or {url} (unverified: awaiting confirmation) for a link that answered 404 once and waits for its confirming check: that one is likely broken rather than blocked. The item’s meta.unverifiedReason says why when the checker knew: blocked (bot protection or a challenge refused the check), unreachable (a timeout or failed connection) or awaiting-confirmation. When you are logged in, the list holds the links the cloud link checker could not verify: refused by bot protection, an error page from the CDN between the checker and the site (a 520 to 530), a timeout or failed connection from the checker’s network, or a 404 seen once and waiting for a second check to confirm it. A vendor named on one of those is a best guess, not proof that the site runs it. When squirrel checks a link itself rather than through the cloud link checker (logged out, or a link the cloud checker did not answer in time), the list also holds every link that answered 401, 403, 405, 406 or 999 whether or not a firewall was identified, a 503 challenge page, and a link that timed out. Three of those answers are the site’s own and stay broken: an object store’s 403 for a missing file (Amazon S3 or Google Cloud Storage, recognised by its headers or its XML error body), an HTTP sign-in prompt (WWW-Authenticate naming Basic, Digest, Negotiate or NTLM), and a 405 whose Allow header leaves out GET. A logged-out audit with any of these says so after the audit and suggests logging in, where the cloud link checker can usually get a real answer.
• rate-limited-external-links reports info for throttled targets. Message: 2 external link(s) rate-limited - status unverifiable.
• not-requested-external-links reports info for external links squirrel never requested because they look like actions or carry a token, such as a one-click unsubscribe link or a signed download URL (see What the external link check never requests). Message: N external link(s) were not requested because they look like actions (log out, unsubscribe) or carry a token or signature in the URL, so their status was not checked, listing up to 50 of them, most-linked first, with up to 10 of the pages that link to each and pageCount when there are more. Each listed URL has its query values, token-like query keys, token- or id-like path segments and path segments holding an email address masked as …, a best effort to keep the credential out of the report, and links that mask to the same URL are listed once with their pages merged. They are neither broken nor working, and they do not count against the per-audit limit.
• refused-external-links reports info for external links squirrel never requested because the page points them at a link-local or cloud metadata address (on any audit), or at a private or loopback address or a name public DNS does not resolve, such as myapp.test (on a public site’s audit; a private site’s links to its own network are checked). Message: N external link(s) point at a private, link-local or cloud metadata address, or a name public DNS does not resolve, so they were not requested and their status was not checked, listing up to 50 of them the same way. They are neither broken nor working. A link to a local-only name such as myapp.test or box.local is also reported as a warning by content/dev-leakage, as a development address on a production page. One refused on its URL alone (a private or metadata IP literal, localhost, a metadata name) takes no slot under the per-audit limit. See What the external link check never requests.

A link counts as broken when the fetch errored, or the status is 400 or above and it is neither unverifiable (a wall or a timeout, above) nor rate-limited.

Which audits check external links

External links are checked at every audit level, Quick included. They are not checked with --offline, with --no-external-links, with [external_links] enabled = false, or in a cloud audit whose website or run has external link checks off. In those audits the rule reports Not checked rather than a pass, every report format says so with the reason, and an open broken external link finding stays open until an audit that checks links runs.

One audit requests at most its page budget or 200 distinct external links, whichever is more, starting with the links on the most pages (the nav and footer links every page shares). The rest are reported in a second broken-external-links check, skipped with reason external-links-over-cap. Message: 40 of 240 external link(s) were not checked: over this audit's limit of 200, which checks the most-linked first. The second number leaves out the links never requested as actions or token links, or refused on their URL alone, which are not over any limit; details.externalLinksFound counts every external link. Set [external_links] max_links to change the limit. A cloud audit is charged for at most one external link per page of its budget, whatever the limit. It is charged only for the links it could verify: an unverifiable or rate-limited link costs nothing (see Credits and pricing).

A link to another of the site’s own domains, such as example.co.uk linking to example.com, is an external link. It is checked and reported like any other, and charged only when it can be verified.

A cloud audit also gives the link check a time budget, and waits only so long on any one request. Links still without an answer when either runs out are reported in a broken-external-links check skipped with reason external-links-out-of-time. Message: N of M external link(s) were not checked: the link check ran out of time before reaching them, where M is the number of links the check selected. They are neither broken nor working, and an open broken external link finding stays open until an audit checks them. The CLI sets no time budget.

How to fix it

Replace the link with a working equivalent, point it at an archived copy, or remove it. For a reference that matters to the argument, host your own copy of the material rather than depending on someone else’s uptime.

Check the unverifiable and rate-limited lists by hand before touching them. Those links usually work in a browser.

Enable / disable

Disable this rule

[rules]
disable = ["links/broken-external-links"]

Disable all Links rules

[rules]
disable = ["links/*"]

Enable only this rule

[rules]
enable = ["links/broken-external-links"]

enable replaces the default ["*"], so this is the only rule selected. Leave disable out: a rule that matches any disable pattern is skipped even when enable names it. Only an enabled setting under a rule’s own [rule_options] table overrides both lists.

Related rules

• links/dead-links: the cloud service that checks these links against a shared cache.
• links/broken-links: the same check for links inside your own site.
• links/external-links: how many external links each page carries.
• security/new-tab: the rel attributes those links should have.

Links findings ship in every audit next to the SEO, performance and agent experience rules. See Fix SEO issues with AI for how an agent works through a report.

References

• Make your links crawlable, Google Search Central
• HTTP status codes and Google Search

Check your site

Run squirrel audit https://example.com and open the Links section of the report. Every failing link is listed with its status and the pages that link to it. Signed-out local audits need no account and cost nothing. Signed-in audits spend credits.