Failed audits

An audit fails when the crawl finishes without a single auditable page, so there is no score and no issue list. A cloud audit can also fail after its crawl, on squirrelscan’s side: see something failed on our side, the audit ran out of time and the audit stalled.

Every failed audit names its cause. The cause appears as a short sentence in the report, in the failure email, in the in-app notification, in squirrel audit output, and on the get_audit_status MCP tool. The JSON report and the MCP tool also carry a machine-readable code. For a cloud audit, troubleshooting by cause has one section per cause, and the failure email and the audit’s run page link to the section that applies.

Blocked is not failed

If your site actively refused the crawler on the audited address, with a 401, a 403, a rate limit (429, 430, or a 503 carrying Retry-After) or a bot protection challenge served as a 503, and no page could be audited, the audit is marked blocked, not failed. That is a separate state with its own guidance: let the squirrelscan crawler through your WAF or bot protection without switching the protection off (see bot protection or WAF challenge and Firewall and WAF exceptions), get it past a login (see sign-in page or access gateway), or run the CLI from a trusted network. A blocked audit is refunded the same way as a failed one.

What a blocked report says about your root files

A site that refuses the crawler has not told us whether its robots.txt, sitemaps, llms.txt or Markdown responses exist. A blocked or failed audit therefore reports none of them as missing, scores no category, and lists the requests the site refused, with the bot protection vendor when the response names one:

Refused by the site (not checked, so not reported as missing):
  /robots.txt: HTTP 403 (Cloudflare)
  sitemap, 8 requests (first /sitemap.xml): HTTP 403 (Cloudflare)

The JSON report carries the same list as refusedFetches, one entry per refused request with its url, resource, status and provider. The list is capped at 20 requests. A partly blocked audit that still read pages reports the refused root files as not checked, rather than missing, in the same way.

The failure reasons

A cloud audit has two more codes of its own: audit_timeout, when the audit reached its overall time limit, and stalled, when it stopped making progress. Both are on squirrelscan’s side; see the audit ran out of time and the audit stalled.

Reading the reason

In the CLI, a failed audit prints the reason and the next step:

AUDIT FAILED
https://example.com - DNS lookup failed for example.com: NXDOMAIN
Check the domain's DNS records and that the hostname resolves from the public internet.

The JSON report carries both the sentence and the code, so a CI job can branch on the cause:

{
  "status": "failed",
  "statusReason": "DNS lookup failed for example.com: NXDOMAIN",
  "statusReasonCode": "dns"
}

The llm output states the same thing for an agent:

<status state="failed" reason="DNS lookup failed for example.com: NXDOMAIN">
  The hostname did not resolve. Check the domain's DNS records, and whether the domain has expired or was recently moved.
</status>

Checking it yourself

A failed cloud audit is worth reproducing before you change anything. Two checks cover most causes.

The first repeats the cloud’s page request with curl. Cloud page fetches send a desktop or mobile browser user agent, so send one too, or a firewall that treats curl differently gives you a different answer:

UA='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36'
curl -sS -o /dev/null -D - -A "$UA" https://example.com/

-D - prints the response headers. A working site answers HTTP/2 200 (or HTTP/1.1 200 OK). Each section below says what to look for when it does not. Add the same -H 'Name: value' options as your website’s custom request headers to see what an audit with those headers gets.

The second runs the audit with the CLI, which fetches from your network. Signed in, the CLI renders pages in the cloud browser by default, so add --http to keep every request on your network:

squirrel audit https://example.com --max-pages 1 --http

If the local run succeeds and the cloud run fails with connection or http_4xx, your origin or its CDN is treating requests from squirrelscan’s network differently from yours.

How a cloud audit requests your site

Every cause below refers to the same sequence of requests. These terms are used throughout:

• Audited address: the URL the audit was started with.
• Origin: the scheme, host and port of a URL together. http://example.com and https://example.com are different origins, and so are example.com and www.example.com.
• Same site: two addresses whose hosts share a registrable domain and whose ports match, such as example.com, www.example.com and staging.example.com. The scheme may differ. Hosts on a shared hosting domain are separate sites: a.vercel.app and b.vercel.app are not the same site, and neither is example.vercel.app and example.com.
• Attempt: one request, or one browser render, with its own time budget.
• Budget: how long an attempt may take before it is abandoned. Where the run recorded them, the run page lists an attempt’s elapsed time and its budget.

The requests, in order

Step 1 decides the crawl’s base. When the seed redirect probe lands on the same site, the crawl starts there. When it lands on a different site, the crawl keeps the audited address and fetches it in step 3, which follows the same redirects again: if they still end on the other site, the audit fails as a redirect to another site. Step 1 never sends your custom request headers, so a site that only lets requests with those headers through always redirects or refuses in step 1. That on its own does not fail the audit.

How the entry page is fetched

Which request serves the entry page depends on the website’s Rendering setting, under Settings → Crawl in the audit level card (see crawl settings). An audit started through the API or the MCP server with a level of its own uses that level’s rendering instead: Quick renders only the pages that need JavaScript, Surface and Full render every page.

Which of these requests carry your custom request headers, under each setting, is set out in rendering and custom headers.

On a plain request, custom request headers are removed at the first redirect to a different origin, and stay removed for the rest of that redirect chain. Only User-Agent, Accept, Accept-Language and Upgrade-Insecure-Requests are kept. A redirect from http:// to https://, or from example.com to www.example.com, is enough to drop them, so audit the exact origin that serves the page. A browser render follows redirects inside the browser, including a page’s own script redirects, and reports where it landed.

How each response is handled

When nothing auditable was fetched, a 401, a 403, a rate limit or a 503 challenge makes the audit blocked. Every other cause makes it failed.

Troubleshooting by cause

A failed cloud audit keeps a record of what it tried: the first attempts at the site, how long each waited against its budget, what came back, and whether our own rendering service was working at the time. The failure email and the audit’s run page in the dashboard turn that record into three answers: what failed, which side the evidence points at, and what to try next. Each cause below has its own link, and the email and run page link to the one that applies.

Each cause section says what the crawler observed, how the cause is classified, which side it is attributed to, how to check it from your side where you can, and what to do. The curl commands set the browser user agent from $UA, defined in checking it yourself.

Whose side it was on

The run page says which side the evidence points at, and it only goes as far as the evidence does.

When a browser render had not answered by the time the plain request took over, the run page also says that a part for us cannot be fully ruled out, even when the attribution is the site.

Audits that failed before we started keeping these records only have the short reason, and the run page says that its details are limited.

To check a result from your side, the run page has a Technical details section listing the attempts we recorded, up to a dozen: the browser render, the direct fetch and the crawler’s verdict, each with the status or error, the time it took and its budget where recorded, and the origin a redirect pointed to (never its path or query). When there were more, it says how many were left out. When your site sits behind Cloudflare, an attempt shows the response’s cf-ray id when it was recorded, which you can search for in your Cloudflare security events.

Refunds and retries

Failed and blocked audits are refunded automatically and in full, whichever side the cause was on. The refund covers every credit charged against the run. It applies to every audit run on your organization’s credits, whether it started from the dashboard, a schedule, the API, the MCP server or a signed-in squirrel audit, that ends failed, blocked or cancelled and delivered either no report or only a report that is itself the failure notice. An audit that delivered a completed report keeps its charge, including a partial audit that stopped early, which is charged for the pages it audited as described in credits and pricing.

The email and the run page show what our credit records hold for that run: refunded, not charged at all, or charged with the refund still to land. If a charge is still there later, send us the support reference from the run page.

Once an audit has failed, nothing runs it again for you unless the site has recurring audits on:

• Recurring audits on. The next scheduled audit runs at its usual time. If the cause is still there, it fails the same way.
• Recurring audits paused. A schedule pauses itself after three failed audits in a row, or when the balance cannot cover a run (see when a schedule pauses itself). Nothing runs until you turn it back on under Settings → Schedule.
• No schedule. The audit runs again only when you start it.

Either way, once you have fixed the cause, Run New Audit on the run page starts a new one straight away.

When a refund lands

1. At the failure. The refund is written when the run is recorded as failed, blocked or cancelled, right after that status is saved. It normally lands within seconds of the failure.
2. If that write did not complete. For a run that delivered no report at all, a sweep that runs every 15 minutes picks up the run once it has been over for 15 minutes and still holds charges, and refunds what is left. It covers runs started in the last 7 days.
3. Where to see it. The failure email and the in-app notification state what the credit records held when they were written, which can be a moment before the refund. The run page reads the records each time it opens, so it shows the current state. The Credit log under Settings → Usage lists the charge and the refund as separate entries, with the run id on each; its Refunds filter shows only refunds.

If the run page still shows a charge half an hour after the failure, contact support with the run page’s reference.

DNS lookup failed

What the crawler observed. The resolver answered that the hostname does not exist (ENOTFOUND, EAI_NONAME, NXDOMAIN), or the lookup did not complete (EAI_AGAIN, a timeout). No connection was attempted. After the failure, the hostname is looked up once more, with a 2-second limit.

How it is classified. dns. The follow-up lookup is the witness: when it says the name does not exist, the cause is DNS even if the first failure was reported as a refused connection or with no recognisable error at all.

Attribution. The site when the follow-up lookup confirms the name does not exist. Unknown when it does not (it timed out or failed, or the name resolved a moment later), because a passing resolver problem on either side looks the same; the run page then says so. Both when our rendering service also failed during the run, whatever the lookup said.

How to check.

dig +short example.com A
dig +short example.com AAAA
dig example.com | grep status

A working name prints at least one address and status: NOERROR. A missing name prints no address and status: NXDOMAIN. Check the exact hostname you audited, bare or www.

What to do.

• Add or fix the A, AAAA or CNAME record for the hostname you audit, or audit the hostname that does resolve: Run New Audit lets you change the address for one run, and adding it as a website makes it stick.
• Check that the domain has not expired and that its nameservers are the ones your DNS host expects.
• After a nameserver or host move, wait for the change to propagate, then run the audit again.

TLS handshake failed

What the crawler observed. The TCP connection opened, but the TLS handshake did not complete, or the certificate the server presented failed verification: it has expired, it does not cover this hostname, it is self-signed, or its chain is incomplete. Certificate verification is on for every cloud request, and there is no setting to turn it off. A TLS failure is retried before it is recorded.

How it is classified. tls, from the certificate or handshake error, such as CERT_HAS_EXPIRED, ERR_TLS_CERT_ALTNAME_INVALID, DEPTH_ZERO_SELF_SIGNED_CERT, UNABLE_TO_VERIFY_LEAF_SIGNATURE or EPROTO.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null https://example.com/ && echo ok
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null | grep 'Verify return code'

A valid certificate prints ok and Verify return code: 0 (ok). A failure prints a curl error with exit code 60 (for example SSL certificate problem: certificate has expired) or 35 (the handshake failed), and a non-zero verify code such as 10 (certificate has expired) or 21 (unable to verify the first certificate).

What to do. Renew or reissue the certificate so it covers the exact hostname, bare and www if you use both, and serve the full chain including intermediates. Then run the audit again.

• certbot (Let’s Encrypt). Point the server at fullchain.pem, not cert.pem, so the intermediate is served.
• Cloudflare (proxied). The visitor sees Cloudflare’s edge certificate. Universal SSL covers the apex and first-level subdomains only, so a host such as dev.www.example.com needs an advanced or custom certificate.
• Vercel, Netlify and similar hosts. Certificates are issued automatically once the domain’s DNS points at the host. A pending or failed issuance shows in the domain settings.

Connection refused or reset

What the crawler observed. No HTTP response: the connection was refused (ECONNREFUSED), reset (ECONNRESET), closed before a response arrived, or the network had no route to the host (EHOSTUNREACH, ENETUNREACH). With Rendering Off the request is retried, up to 3 attempts. A server that is down does this, and so does a firewall that drops requests from networks it does not recognise.

How it is classified. connection. When the hostname resolves to an address outside the public internet, the cause is private address instead. When the follow-up DNS lookup finds the name does not exist, it is DNS.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -w '%{http_code}\n' https://example.com/

A reachable server prints a status such as 200. A refused connection fails with curl exit code 7 (could not connect), a reset with 56 (receive failure), and a connection closed without an answer with 52 (empty reply). Run it from a network you do not normally use, such as a phone on mobile data: a firewall that trusts your office network answers you and refuses the cloud.

What to do.

• Bring the server back up, and check that it listens on port 443 (or 80 for http://) on its public address.
• Look in your firewall, host or CDN logs for refused connections at the time of the audit. Cloud audits come from Cloudflare’s network, not from a fixed squirrelscan address (see Firewall and WAF exceptions).
• Let squirrelscan through the firewall with a secret request header rather than an IP rule.
• A site behind a CDN rarely fails this way: the CDN answers, and an unreachable origin shows up as a 5xx from the CDN instead, such as Cloudflare’s 521, 522 or 523 (see server error).

No response in time

What the crawler observed. Requests to the site did not complete before their budget ran out (see the requests, in order): 30 s per page attempt, then the entry page’s 60 s plain retry. A request can time out before any response, or after the headers while the body is still arriving. With Rendering All, the browser render has the first 18 s of a page’s 30 s to itself, and a plain request races it for the rest. The run page lists each attempt that timed out and its budget. The record cannot tell a server that accepted the connection and then stalled from one that never answered, so neither is claimed.

How it is classified. timeout, from the runtime’s timeout error or an abort at the deadline. When the hostname resolves to an address outside the public internet, the cause is private address instead.

Attribution. The site, or Both when our rendering service was failing during the run. When the browser render had not answered by the time the plain request took over, the run page says a part for us cannot be fully ruled out.

How to check.

curl -sS -o /dev/null -m 30 -A "$UA" \
  -w 'connect %{time_connect}s, first byte %{time_starttransfer}s, total %{time_total}s, status %{http_code}\n' \
  https://example.com/

A healthy page shows a first byte well under 30 seconds. curl exit code 28 (operation timed out) is what the cloud saw. A site can be fast from your desk and slow from far away, so a fast result here does not rule out a slow path to squirrelscan’s network.

What to do.

• If the run page also says our rendering service failed, run the audit again first: a second run shows whether the site is really slow. See both sides failed.
• Fix whatever makes the first response slow: server load, slow database queries, cold starts on a serverless host.
• Some firewalls hold unknown clients open without answering instead of refusing them. Let the crawler past that rule with a secret request header.
• If the site is fast for you, audit it from your own machine with the CLI, which uses your network: squirrel audit https://example.com --http.
• If the site is fast for you and the audit still times out, contact support with the reference from the run page.

Bot protection or WAF challenge

What the crawler observed. The site answered with a challenge instead of the page, such as a Cloudflare “Just a moment” interstitial. A challenge to a plain request is a refusal and is not retried. With rendering on, a challenged render is replaced by a plain request, which carries your custom request headers.

How it is classified. Only when the failed response itself proved it: a cf-mitigated: challenge header, or a body that is a recognised challenge interstitial, on a response with a status of 400 or above. A Server: cloudflare header or a detected CDN is not proof, and a plain 403 without either signal is classified as access denied. With Rendering Off, only a challenge served as a 503 is recognised, so a 403 challenge is reported as access denied. The run page names the protection when the response did.

A body is recognised as a challenge by signatures of challenge and block pages in its first 10 KB: Cloudflare’s challenge script (/cdn-cgi/challenge-platform/h/…/orchestrate/), _cf_chl_opt or a __cf_chl_ token, cf-browser-verification, Imperva’s Incapsula incident ID line with its id, Akamai’s entity-encoded reference line or its reference id after an Access Denied heading, or PerimeterX’s captcha. Script and markup signatures count only inside a tag or an inline script, and wording only outside <code>, so a page that quotes one, such as a tutorial printing a provider’s script tag or a support page showing an incident line in a code block, is neither a challenge nor attributed to the provider. On a 200 these also need the page to be attributed to a provider. Challenge wording such as Just a moment or Checking your browser counts only on a 401, 403, 429 or 503. The JavaScript Detections script that Cloudflare adds to ordinary pages also loads from /cdn-cgi/challenge-platform/, and it never makes a page a challenge. The same test decides which crawled pages are set aside from page-level scoring as challenge pages, so an ordinary 200 page on a protected site is scored.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -D - -A "$UA" https://example.com/ | grep -i -E '^(HTTP/|cf-mitigated|cf-ray|server)'

An unchallenged request shows HTTP/2 200. A Cloudflare challenge shows a 403 or 503 with cf-mitigated: challenge. Your own IP address may not be challenged the way squirrelscan’s network is, so also open your WAF or bot protection event log for the time of the audit. On Cloudflare that is Security → Events, where you can search for the cf-ray id from the run page’s technical details. Look for the rule or product that issued the challenge: Bot Fight Mode, Super Bot Fight Mode, a managed challenge rule, or a custom rule.

What to do. Keep the protection on and add a narrow exception for squirrelscan:

1. In the dashboard, open the website’s Settings → Crawl and add a custom request header (every plan has them) with a long random value only you know, for example X-Squirrelscan-Auth.
2. Add a WAF rule that skips the challenge only for requests carrying that header with that exact value. Firewall and WAF exceptions has the rule for Cloudflare, nginx and other WAFs. On Cloudflare, Bot Fight Mode cannot be skipped by any rule; the firewall page says what to do instead.
3. Run the audit again.

Avoid switching bot protection off for the whole site, and avoid allowlisting IP ranges: cloud audits leave from Cloudflare’s shared addresses, so an IP rule opens your site to far more than squirrelscan. If your platform verifies crawlers with signed requests, Web Bot Auth is another way in.

Access denied (403)

What the crawler observed. The site answered 403 Forbidden, without a challenge header or a challenge page. The response did not say what refused the request: a WAF rule, a geographic or IP block, hotlink or referrer protection, or a page that really is private. A 403 is not retried.

How it is classified. http_4xx with status 403, shown as access denied. Unlike a challenge, a plain 403 does not prove bot protection, so none is named. The audit is blocked.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -D - -A "$UA" https://example.com/ | grep -i -E '^(HTTP/|server|cf-ray)'

HTTP/2 403 here too points at the server’s own rules. A 200 here only shows that this request got through: the rule may key on the network, the request’s headers or its timing, so look in your WAF, CDN and server logs for the 403 at the time of the audit and the rule that produced it, and check for country or IP blocks. Cloud audits come from Cloudflare’s network.

What to do. Find the rule that returns the 403 and add an exception for squirrelscan keyed on a secret request header, the same way as for a challenge. If the page is meant to be private, audit a public page instead.

Login required (401)

What the crawler observed. The site answered 401 Unauthorized, or 407 Proxy Authentication Required from a proxy: it wants credentials before it serves the page. A 401 whose WWW-Authenticate header starts with Basic is HTTP basic auth, which staging sites often use. The page is stored as an error page, and nothing is auditable.

How it is classified. http_4xx with status 401 or 407, shown as login required. A 401 makes the audit blocked. A sign-in that works by redirecting to another host, such as Cloudflare Access or a single sign-on provider, is not a 401: it is classified as a redirect to another site. See sign-in page or access gateway.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -D - https://staging.example.com/ | grep -i -E '^(HTTP/|www-authenticate)'
curl -sS -o /dev/null -w '%{http_code}\n' -u 'username:password' https://staging.example.com/

The first shows 401 and, for basic auth, www-authenticate: Basic realm="...". The second, with the right username and password, prints 200.

What to do. A cloud audit cannot fill in a login form, but it can send credentials as a header:

• HTTP basic auth. Add an Authorization header with the value Basic followed by the Base64 encoding of username:password under Settings → Crawl → Custom request headers. HTTP basic auth has the steps.
• An access gateway or single sign-on. See sign-in page or access gateway for Cloudflare Access, Vercel deployment protection and the gateways that cannot be passed with a header.
• Neither. Audit the public address of the site instead, or run the CLI from somewhere that can reach the site: squirrel audit https://staging.example.com.

Rate limited (429)

What the crawler observed. The site answered 429 Too Many Requests, the 430 some platforms use, or a 503 carrying Retry-After: a rate limit decided too many requests were arriving, often from a rule that counts requests per network rather than per visitor. The crawl backs off for the whole host, honours Retry-After within its own backoff limits, and retries while its attempt, backoff and time limits allow. See rate-limited hosts for the pacing and backoff.

How it is classified. http_4xx, reported with status 429 whichever of the three the site sent (the status it actually sent is in the message), shown as rate limited. When no page could be fetched, the audit is blocked.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

for i in $(seq 1 20); do curl -sS -o /dev/null -w '%{http_code}\n' -A "$UA" https://example.com/; done | sort | uniq -c

Twenty requests in a row that all print 200 mean your own rate is under the limit. A count of 429 or 430 lines shows where it starts. Look for the rate limit rule in your CDN or WAF, and for 429s in its log at the time of the audit.

What to do.

• If you control the rate limit rule, exempt squirrelscan from it with a secret request header. That also makes the crawl authorised, and an authorised crawl still slows down when your server asks it to.
• On a platform whose limits you cannot change, such as a Shopify storefront, audit fewer pages instead: rate-limited hosts covers what works there.
• Then run the audit again, or wait for the limit to reset first.

Page not found (404 or 410)

What the crawler observed. The audited address, after any redirects on the same site, answered 404 Not Found or 410 Gone. The server is up, but there is no page at that address.

How it is classified. http_4xx with status 404 or 410, shown as page not found.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -D - -L -A "$UA" https://example.com/the/audited/path | grep -i -E '^(HTTP/|location)'

-L follows the redirects, so the last HTTP/ line is the final status. It should be 200. Check the hostname (bare or www), the path and any trailing slash against the exact address on the run page.

What to do. Publish the page, or audit the right address: Run New Audit lets you change it for one run, and adding the site’s real home page as a website makes it stick. A site that moved to a new domain usually shows up as a redirect to another site instead.

Other 4xx responses

What the crawler observed. The site answered with a 4xx status not covered above, such as 400, 405, 406, 421 or 451. The page is stored as an error page. The status is on the run page.

How it is classified. http_4xx with that status, shown as a client error.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -w '%{http_code}\n' -A "$UA" https://example.com/

Compare the status with the run page’s, and look in your server logs for the request at the time of the audit. A status that only squirrelscan gets points at a rule that matches its requests.

What to do. Fix the rule or route that returns that status for the address, then run the audit again.

Server error (5xx)

What the crawler observed. The site answered with a server error of its own, such as 500, 502 or a 503 without Retry-After, for the pages we asked for. With Rendering Off a 5xx is retried, up to 3 attempts. A 503 carrying Retry-After is treated as a rate limit, and a 503 that is a challenge as a challenge.

How it is classified. http_5xx, unless the response proved a challenge, in which case it is a challenge.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -D - -A "$UA" https://example.com/ | grep -i -E '^(HTTP/|server|retry-after|cf-ray)'

A 5xx here too is the server’s own error: look in your error logs and your hosting status page for the time of the audit. On Cloudflare, 521 (web server is down), 522 (connection timed out) and 523 (origin is unreachable) mean Cloudflare could not reach your origin, and 525 that the TLS handshake between Cloudflare and your origin failed.

What to do. Fix the error, then run the audit again once the site serves normally. A 503 from a CDN while the origin is healthy can also be a protection rule: check the CDN’s security log for it.

Redirected to another site

What the crawler observed. Following the audited address’s redirects ended on a different site (see same site). The redirect is an HTTP 3xx with a Location header or, with rendering on, a redirect the page makes in the browser. The request follows it, but the crawl neither stores nor audits the page it lands on, or any other page of that site. The run page and the email name the destination’s origin, without its path or query.

How it is classified. redirect, shown as redirected off-site. The destination decides the wording of the next step:

A destination counts as a sign-in host when it is a subdomain of cloudflareaccess.com, okta.com, oktapreview.com, auth0.com, onelogin.com, amazoncognito.com, b2clogin.com or clerk.accounts.dev; when it is accounts.google.com, login.microsoftonline.com, login.live.com, vercel.com, accounts.shopify.com, id.atlassian.com or appleid.apple.com; or when it has at least three labels and its first label is login, signin, sign-in, sso, auth, accounts or account.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -o /dev/null -D - -L -A "$UA" https://example.com/ | grep -i -E '^(HTTP/|location)'

Each location line is one hop. The last one shows where the audited address ends up. If it is on another site, the audit fails this way. Add your custom request headers with -H to see whether they change where it ends up.

What to do.

• The site moved. Audit the canonical address, the one the site actually serves: add the destination as a website in the dashboard and audit it there. Audits of the old address keep failing while the redirect is in place.
• A sign-in page. See sign-in page or access gateway.
• Anything else. Fix the redirect so it stays on the same site, or audit the address the site actually serves.

Sign-in page or access gateway

What it means. An access gateway is a service in front of the site that sends every request without valid credentials to a sign-in page: a zero-trust proxy such as Cloudflare Access, a hosting platform’s deployment protection, single sign-on in the application, or HTTP basic auth. A cloud audit cannot sign in. The gateway shows up as one of two causes:

• a redirect to the gateway’s or the identity provider’s host, reported as a redirect to another site with sign-in wording, or
• a 401 or 407 from the site itself, reported as login required.

What the crawler observed. The seed redirect probe never sends your custom request headers, so on a gated site it always lands on the sign-in page or gets the 401. That alone does not fail the audit: when the sign-in page is on another site, the crawl keeps the audited address and fetches it again with the headers. The audit fails as a redirect when that entry page request also lands on the sign-in page, which happens when:

• no header the gateway accepts is set, or the token or secret in it has expired,
• the gateway’s policy does not cover this hostname (see the verification checklist), or
• the audited address redirected to another origin before reaching the gateway, which removes the headers (for example http:// to https://, or example.com to www.example.com).

Gateways and the way through.

A sign-in page on the same site, such as login.example.com in front of staging.example.com, is not a redirect to another site. The seed redirect probe lands there, the crawl starts from the sign-in page, and the audit can complete with that page as its content. A Shopify store that serves its password page from its own domain behaves the same way. Check the audited pages, not only the status.

How to check.

curl -sS -o /dev/null -D - https://staging.example.com/ | grep -i -E '^(HTTP/|location|www-authenticate)'
curl -sS -o /dev/null -D - -H 'Header-Name: <value>' https://staging.example.com/ | grep -i -E '^(HTTP/|location|www-authenticate)'

The first request, without credentials, shows what the gateway does: a 302 with a location on the sign-in host, or a 401. The second, with the same headers as the website’s custom request headers, should show a 200 and no location. If the second still redirects, the gateway does not accept the headers, and no audit will get through until it does.

What to do.

1. Set the headers the gateway accepts under Settings → Crawl → Custom request headers, using the gateway’s section in Firewall and WAF exceptions.
2. Audit the exact origin the second curl used, with the same scheme and host. Rendering can stay on: the cloud browser sends the headers to that origin too.
3. Run the audit and work through the verification checklist.

For a gateway with no header route, audit from somewhere the gateway already lets through: the CLI on a network the gateway admits, such as an office network or VPN, or the CLI against a local build of the same site (squirrel audit http://localhost:3000). Signed in, the CLI renders pages in the cloud browser by default; add --http when the audit depends on headers passed with -H (see rendering and custom headers).

Localhost or private address

What the crawler observed. The hostname resolves to an address that only exists on your machine or your private network: a loopback address such as 127.0.0.1 or ::1, a private network address such as 10.x.x.x or 192.168.x.x, a link-local address, or a reserved one. Local development hostnames often do this. A cloud audit runs on the internet and does not connect to addresses outside the public internet. The follow-up DNS lookup records the class of address and, because it is not public, the address itself, which the run page shows.

How it is classified. Private address, when the failure would otherwise be a connection failure, a timeout or unknown, and at least one address the hostname resolves to is not public.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

dig +short your-site.test

A cloud-reachable site prints only public addresses. 127.0.0.1, ::1, 10., 172.16. to 172.31., 192.168. or 169.254. addresses are the cause.

What to do.

• Audit it locally with the CLI, which runs on your machine and can reach it: squirrel audit http://localhost:3000. See local audits.
• Or deploy to a publicly reachable staging address and audit that. If the staging site should stay private, put it behind a gateway a header can pass (see access gateways) and add the header under Settings → Crawl.

Disallowed by robots.txt

What the crawler observed. The site’s robots.txt, fetched at the start of every audit, disallows the audited address for the user agent the crawl sends, and the audit respects it. This only happens when Respect robots.txt is on under Settings → Crawl, which it is not by default, and only on a crawl that is not authorised: a website with a custom request header never applies robots.txt.

How it is classified. robots.

Attribution. The site, unless the site-side rules make it Both or Unknown.

How to check.

curl -sS -A "$UA" https://example.com/robots.txt

Look for a Disallow rule covering the audited path, under User-agent: * or a group that matches a browser user agent.

What to do. Allow the path in robots.txt, or, for a site you own, turn off Respect robots.txt under Settings → Crawl (see crawl settings). Or point the audit at a path robots.txt permits.

Our rendering service failed

What the crawler observed. Our cloud browser service, which renders pages when rendering is on, refused render requests, failed render jobs, or was switched off for the rest of the run after failing. That one is ours. The crawl falls back to fetching pages with a plain request when the browser fails, so this alone rarely fails an audit.

How it is classified. Rendering service failure, when our renderer failed and nothing at the site was recorded failing. When the site also failed, the cause is the site’s and the attribution is Both.

Attribution. squirrelscan, or Both when the site also failed during the same run, for example by not answering in time. A busy or failing renderer and a slow site can each make the other look worse, so we do not pick one.

What to do. Run the audit again. If it fails the same way while the run page shows our side working, the cause is the site’s, and its own section above applies. If our side keeps failing, contact support with the reference from the run page. Setting Rendering to Off under Settings → Crawl takes the renderer out of the audit while you wait.

Something failed on our side

What the crawler observed. A step of ours failed after the site was reached, such as a call to our own API or publishing the report, or the audit’s container refused to start because one of its own safety checks failed.

How it is classified. Failed on our side.

Attribution. squirrelscan. Your site was not the cause.

What to do. Run the audit again. If it fails again, contact support with the reference from the run page.

The audit ran out of time

What the crawler observed. The audit did not finish within its overall time limit. That usually comes from the size or the pace of the crawl, not from a site that is down: large crawls, slow pages and a Crawl-delay in robots.txt all make an audit take longer. A crawl that runs out of its own crawl-phase time is a different case: it stops early and publishes a partial report instead of failing.

How it is classified. audit_timeout, shown as ran out of time.

Attribution. squirrelscan: the limit is ours, not a failure seen at your site.

How to check. curl -sS https://example.com/robots.txt | grep -i crawl-delay shows whether a Crawl-delay slows the crawl. The run page’s page count shows how far it got.

What to do. Audit fewer pages, or narrow the crawl to the sections you care about with include patterns under Settings → Crawl (see crawl settings). Then run it again.

The audit stalled

What the crawler observed. The audit stopped making progress part way through: no page fetched, no batch of rules finished and no stage completed, for the whole of the stall window of the phase it was in. We ended it. The run page names the phase.

How it is classified. stalled, shown as audit stalled.

Attribution. squirrelscan. Your site is not the problem, and there is nothing for you to change.

What to do. Run the audit again. If it stops in the same place twice, contact support with the reference from the run page, because that one is ours to fix.

Unknown cause

What the crawler observed. No cause could be identified, or the run kept only a failure sentence that names none. The reason still carries whatever the fetch reported.

How it is classified. unknown.

Attribution. Unknown.

How to check. Load the site from outside your network, then run the same audit with the CLI from your machine: squirrel audit https://example.com --max-pages 1. A local run that works while the cloud run fails points at how your site treats requests from our network. The curl checks in checking it yourself narrow it further.

What to do. Run the audit again. If it keeps failing, contact support with the reference from the run page: the run id and the audit id.

When the CLI cannot write its local store

A CLI audit keeps what it crawls in SQLite files under ~/.squirrel: the project database, the content store and the link cache. Before it registers the run or sends a request to the site, squirrel audit checks that it can write to the stores the audit needs. When it cannot, it stops with the file, the cause and the fix:

✗ Cannot write to ~/.squirrel/content-store.db: it is owned by another user (uid 0), usually because squirrel once ran with sudo. Fix: sudo chown -R "$(id -un)" ~/.squirrel

A run that stops here is never registered, so it costs nothing. The causes it names:

If a store breaks partway through the crawl, the audit fails with its error instead of finishing with fewer pages. A store that another squirrel process has locked is not an error: the audit waits for it. If no page could be saved even so, the failure reason says that, under the unknown code, rather than only “No pages were crawled”.