squirrelscan
GitHub

Website ownership confirmation

What confirming website ownership means, how to confirm it, and how hosted MCP and API audits treat a website you have not confirmed

Confirming ownership tells squirrelscan that a website is yours, or that you are authorized to audit it. It is recorded per website, by you, and it decides how a cloud audit started over the hosted MCP server or the API crawls that site.

What confirming means

A confirmed website is crawled as its owner’s: the website’s own robots.txt setting, its crawl profile and its authorized-crawl headers apply. A website you have not confirmed is crawled with the manners of a visitor, because an MCP or API audit can name any public host.

Some rules to know:

  • Confirmation applies to one website. Confirming example.com says nothing about any other site.
  • A website classified as a prospect cannot be confirmed. If it is your own site, reclassify it as owned first (add_website with kind: "owned" over MCP).
  • You can withdraw a confirmation at any time.
  • Audits started from the dashboard, the CLI and the scheduler are not covered by the visitor policy today.

How to confirm

In the dashboard

The dashboard toggle has not shipped yet: confirm over MCP for now.

Over MCP

Confirmation over MCP takes two calls to update_website_settings, so the instruction to ask you is in front of the agent before anything is applied:

  1. The agent calls update_website_settings with confirm_ownership: true. The response has status: "confirmation_required" and a confirmation_token. Nothing is written.
  2. The agent asks you whether the site is yours. Only if you say so, it calls again with the same arguments plus confirm: true and the token.

The token is bound to the website and to you, and is good for 15 to 30 minutes. Withdrawing (confirm_ownership: false) and confirming a site that is already confirmed need no token.

get_website_settings reports the result as ownership_confirmed, with ownership_confirmed_at when it is true.

If the server cannot mint a token, the call fails with OWNERSHIP_CONFIRMATION_UNAVAILABLE: retry later.

How unconfirmed websites are audited

This section describes the policy as it runs today, and the stage it is rolled out in. The visitor crawl policy is switched on in stages: off, observe and enforce. Hosted squirrelscan is at observe.

The policy covers cloud audits started over MCP (run_audit) or the API (POST /v1/agent-runs) for a website that is not confirmed, which squirrelscan calls a visitor run.

observe (current)enforce
robots.txtYour website’s own setting appliesHonoured, whatever the website stores
Authorized-crawl headersApplied if the website has themNot sent on a visitor run
Per-host limitVisitor starts are counted and logged, none is refusedStarts over the limit are refused with HOST_RATE_LIMITED
Identifying user agentNot yet appliedNot yet applied

In plain terms: at observe squirrelscan records how often the policy would have changed an audit and refuses nothing, so an unconfirmed website audits as it did before. The enforce column is what is built and will apply when the stage is switched. The identifying user agent for visitor runs is planned and not part of either stage yet.

Per-host limits

When the limit is enforced, a visitor run takes a slot for its target host before anything is written, so a refused start creates no run and is not charged. The limits are counted across every organization, per host and per registrable domain:

LimitDefault
Visitor audits of one host running at once1
Visitor audits started per host per 24 hours10
Of those, by one organization3
Visitor audits started per registrable domain per 24 hours30
Of those, by one organization10

Owner runs are never limited by host. Confirming ownership removes the limit for that website.

The HOST_RATE_LIMITED error

A start refused by a per-host limit returns HOST_RATE_LIMITED.

  • Over MCP it is a tool error with limit, retry_after_seconds and a hint.
  • Over the API it is 429 with retryAfterSeconds in the body and a Retry-After header.

retry_after_seconds is the time until a slot frees by the clock. It is null when the start waits on another audit of the host that is still running: try again when that audit finishes. limit names which limit fired: host_concurrent, host_window, org_window, domain_window, org_domain_window, or invalid_host when the URL names no host to count the run against.

If the site is yours, register it, confirm ownership as above, and the limit no longer applies.

Type to search…

↑↓ navigate↵ openesc close