squirrelscan
GitHub

pages

List the pages of a stored audit with their titles, descriptions, links and flags

An issue list only shows the pages that break a rule. The pages command lists every page the audit crawled, one row each, so you can read a site’s titles and descriptions side by side, spot orphans, or see that every post title ends in the same suffix.

Every column is data the audit already stored: the pages, their extracted features, their links and the rule results. The command fetches nothing, so it is instant and costs no credits.

Usage

squirrel pages [options]

Options

OptionAliasDescriptionDefault
--crawlCrawl ID (or 8-character prefix) to read; repeat twice with --difflatest audit
--diffCompare two audits of the site: pages added, removed and changedfalse
--format-fOutput format: json, csv, markdown, html (markdown, json or csv with --diff)console table, or markdown when piped
--output-oWrite to a file instead of stdout-
--missingOnly pages missing title, description, h1 or schema (repeatable or comma-separated)all
--duplicateOnly pages the audit flags as a duplicate title or description (repeatable or comma-separated)all
--too-longOnly pages with a title over 75 or a description over 160 charactersfalse
--orphanOnly pages the audit flags as orphans: fewer internal links to them than the orphan rule’s minimum (2 by default). A crawl the rule skipped (a capped crawl) flags nothingfalse
--sectionOnly pages under this URL path, for example /blog; case-insensitive, and /café matches /caf%C3%A9all
--pageOnly pages whose URL contains this text, case-insensitive (repeatable; a comma splits the value)all

Filters combine: a page has to match every filter you pass. Within --missing, --duplicate and --page, matching any one value is enough.

# Blog posts with no description
squirrel pages --section /blog --missing description

# Every page as a spreadsheet
squirrel pages -f csv -o pages.csv

Columns

ColumnMeaning
urlThe URL the crawl requested
finalUrlWhere it was served from after redirects (the same as url when it did not redirect)
statusHTTP status
depthClicks from the start URL
canonicalThe page’s canonical URL, if it declares one
noindexThe page asks not to be indexed (meta robots or X-Robots-Tag)
title, titleLengthThe <title> and its length in characters
description, descriptionLengthThe meta description and its length
h1CountNumber of <h1> elements
h1Text of the first <h1>
headingCountNumber of headings, h1 to h6
wordCountWords of visible content
schemaTypesschema.org @type values declared on the page
schemaIdsJSON-LD @id values declared on the page (the first 20)
inlinksCrawled pages that link here, each counted once
internalOutlinksLinks from this page to other pages of the same site
externalOutlinks, externalDomainsLinks to other sites, and the hosts they point at
loadTimeMsResponse time of the fetch
lastModifiedThe Last-Modified header, if the server sent one
flagsWhat the audit found on the page (below)

A link to the page itself is neither an in-link nor an out-link. Pages that differ only by their query string (/items?id=1 and /items?id=2) keep their own out-links, but share one inlinks count, because the orphan and internal-link rules count links to a page without its query string. Links come from the pages’ stored parse, the same source the orphan and internal-link rules read, so a nofollow link counts here even though the orphan rule does not count it.

Flags

The flags column comes from the audit’s own rule results, so the table and the issue list agree.

FlagSet when
titlecore/meta-title warned or failed on the page
descriptioncore/meta-description warned or failed
h1core/h1 warned or failed
title-not-uniquecore/title-unique names the page
duplicate-titlecontent/duplicate-title names the page
duplicate-descriptioncontent/duplicate-description names the page
orphanlinks/orphan-pages names the page
canonicalcore/canonical warned or failed
robotscore/robots-meta warned or failed
thin-contentcontent/word-count warned or failed
noindexThe page asks not to be indexed (a fact from the crawl, not a rule)
no-schemaThe page answered with a success status and declares no schema.org type (also a fact)

--duplicate and --orphan select on these flags, so they list exactly the pages the issue list names. A rule that was disabled, or skipped for the audit, sets no flag. --missing and --too-long read the columns themselves, and --missing only judges pages that answered below 400 and were parsed (a PDF, an image or a 404 has no title to be missing): the title and description limits are the rules’ defaults (75 and 160 characters), so a site that changed max_length in squirrel.toml sees its own limit in the title and description flags instead.

h1, headingCount and schemaIds are read from the page’s stored parse, the same parse the links come from; a page that was not parsed (a PDF, an image, an error page) has none of them. Anchor text is not a column: use squirrel entities for the declared schema graph.

Output formats

With no --format you get a console table of the first 50 pages on a terminal, and markdown when the output is piped or goes to a file, so squirrel pages | ... is parseable without a flag.

FormatWhat you get
json{ site, crawlId, startedAt, total, filtered, pages: [...] }: total is every page in the audit, filtered is how many matched, pages holds the rows with every column
csvOne header row and a row per page. List columns (schemaTypes, externalDomains, flags) join with a space. A cell that starts with =, +, - or @ is prefixed with an apostrophe so a spreadsheet does not run it as a formula
htmlOne standalone page, no network requests, with the table sortable by any column and filterable by text and by flag. At most 5,000 rows are inlined (the page says when more matched; use csv or json for all of them)
markdownA table of the columns a person scans: URL, status, title and length, description and length, H1 count, words, schema, in and out links, flags

Titles, descriptions and URLs come from the audited site, so each format treats them as hostile. The console, csv, markdown and html outputs remove control characters, escape sequences, invisible characters and bidirectional overrides, a pipe in a URL is escaped so it cannot end a markdown table cell, and csv prefixes a formula start with an apostrophe. json keeps the text and writes DEL, the C1 controls, bidirectional overrides, invisible characters and line separators as \uXXXX, so the file is safe to print and the value is unchanged.

Comparing two audits

--diff shows what changed between two audits of the same site, which is how you check a fix: did the title rewrite land, did the 404 go away, did a page lose its schema.

# The latest audit against the one before it
squirrel pages --diff

# A named audit against the one before it, or two named audits
squirrel pages --diff --crawl 4f2ab91c
squirrel pages --diff --crawl 4f2ab91c --crawl 9a01c3de

Pages are matched by URL. The report lists the pages added (only on the newer audit), removed (only on the older one) and changed, with each changed field’s before and after: status, finalUrl, canonical, noindex, title, description, h1Count, h1, schemaTypes, schemaIds and flags. Counts that move with every crawl (links, word count, load time) are not compared. A list compares as a set, so a reorder is not a change.

With one audit named, or none, the older side is the audit of the same site that ran just before it. The store holds every project, so without that rule “the two newest audits” would often be two different sites. Naming two audits compares exactly those, in whichever order you give them, even across sites. With only one audit of the site, the command says so and exits 0; if the earlier audit was reclaimed (squirrel self disk --prune) it says that instead. When the two audits crawled clearly different numbers of pages (a quick audit against a full one, a difference of more than 20%), the markdown adds a note and the JSON a scopeNote, because pages then come and go with the crawl’s scope and not with the site. More than two --crawl values is a usage error.

--diff takes markdown (default), json or csv. --section and --page narrow both sides. --missing, --duplicate, --too-long and --orphan are rejected with --diff: they select on the state a diff compares, so a page whose title was fixed would stop matching --missing title on the newer side and be reported as removed.

Exit codes

The command exits 0 whenever it has an answer: an audit with no matching pages prints an empty table, and a store with no audit prints how to create one. It exits 1 for a usage mistake (an unknown --missing, --duplicate or --format value) or a project store it cannot read. --diff with a format it does not support, or with a state filter, is also a usage error.

Type to search…

↑↓ navigate↵ openesc close