Page Classifier Configuration

The [classifier] section controls the page classifier, a small model that is bundled with squirrel. Its weights are treetop-pagetype-v1 (the file classifier-v2.json), MIT licensed like the rest of the repository, and ship inside the executable.

What it does

When a finding names a link, squirrel can say where on the page that link sits: the footer, the navigation, the pagination, a related-content block. Most of that comes from the page’s own markup (landmark tags, ARIA roles, comment threads). Plenty of pages have none of it, so the classifier looks at the page structure and fills the gap.

• It runs on your machine. There is no network call and no credit cost.
• The markup wins for the header, the footer and comment threads, which are never overridden, and comment threads are only ever read from markup. A section the classifier names (the pagination, the breadcrumbs) refines a navigation or sidebar landmark.
• It answers only when it is confident, using a per-class bar chosen for precision. An unsure link keeps no section.
• It also names a page type for each page (article, product page, docs page, legal page and so on).
• The model is about 9 MB, loaded the first time a page is classified. Commands that never audit never read it.

With the classifier off, links keep only the sections the page markup names, and no page type is stored.

Configuration

[classifier]
enabled = true

Options

enabled

Type: boolean Default: true

false switches the classifier off. Link sections then come from page markup alone, and no classification is stored.

For one run, --no-classifier does the same:

squirrel audit https://example.com --no-classifier

Known limitations

The classifier is a model, and models guess. It answers only when it is confident, but these mistakes happen:

• On a single article, the previous and next post links can read as pagination.
• A “follow us” row of profile icons can read as social sharing links.
• The comments label comes from page markup only. A comment thread built with unusual class names or ids is not recognised, and the model never names one.
• On pages with good markup it rarely speaks, because the markup has already answered.
• Pages served from the crawl cache keep the sections stored when they were last fetched, until they are fetched again. An audit that reuses unchanged pages therefore shows the old sections for them.

A wrong phrase never hides a broken link. It only changes the words that say where the link sits.

Check that it loads

squirrel self doctor reports the bundled artifact, the featurizer version it was built for and how long it took to load:

[OK] Page classifier: artifact v1 (sha256 f0de5687de8b, 8.4 MB), featurizer v2.3.0, region+pageType+section heads, loaded in 21 ms

A warning means the model could not be used. Audits still run and link sections fall back to page markup.