Project Settings

The [project] section configures project-level metadata and multi-domain crawling.

Configuration

[project]
name = "mysite"
domains = ["example.com"]

Options

name

Type: string Default: Current directory name Required: No

Project name used for identifying crawls of local URLs (localhost, 127.0.0.1, local IPs).

When crawling local addresses, squirrelscan needs a project name to organize stored data. If not specified, it uses the current working directory name.

Examples:

Explicit project name:

[project]
name = "my-nextjs-app"

Using default (directory name):

[project]
# name omitted - uses directory name

When crawling localhost:

cd /Users/you/projects/my-app
squirrel audit http://localhost:3000
# Uses "my-app" as project name (from directory)

With explicit name:

# In squirrel.toml:
[project]
name = "staging-api"

# Then run:
squirrel audit http://localhost:8080
# Uses "staging-api" as project name

Interactive prompt:

If crawling localhost without a configured name and in a TTY, squirrelscan prompts for a name:

Crawling local address: http://localhost:3000
Project name [mysite]: staging-frontend

domains

Type: string[] Default: [] (empty array = seed domain only) Required: No

Allowed domains for crawling. When empty, only the seed URL’s domain is crawled.

Subdomain Wildcards:

Setting domains = ["example.com"] automatically allows all subdomains:

• example.com ✓
• www.example.com ✓
• blog.example.com ✓
• api.example.com ✓
• docs.example.com ✓

Does NOT allow:

• example.org ✗ (different TLD)
• test-example.com ✗ (not a subdomain)

Examples

Single domain (default)

Crawl only the seed domain and its subdomains:

[project]
domains = []

Running squirrel audit https://example.com crawls:

• https://example.com ✓
• https://www.example.com ✓ (subdomain)
• Internal links to https://blog.example.com ✗ (not followed without config)

Multi-domain site

Allow main site plus specific subdomains:

[project]
domains = ["example.com"]

Running squirrel audit https://example.com crawls:

• https://example.com ✓
• https://www.example.com ✓
• https://blog.example.com ✓
• https://docs.example.com ✓
• https://api.example.com ✓

All subdomains are automatically allowed.

Multiple root domains

Crawl multiple distinct domains (rare):

[project]
domains = ["example.com", "example.org"]

Crawls both domains and all their subdomains.

Note: This is uncommon. Most sites should use include patterns in the crawler section instead.

Local development

For local projects:

[project]
name = "myapp-local"

Then run:

squirrel audit http://localhost:3000

Domain matching rules

Subdomain matching

Domain "example.com" matches:

Port handling

Ports are ignored for matching:

[project]
domains = ["localhost"]

Matches:

• http://localhost:3000 ✓
• http://localhost:8080 ✓
• http://localhost ✓

Protocol handling

Protocols (http/https) are ignored for matching:

[project]
domains = ["example.com"]

Matches:

• http://example.com ✓
• https://example.com ✓

When to use domains vs include

Use domains when:

You want to crawl an entire domain and all its subdomains:

[project]
domains = ["example.com"]

Crawls everything under *.example.com.

Use include when:

You want fine-grained URL pattern control:

[crawler]
include = [
  "/blog/**",
  "/docs/**"
]

See Crawler Settings for URL patterns.

Interaction with crawler settings

Priority

include narrows within the hosts domains allows. It does not replace them:

[project]
domains = ["example.com"]

[crawler]
include = ["/blog/**"]

This crawls only /blog/** URLs on example.com and its subdomains. A path-only pattern never admits another host. To reach one deliberately, use an absolute pattern such as https://docs.example.org/*; see include.

domains is a hard allowlist: when it is set, an absolute include pattern cannot reach a host it does not list. The URL is skipped and the crawl logs a warning that names the include pattern and the host. List the host in domains, or leave domains empty, if the include should reach it.

Recommendation

For most sites, leave domains empty and use include/exclude for fine control:

[project]
domains = []  # Default: seed domain only

[crawler]
include = []  # Empty = all URLs from seed domain
exclude = ["/admin/*", "*.pdf"]

Data storage

Project data is stored at:

~/.squirrel/projects/<project-name>/

For production domains:

For local domains:

Complete example

[project]
# Project name for local development
name = "ecommerce-staging"

# Allow main domain and all subdomains
domains = ["example.com"]

[crawler]
# Exclude admin and API endpoints
exclude = ["/admin/*", "/api/*"]

# Crawl up to 200 pages
max_pages = 200

Running squirrel audit https://example.com:

• Crawls example.com and all subdomains
• Skips /admin/* and /api/* paths
• Stops at 200 pages
• Stores data in ~/.squirrel/projects/example.com/

Related

• Crawler Settings - URL patterns, include/exclude
• Configuration Overview - Full config system
• init - Create config file