Skip to main content
Checks linked PDF sizes against Googlebot 60MB truncation limit
Rule IDcrawl/pdf-size
CategoryCrawlability
ScopeSite-wide
Severityerror
Weight4/10

​
Solution

Googlebot truncates PDFs at 60MB—content beyond that limit is ignored during indexing. Split large documents into smaller parts, compress images within PDFs, or add a noindex X-Robots-Tag header if the PDF doesn’t need to appear in search results.

​
Options

This rule supports the following configuration options:
OptionTypeDefaultDescription
warn_bytesnumber31457280 (30MB)Byte size to trigger warning
error_bytesnumber62914560 (60MB)Byte size to trigger error (Googlebot truncation limit)
max_pdfs_to_checknumber50Maximum number of PDF links to check

​
Configuration Example

squirrel.toml
[rules."crawl/pdf-size"]
warn_bytes = 31457280
error_bytes = 62914560
max_pdfs_to_check = 50

​
Enable / Disable

​
Disable this rule

squirrel.toml
[rules]
disable = ["crawl/pdf-size"]

​
Disable all Crawlability rules

squirrel.toml
[rules]
disable = ["crawl/*"]

​
Enable only this rule

squirrel.toml
[rules]
enable = ["crawl/pdf-size"]
disable = ["*"]