Mojibake: ’ and other encoding corruption in page text

What mojibake is

Mojibake is text that was encoded one way and decoded another. The most common form on the web is UTF-8 bytes read as latin1 or cp1252, which turns a right single quote into ’, an em dash into — and an accented é into é.

Two other shapes appear alongside it. The replacement character, U+FFFD, is what a decoder writes when it gave up on a byte sequence entirely, so it is unambiguous corruption. A literal   on screen means the source said  , an entity that was encoded one time too many, usually during an import.

What squirrel checks

The rule runs on every crawled page over the rendered prose in <body>, the same text content/placeholder-text reads. It leaves out <script>, <style>, <noscript>, the code-like elements <code>, <pre>, <samp> and <kbd>, <template> content the browser never renders, and the containers of in-page code editors and syntax highlighters (classes hljs, shiki, prism, torchlight, codeblock, code-block, cm-editor, monaco-editor, and any class starting language-, cm- or monaco-). Every block boundary, such as one table cell to the next, counts as a line break, so adjacent blocks do not fuse into one sequence. Attributes and raw HTML are not read, since a reader never sees them. It emits a single check named encoding-corruption:

• Skipped when there is no parsed document. Message: No document available.
• Skipped when there is no <body>. Message: No body element to read visible text from.
• Pass when nothing matched. Message: No encoding corruption in visible text.
• Warn or fail otherwise. Message: 14 encoding corruption occurrence(s) in visible text (utf8-as-latin1): charset is correctly utf-8, so the source itself is double-encoded: re-import the content from its original source.

The status is fail when the replacement character was found, since that is not arguable, and warn for the other two families. The cause named in the message depends on the page’s declared charset: no <meta charset> at all means the browser guessed; a correct utf-8 means the source bytes are already double-encoded; any other value means the declaration is wrong and the source may be too.

Eighteen byte sequences are matched, each a real UTF-8 lead byte rendered as its latin1 character, so legitimate French, German and Spanish copy stays clean. They are matched as one alternation ordered longest first, so a longer sequence is consumed whole rather than partly matched by a shorter entry.

How to fix it

<meta charset="utf-8" />

Fix the declaration first: <meta charset="utf-8"> as the first thing in the head, and Content-Type: text/html; charset=utf-8 on the response. If the declaration is already correct, the stored content is corrupt and the fix is upstream.

Re-import from the original source rather than replacing the visible symptoms. The same corruption is usually in fields you cannot see from the page: meta descriptions, alt text, structured data.

Enable / disable

Disable this rule

[rules]
disable = ["content/mojibake"]

Disable all Content rules

[rules]
disable = ["content/*"]

Enable only this rule

[rules]
enable = ["content/mojibake"]

enable replaces the default ["*"], so this is the only rule selected. Leave disable out: a rule that matches any disable pattern is skipped even when enable names it. Only an enabled setting under a rule’s own [rule_options] table overrides both lists.

Related rules

• content/unrendered-markup: literal markdown or escaped HTML reaching the reader.
• content/placeholder-text: template leftovers in the same body copy.
• core/charset: whether the page declares an encoding at all.
• content/mime-type: the header that carries the same charset parameter.

Content findings ship in every audit next to the SEO, performance and agent experience rules. See Fix SEO issues with AI for how an agent works through a report.

Glossary: Character encoding.

References

• Character encoding, MDN glossary
• Specifying the document’s character encoding, HTML Standard

Check your site

Run squirrel audit https://example.com and open the Content section of the report. Each page is listed with the corruption families found, a sample and a count. Signed-out local audits need no account and cost nothing. Signed-in audits spend credits.