JBIG2 Compression: Why Numbers Can Change in Scanned PDFs

JBIG2 is a compression method for black and white scans, with lossless and lossy modes. In lossy mode it stores one sample of each character shape and reuses it wherever a similar shape appears. If two characters look alike at scan resolution, such as 6 and 8, the wrong one can appear in the file, and it looks perfectly sharp.

That last point is what makes JBIG2 a data integrity issue rather than a quality issue. A blurry JPEG looks blurry, so people notice. A JBIG2 substitution error looks like a clean, legible digit. For scanned batch records, logbooks, certificates of analysis and anything else where a number matters, it is worth knowing how JBIG2 works and how to keep it out of your records.

What JBIG2 is

JBIG2 is an international standard (ITU-T T.88 and ISO/IEC 14492) for bi-level images, meaning images where every pixel is either black or white, as with most scanned text pages. The JPEG committee's overview describes it as prepared for lossy, lossless and lossy-to-lossless compression, and says pattern matching and substitution techniques allow lossy compression at much higher ratios with almost no visible quality loss.

PDF has supported JBIG2 since version 1.4 through the JBIG2Decode filter. ISO 32000-1, the PDF 1.7 standard, describes the core idea in section 7.4.7: the encoder builds a table of unique symbol bitmaps found in the image, other symbols found later are matched against the table, and matching symbols are replaced by an index into the table. Table 6 of the same standard is careful about the result: the filter reproduces the original monochrome image data "or an approximation of that data".

The standard also notes its main advantage: for a page full of text, typical compression ratios are 20:1 to 50:1, considerably better than the older CCITT fax methods.

Lossless vs lossy JBIG2

The danger is specific to the pattern-matching step. If the encoder's test for "similar enough" is too loose, or the scan resolution is too low to tell two characters apart, a symbol from one place is reused somewhere it does not belong. Ordinary lossy compression such as JPEG degrades pixels; lossy JBIG2 can replace meaning.

The Xerox case

The most widely documented example came to light in 2013. David Kriesel reported on his blog in August 2013 that Xerox WorkCentre scanners were altering numbers in scanned documents: on a scanned construction plan, room area figures no longer matched the rooms, and in a cost table a 65 appeared as an 85. Whole patches of the image had been replaced, so the wrong numbers looked entirely genuine. He attributed the cause to the JBIG2 pattern matching used in the scanners' compression and stressed it was not an OCR problem.

His write-up records the timeline. Xerox at first said the issue arose only with non-default settings; PCWorld reported Xerox's statement on 7 August 2013 that "You will not see a character substitution issue when scanning with the factory default settings", and that a software patch would disable the highest compression mode. Kriesel then reproduced errors in other compression modes, reports that Xerox confirmed the issue, and later tested patches that eliminated the errors in his tests.

The lesson is not about one manufacturer. It is that a scanner's compression setting can change the content of a record, silently, and that default settings and vendor assurances need checking against your own tests.

Why it matters in GMP and other regulated work

Data integrity guidance assumes a copy has the same information as the original. The MHRA's GxP data integrity guidance (March 2018, section 6.11.2) defines a true copy as one verified, by a dated signature or through a validated process, to have the same information as the original, and says true copies may replace originals, such as a scan of a paper record, only if a documented system verifies and records the integrity of the copy. It also asks organisations to consider the risk of destroying original records. The FDA's data integrity guidance (December 2018, question 9) similarly says electronic copies can be used as true copies provided they preserve the content and meaning of the original.

A scanning process that can substitute digits cannot meet that standard unless it is controlled. Weights, lot numbers, expiry dates, temperatures and handwritten dates are the kind of small, repeated characters where a substitution matters most.

How to tell if a PDF uses JBIG2

You can check without special software. Image dictionaries are never hidden inside compressed object streams, because ISO 32000-1 section 7.5.7 says stream objects cannot be stored in an object stream. So:

1. Make a copy of the PDF and open the copy in a plain text editor.
2. Search for JBIG2Decode.
3. If it appears, at least one image in the file uses JBIG2. If it does not, check for CCITTFaxDecode (lossless fax compression), DCTDecode (JPEG) or FlateDecode.

What the file cannot tell you is whether the JBIG2 encoding was lossy or lossless. The same filter decodes both. For that you need the settings of the scanner or software that produced it.

How to keep JBIG2 substitution out of your records

• Find the setting. Check your scanners and capture software for JBIG2, "high compression", "small file" or similar modes, and record what each produces. Fix the configuration and control changes to it.
• Prefer lossless methods for records. The FDA's PDF Specifications (version 4.1) recommend compressing submission files with Zip/Flate or CCITT Group 4, describing both as lossless. The Library of Congress describes Group 4 as lossless compression of bitonal images, with compression ratios typically around 15:1.
• Scan at an adequate resolution. The FDA specification recommends 300 dpi for typical black-ink documents and handwritten notes, and advises against resampling to a lower resolution after scanning.
• Test with a stress page. Scan a page of small, similar digits and characters (6 and 8, 1 and 7, 0 and 8) at each setting you use, and compare the result with the paper character by character.
• Verify before you destroy paper. If scans replace originals, your true copy procedure should include checking critical values against the source.

What our tools do with bitonal scans

Compress PDF does not create JBIG2. Its standard engine only re-encodes JPEG images and 8-bit Flate images in RGB or greyscale; 1-bit images and images already stored as JBIG2 or CCITT fax are left exactly as they are. If the standard engine fails on a file, the tool falls back to a second engine that can re-encode images as JPEG, and then to a third that turns whole pages into JPEG images; the result shows "via PyMuPDF" or "via Rasterise" when that happens. For scanned records, check that label. Our guide on compressing a PDF explains the settings.

OCR PDF adds an invisible text layer on top of the existing page images and leaves the images unchanged. It cannot detect a JBIG2 substitution: if the image shows an 8 where the paper said 6, OCR will faithfully read 8. Proofread OCR output against the source, as our guide on making a PDF searchable with OCR recommends.

This article is general information, not regulatory advice. Scanning procedures for GMP records should be defined and verified under your organisation's quality system.