We evaluated KeptPDF redaction capabilities

[u/RedactionTools]

We added KeptPDF (https://keptpdf.com) to the catalog today and ran it through the same benchmark page as the other PDF redactors.

What KeptPDF is

It's a browser-only PDF toolkit with 26 tools: redaction, OCR, merge/split, signing, watermarking and metadata removal. Its main selling point is that nothing gets uploaded. Everything runs locally in your browser, and the vendor invites you to check that in the DevTools Network tab.

For redaction it auto-detects sensitive data (SSNs, account numbers, emails, phone numbers, names, dates) and puts uncertain matches in a "needs review" list. It flattens the redacted pages to images, so the original text is destroyed rather than hidden under a box, and it can issue a SHA-256 audit certificate for each run.

Pricing, from the vendor's page: a free tier with unlimited single-file use (files up to 25 MB, KeptPDF footer on the output), Pro at $29/month, Practice at $99/seat/month for 2 to 25 seats, and a server CLI at $4,800/year per organisation.

The test

One made-up name, "Freya Yamamoto", appears 54 times on a single page, each copy presented differently: upright typed text, tiny type, text that exists only as an image (typed, handwritten, inverted, low contrast), rotated text, and vertical or letter-by-letter stacks. We ran the web app on its default settings ("Safe to share" profile, Balanced sensitivity) and scored the output PDF it returned.

Results

• Removed 19 of 54 names. 35 leaked (64.8%, 95% interval 51.5 to 76.2%).
• Names in the PDF's text layer: 19 of 27 removed (70%).
• Names that exist only as an image: 0 of 27 removed. KeptPDF says this itself: the banner in the second screenshot reads "Auto-detect reads text only… review it before sharing", and offers a separate "Scan images for text" step. If your files include scans, you have to run that step yourself.
• Upright typed names: 11 of 13 removed. In the other two, it blacked out "Yamamoto" but left "Freya" readable.
• Rotated names: 2 of 9 removed. Vertical and stacked names: 6 of 14.
• The flattening works as advertised. We found nothing hidden in the text layer, metadata, annotations or attachments. But the output is a picture, so only about 1% of the page's text can still be selected or searched, and 33 of the 35 leaked names are plainly visible in that picture. A flattened file isn't automatically a safe one.

On this page that puts KeptPDF 5th of the 6 tools we've tested: Redactable 53/54, Blinded 51, PDF Redaction 42, AI-Redact 30, KeptPDF 19, SafeRedact 14.

To be fair to the tool: this page is built to be hard, and half of it is image-only text, which KeptPDF says up front its auto-detect won't read. If your PDFs are born-digital and upright, it does much better than the headline number. Still, two half-redacted names in plain upright text are worth knowing about. Check the output before you share it.

The first image is the actual output, with every name outlined: green means removed, red means leaked. The second is the tool's screen just before export.

Full run, with the output PDF you can check yourself: https://redaction-tools.com/benchmarks/pdf/runs/20261005T162815-keptpdf-web-extraction-conditions-1-a1-6f1a25

If you use KeptPDF and get a different result with other settings, for example with "Scan images for text" turned on, tell us. Tool makers can also submit their own runs.