To judge whether a redacted document is safe to share, count the sensitive words the tool left readable. We ran that test in October 2026 on 48 fictional documents, with every piece of personal data labeled. Presidio's default install left 1,327 of 4,489 sensitive words readable. promptShield 1.4.1 left 92.
Why counting detections is misleading
A raw count of detections mixes real finds with noise and cannot tell the two apart. Our first comparison, in May 2026, counted detections this way, and that was the wrong measure.
Two numbers matter instead. The first is the number of sensitive words left readable. The second is how many harmless items a tool wrongly flags, since a reviewer has to clear each one by hand.
How the test was built
The corpus has six documents in each of eight languages: English, French, German, Spanish, Italian, Dutch, Portuguese and Polish. Each set holds a contract, an invoice, a bank statement, a medical letter, an HR letter and legal correspondence. The documents are fictional and were written for this test.
AI agents wrote the documents and their labels from a published labeling specification, with each agent covering one language. They had no access to promptShield's detection code. The labels were checked by a script, not by a human annotator.
Both tools read the same extracted text. Presidio ran with its built-in recognizers and one spaCy model per language. It had no custom recognizers and no score threshold. We ran it once with the large models and once with the small ones.
The files are clean, born-digital PDFs, so the test does not cover scans or OCR errors. Scanned documents are covered in a separate article.
Where each tool missed
With the small models, Presidio left 1,583 words readable, catching 64.7%. The table below shows the share caught in each category, using Presidio's large models.
| Category | Presidio (large) | promptShield 1.4.1 |
|---|---|---|
| Names | 87% | 98% |
| Organizations | 57% | 98% |
| Postal addresses | 50% | 98% |
| ID numbers | 37% | 95% |
| IBANs | 79% | 100% |
| Phone numbers | 78% | 98% |
| Dates of birth | 72% | 100% |
| Email addresses | 100% | 99% |
Presidio's misses fall where the risk is highest. It caught 37% of ID numbers, a category that covers national IDs, tax and company numbers, and account and patient numbers. In a US file, that is the category a Social Security number would fall into. Postal addresses came in at 50%.
Presidio led on email addresses, 100% to 99%. promptShield left nothing readable in 24 of the 48 documents, while Presidio left something readable in all of them.
Presidio needed one extra configuration step for Polish. Without a label mapping, Presidio does not recognize the entity labels from the Polish spaCy model and finds no Polish names (20.5% caught). With that one mapping it caught 64%, compared with 97% for promptShield.
The other number: wrong highlights
Each wrong highlight is a harmless piece of text that a reviewer has to remove. Presidio averaged 9.8 per document with the large models and 13.7 with the small ones. promptShield averaged 1.9.
No local model finds every identifier, so review stays part of the job. In promptShield, a file with protected regions cannot be downloaded until they are verified in Review. Pressing Download early opens a prompt that explains what is blocking the download and offers to open the review dialog.
How promptShield's results have changed
Across seven languages, release 1.0.14 (June 2026) caught 74.3% with 4.5 wrong highlights per document. That was close to Presidio's catch rate, with less noise. Four rounds of detection work, from September 30 to October 2, 2026, raised the catch rate to 98.1% in release 1.3.4, with 2.0 wrong highlights per document. Polish support was added in 1.4.0, so it is not part of that history.
When Presidio is the better choice
Presidio was built at Microsoft and is now governed by its community. It is solid software. Choose it when you need a Python library inside your own pipeline, running over text streams such as logs or chat messages. It gives you library-level control and lets you plug in a cloud analyzer through the same interface.
promptShield does a narrower job: it handles one self-contained document at a time, on one machine, with a person checking the result. A tuned Presidio would do better than the default setup we measured. Our run used no custom recognizers and no score threshold.
For that narrower job, the number of words left readable is what counts. A single readable account number can be enough to breach a confidentiality agreement. HIPAA's Safe Harbor method requires removing dates of birth before health data counts as de-identified. When the people in a document are in Europe, the GDPR or the UK GDPR applies as well.
Rerun the benchmark yourself
The documents, the labels, both tools' raw output and the scoring script are public in the promptshield-Inc/pii-detection-benchmarks repository on GitHub. You can rerun Presidio without installing promptShield. The comparison page has the charts, the release history and the methodology. Every figure here comes from the October 2026 run.
Frequently asked questions
Is Presidio accurate enough to redact documents?
In our October 2026 run, its default install caught 70.4% of sensitive words. Its weakest categories were ID numbers, postal addresses and organization names.
When should I choose Presidio instead?
When you need a Python library that runs over text streams and gives you control at the library level, Presidio is the better fit.
Can I rerun this benchmark myself?
Yes. The documents, the published labels and scorer, and both tools' raw output are in the public repository. Presidio runs without promptShield installed.
Would a tuned Presidio close the gap?
It would do better than the default install we measured. You can measure how much better by testing it against the same labels.