Vendor reported
Nightfall
Its Developer API documentation reports at least 95% precision and recall, over 1,000 requests per second, and under 100 ms latency. These numbers were not rerun by RedactCloud.
Official documentation →Public benchmark report
Benchmarks with the methodology attached.
RedactCloud is measured on a versioned synthetic corpus. Microsoft Presidio is measured separately on the exact shared subset. Vendor-published market claims are kept distinct from our measurements.
Run date: July 26, 2026
Source commit:
76568808949e
Corpus: v1.0.0
- Value coverage recall
- 100.00%
- 53 true positive, 0 missed
- Value precision
- 100.00%
- 0 unmatched redactions
- Exact type labels
- 94.34%
- Coverage and label correctness are reported separately
- Clean case pass
- 100.00%
- 25 clean prompts evaluated
Same-corpus comparison
RedactCloud and Microsoft Presidio
This comparison includes 46 cases, 25 sensitive values, every clean case, and six shared PII types: credit card, email, IBAN, IPv4, US phone, and US SSN.
| Product | Version / configuration | Precision | Recall | F1 | Clean pass |
|---|---|---|---|---|---|
| RedactCloud | Structured patterns, NER disabled | 100.00% | 100.00% | 100.00% | 100.00% |
| Microsoft Presidio | v2.2.364, en_core_web_sm, threshold 0.0 | 92.00% | 92.00% | 92.00% | 96.00% |
This is a RedactCloud-run synthetic comparison, not an independent audit. Presidio supports custom recognizers and different NLP models that can change its result. Secret types without a shared default recognizer are excluded.
Engine latency
Measured without hiding network time inside the number.
These figures measure only the in-process redaction engine after 25 warm-up runs. They exclude Laravel, network, database, and upstream AI-provider latency.
| Payload | p50 | p95 | p99 | Throughput |
|---|---|---|---|---|
| 1 KiB | 0.008 ms | 0.009 ms | 0.011 ms | 117.31 MiB/s |
| 8 KiB | 0.043 ms | 0.047 ms | 0.095 ms | 179.24 MiB/s |
| 32 KiB | 0.159 ms | 0.167 ms | 0.185 ms | 196.97 MiB/s |
10,000 timed runs per payload on Apple M4, 16 GiB RAM, PHP 8.5.3.
Methodology
What was counted
-
01
Versioned corpus
69 fabricated English prompts cover supported structured secrets and PII, plus realistic clean lookalikes designed to expose false positives.
-
02
Coverage-based matching
A true positive requires the output redaction to fully cover the expected sensitive value. Exact detector labels are scored separately so a hidden value cannot be counted as leaked because of a label mismatch.
-
03
Failing command
The benchmark command exits unsuccessfully when any corpus case gains a false positive or false negative, making it usable as a release regression check.
Market context
Vendor claims are not treated as our benchmark results.
Commercial products require equivalent accounts, regions, policies, and request payloads for a fair head-to-head run. Until then, their official claims remain clearly marked as vendor reported.
Vendor reported
Private AI
Its product documentation advertises 50+ entity types, 52 languages, and up to 70,000 words per second on GPU. These numbers were not rerun by RedactCloud.
Official documentation →Documented coverage
Google Sensitive Data Protection
Google documents more than 200 built-in infoType detectors plus custom dictionaries, regex, context rules, and configurable transformations.
Official documentation →Documented coverage
AWS Comprehend
AWS documents 22 universal and 14 country-specific PII entity types, and explicitly recommends evaluation on your own content because performance varies by application.
Official service card →Inspect the evidence directly.
Download both machine-readable reports or reproduce them from the repository runner and pinned Presidio environment.