Regex vs NER vs LLM redaction: which catches more PII in free text?

Which method catches more PII in free text?#

No one method catches the most PII in free text, because regex, NER and LLM detection are built for different kinds of identifiers. The useful question is which combination leaves the fewest misses in your own records at a cost and speed you can accept.

Mature open-source tools already combine methods. Presidio, an MIT-licensed PII toolkit, combines named-entity recognition, regular expressions, rule-based logic and checksums with context. Its documentation also warns that automated detection gives no guarantee of finding all sensitive information and that additional systems and protections should be employed.

Presidio has moved from a Microsoft-owned project to an independent, community-governed one under the Data Privacy Stack organization on GitHub, a change recorded in a June 2026 release, and it remains MIT-licensed.

Decision matrix: which method fits which identifier#

A decision matrix assigns each identifier type to the method that handles it best, with the others as backup. The ratings below are qualitative and describe typical behavior on business text such as support tickets, email and CRM notes.

Read the matrix as a starting point, not a verdict. A support desk that pastes logs needs heavier secret rules, while a consulting firm's email needs stronger name and context handling.

What regex does well and where it fails#

Regex does well on anything with a fixed shape: email addresses, phone formats, card numbers validated with a checksum, government ID formats, IP addresses and known key prefixes. It is deterministic, fast, cheap and easy to audit, and it behaves the same on next year's records as on this year's.

Regex fails on anything that depends on meaning. It cannot tell a person's name from a product name, and it over-matches numbers that look like IDs, such as order numbers, part numbers and invoice references. Over-matching quietly destroys useful content, so test every pattern against real records before running it at scale.

What NER adds, and what it misses#

NER, or named-entity recognition, adds the ability to find people, organizations and places from context, which regex cannot do. Cloud services offer the same idea as managed APIs: the Amazon Comprehend developer guide, in its June 2023 snapshot, listed 22 universal PII entity types, including NAME, ADDRESS, EMAIL and PHONE as well as credential types such as PASSWORD and AWS_SECRET_KEY.

NER models are often trained on clean, formal text, and business records are not clean. Lowercase names in chat, first names on their own, names that are also ordinary words, and agent or technician names inside templated signatures are common misses. Product and feature names are common false positives.

Test any model on a sample of your own tickets or emails before trusting it. Performance on news articles says little about performance on a dispatcher's shorthand or a support agent's macro.

What an LLM pass adds, and the risks it brings#

An LLM pass adds judgment about context: it can flag that the only site manager at a named depot is identifiable, or that a note about a caller's oxygen equipment is health information. Those contextual leaks are exactly the misses that regex and NER leave behind.

The risks are real. Output can vary between runs, models sometimes rewrite text they were asked only to inspect, and long archives make the pass slow and costly. Sending records to an external API also raises its own data handling questions before any redaction happens.

Use the LLM to return flagged spans with a reason, not to rewrite the record, and apply replacements with deterministic code. That keeps every edit auditable and lets a reviewer accept or reject each flag.

A layered redaction pipeline#

A layered redaction pipeline runs cheap, precise methods first and expensive, contextual ones last, then measures what is left with human review. Each layer reduces the load on the next.

The dictionary step is often underrated. Your directory of employees and your list of customer contacts name exactly the people most likely to appear, and an exact match on those lists catches names that generic models miss.

• 1Normalize text: strip HTML, extract text from in-scope attachments and keep record IDs.
• 2Run a secret scanner and rotate any live credential it finds.
• 3Apply regex rules with checksums for structured identifiers.
• 4Match exact names from your employee directory, customer list and vendor list.
• 5Run an NER model for remaining person, organization and place names.
• 6Run an LLM pass over residual text to flag indirect identifiers and sensitive context.
• 7Replace every finding with consistent placeholders using deterministic code.
• 8Review a random sample of processed records by hand and log every miss.

Illustrative: a field service software company tests three approaches#

Illustrative: a fictional company that sells field service software to HVAC and plumbing contractors wants to prepare years of Zendesk tickets. The tickets mix customer emails, agent replies, pasted error output and technician notes that customers forwarded.

A regex-only pilot removed emails, phone numbers and keys cleanly but left technician first names, customer company names and a ticket that described a homeowner by their medical equipment. Adding a dictionary of agent and customer names plus an NER model fixed most name misses. An LLM flagging pass, run on a model hosted inside the company's own cloud account, caught most of the remaining contextual cases.

The reviewers' sample showed far fewer misses with all layers than with regex alone, and the team documented which layer caught which category. That record answered counsel's first question about the method.

How SourceX approaches redaction methods#

SourceX treats the redaction method as part of the Preparation step in the SourceX five-step transaction, chosen per record family rather than one tool for everything. The supplier approves the method and the sample review results before Approval.

The layers used, the sample reviewed and the misses found are recorded in the privacy record of the SourceX Evidence Packet.