Privacy and preparation
Questions to ask a de-identification vendor before handing over records
By SourceX Editorial · Reviewed by Noah Loul ·
Short answer
Before handing records to a de-identification vendor, ask 15 questions across five areas: where processing happens, how long raw data is kept, which subprocessors touch it, what accuracy evidence exists on records like yours, and how deletion is proven. The vendor sees your data before it is cleaned, so its controls matter more than any later recipient's.
Key takeaways
- A de-identification vendor handles raw records, so its controls deserve more scrutiny than the eventual licensee's.
- Ask first whether the vendor's software can run inside your own environment instead of receiving a copy.
- Accuracy claims mean little until they are tested on a sample of your own record types.
- Ask whether any outside AI model API processes your records, and under which retention and training terms.
- Require a deletion certificate covering raw copies, outputs, backups, logs and token mappings.
Why the vendor review comes before the first file#
The vendor review comes before the first file because a de-identification vendor sees your records in their rawest form: names, emails, signatures, attachments and everything else the process is meant to remove. Whatever the eventual licensee receives, the vendor receives more.
That makes the vendor a processor of personal data in the full sense, usually bound by a data processing agreement, and it makes the answers to the questions below as important as price or speed. Ask them in writing and attach the answers to the agreement.
One option changes many answers at once: running the vendor's software inside your own cloud account or network, so raw records never leave. Ask whether that is offered before evaluating the rest.
Where processing happens: questions 1 to 3#
Processing location questions establish who physically holds the raw data and under which jurisdiction. The answers also decide which of the later questions apply.
| Question | Listen for |
|---|---|
| 1. Can the software run in our environment, or must records be sent to you? | A deployable option, or a clear description of the vendor environment |
| 2. In which countries and cloud regions will raw records be stored and processed? | Named regions, with a commitment not to move data without notice |
| 3. Will any outside AI model or API process our records? | Named providers, their retention and training terms, and a way to opt out |
Retention, subprocessors and access: questions 4 to 9#
Retention, subprocessor and access questions define how long raw records exist outside your control and how many people and companies can reach them. Vague answers here are the most common warning sign in vendor reviews.
Subprocessor lists change over an engagement. Ask for advance notice before a new subprocessor touches raw records, a right to object, and confirmation of whether the vendor's own cloud provider appears on its list.
| Question | Listen for |
|---|---|
| 4. How long are raw records and intermediate files kept? | Retention tied to the project, with deletion at completion |
| 5. Which subprocessors touch raw records? | A complete list with locations and roles, plus notice of changes |
| 6. Are our records used to improve your models or rules? | No by default, or only with written opt-in |
| 7. Who at your company can access raw records? | Named roles, background checks and need-to-know access |
| 8. Do offshore or contract reviewers see raw records? | A direct answer, with the agreements and locations involved |
| 9. Which independent security reports can you share? | Current reports whose scope covers the processing environment |
Accuracy evidence: questions 10 to 12#
Accuracy evidence is where vendor claims are hardest to verify. A detection score on a public benchmark says little about your support tickets, CAD notes or dispatch comments, which are full of abbreviations, product codes and half-sentences.
Ask for the test to run on records your team has already labeled, so the vendor's results are compared with a known answer rather than with its own judgment of what it found.
Expect honesty on the last question. Every automated approach misses something; Presidio's documentation, for example, says there is no guarantee it will find all sensitive information and that additional protections should be used. A vendor claiming complete detection is describing its marketing, not its testing.
- 10. Will you run a test on a sample of our own record types and report results by category?
- 11. How do you measure missed identifiers, and who reviews the misses?
- 12. Which categories do you handle least well, such as names in signatures, text inside scanned images or spoken numbers in transcripts?
Audit trail and deletion: questions 13 to 15#
Audit and deletion questions decide what you can prove later, to your counsel, to a licensee or to a regulator. Ask for sample documents now rather than promises of them.
Ask how any token mapping is handled. A table linking tokens to real names is as sensitive as the raw data, so decide in advance whether it is returned to you, held under your control or destroyed, and make that decision part of the contract.
| Question | Listen for |
|---|---|
| 13. What audit logs do you keep of access to and processing of our records? | Per-user access logs and job logs available on request |
| 14. What documentation comes with the cleaned output? | A redaction log covering methods, tool versions, QA results and exceptions |
| 15. How do you prove deletion at the end? | An officer-signed certificate covering raw copies, outputs, backups, logs and token mappings |
Answers that should pause the engagement#
Some answers should pause the engagement until they are fixed in writing, whatever the price. Most involve data going somewhere you did not agree to, or evidence you will not be able to produce later.
Treat a pause as a negotiation point rather than an automatic rejection. Many vendors can switch off model improvement, name their subprocessors or extend deletion to backups when asked; the ones that cannot are telling you how the engagement will go.
- Raw records may be used to train or tune the vendor's own models.
- Records go to an outside AI API whose terms the vendor cannot show you.
- The subprocessor list is unavailable or described only in general terms.
- Accuracy is stated as a single figure with no test on your records.
- Deletion is promised for raw files but not for backups, logs or mappings.
Illustrative: a home services company compares two vendors#
Illustrative: a fictional plumbing and HVAC company with several branches wants its ServiceTitan job notes and customer call transcripts de-identified before a licensing review. Homeowner names, addresses and gate codes appear throughout technician notes.
The first vendor quotes a low price but sends text to an outside model API and cannot share that provider's retention terms. The second deploys its software in the company's own cloud account, runs a test on sample job notes, and reports that spoken phone numbers in transcripts were its weakest category. The company chooses the second vendor, adds a human review step for transcripts, and makes the redaction log and deletion certificate contract deliverables.
How SourceX approaches de-identification vendors#
SourceX treats de-identification as part of the Preparation step of the SourceX five-step transaction: Supply, Rights, Preparation, Approval and Delivery. Whoever performs the work, the supplier approves the method, and the results go into the privacy record of the SourceX Evidence Packet.
Where possible, preparation is scoped so raw records stay in the supplier's environment. SourceX does not host very large datasets; they stay on infrastructure the supplier controls or travel on encrypted drives.
Frequently asked questions
Do we need a data processing agreement with the vendor?
In most cases, yes. The vendor processes personal data on your behalf, and privacy laws and customer contracts often require written terms that limit its use, require security and set deletion. Counsel can confirm which terms your records and jurisdictions call for.
Can we send a sample before signing?
Use synthetic records, or a small sample under a signed NDA and DPA. A test on your real record types is valuable, but it is still a transfer of personal data, so it needs the same protections as the full engagement.
Is a vendor that uses large language models riskier?
Not inherently, but it adds questions: whether the model runs in the vendor's environment or through an outside API, whether prompts are retained, and whether the model provider may train on them. Get each answer in writing.
Should we run open-source tools ourselves instead?
Sometimes. Running tools internally keeps raw records at home, but it shifts tuning, review and documentation to your team. Many companies combine both: an internal first pass, then a vendor for review of the hardest categories under tight terms.
What should a deletion certificate say?
It should name the dataset and engagement, list every location deleted, including raw copies, outputs, backups, logs and token mappings, state the method and date, and be signed by an officer. Ask to see a sample certificate before signing.
Sources
- Presidio's documentation warns that because it uses automated detection mechanisms, there is no guarantee it will find all sensitive information, and additional systems and protections should be employed. Source
Related resources
See if your company qualifies
A short company assessment. No data uploads are needed.