Skip to content

Privacy and preparation

Dataset redaction utility experiment: measuring what each level costs

By SourceX Editorial · Reviewed by Noah Loul ·

9 sections

Short answer

To measure how much task utility survives each redaction level, prepare the same records at several levels, train or evaluate one model with an equal budget on each, and score every arm on a frozen holdout. Add a random-deletion arm at matched token share, report paired differences with clustered intervals, and track redaction recall alongside utility. The page sets out the method only and publishes no findings.

Redaction

Dataset redaction utility

This page covers dataset redaction utility experiment: measuring what each level costs.Illustrative

Key takeaways

  • Run identical records through a ladder of redaction arms that differ in one setting each, from typed tags to broad removal.
  • A random-deletion control at the same masked-token share separates the cost of removing personal data from the cost of removing text.
  • Fix training steps and tokens seen rather than epochs, because redaction changes token counts.
  • Score each arm on both a redacted and an unredacted holdout, and report paired differences clustered by account or thread.
  • Pre-register arms, thresholds and the smallest decision-changing gap; the page publishes the method and reports no results.

What a dataset redaction utility experiment answers, and what it does not

A dataset redaction utility experiment answers one question: for a fixed task and model, how much measured performance remains when the same records are prepared at progressively stronger redaction levels. Everything else stays fixed, so any difference in the task metric can be traced to the transformation and not to a change in data, budget or evaluation.

This page is a protocol. It gives the design, the arms, the variables, the metrics, the uncertainty rules and a pre-registration checklist. It reports no results, and none of the numbers below are findings; where a figure appears it is a design parameter you choose before the run.

The protocol exists because privacy and utility pull in opposite directions and are easy to argue about without evidence. NIST's report on de-identifying personal information (NISTIR 8053) describes this tension directly: more aggressive de-identification lowers privacy risk and leaves less utility in the resulting data. The experiment puts a measured number on the utility side for your own records and task, so the privacy reviewer and the data buyer argue from the same table.

The redaction ladder: which arms to run and what each isolates

Run at least five arms on identical source records, each produced by one pipeline with one configuration change. Microsoft's open-source Presidio toolkit names the standard operators: replace with a value, redact (remove the text completely), mask characters, salted hash, and encrypt. The ladder below uses those operator names so the configuration of each arm can be written down exactly. One setting needs care: Presidio's hash operator now generates a random salt per entity unless you supply one, so the R4 arm must pass a single explicit salt for each thread or its pseudonyms will not link.

Two arms are controls rather than redaction levels. The reference arm shows the ceiling and must stay inside the data owner's controlled environment. The random-deletion arm removes the same number of tokens as the strongest redaction arm, but from positions chosen at random, so you can tell whether utility loss comes from losing personal data or simply from losing tokens.

The redaction ladder: which arms to run and what each isolates
ArmTransformationOperator settingWhat it isolates
R0 reference (enclave only)No redaction; never leaves the owner's controlled environmentNoneUpper bound on task utility
R1 direct identifiersNames, emails, phone numbers, account numbers replaced with typed tagsreplace with entity tagCost of removing direct identifiers alone
R2 plus quasi-identifiersR1 plus dates generalized to month and locations to regionreplace plus generalization rulesCost of coarsening time and place
R3 surrogatesSame spans as R2, replaced with consistent fake valuesreplace with surrogate generatorWhether realistic surrogates beat tags
R4 consistent pseudonymsSame spans as R2, replaced with salted hashes stable within a record threadhashValue of keeping entity linkage across a thread
R5 broad removalEvery detected entity type removed, at a low confidence thresholdredactUtility left after aggressive, recall-first removal
C1 random deletionSame token count as R5 removed at random positionsCustom scriptToken loss versus loss of personal data

Population, sampling and the untouched holdout

Define the population as the records a buyer would actually receive: one system, one record type, one period, after the owner's normal filters. Write down the denominator, the date range and every exclusion before any arm is built, so nobody can quietly drop records that redact badly.

Split by time or by account, not at random across rows. Random row splits put messages from the same customer thread on both sides, and a model can score well by memorizing thread-specific details. Hold out the most recent slice, or a set of whole accounts, and do not look at it until the final run.

Pick tasks that match how the buyer will use the records, not tasks that are convenient to score. Support tickets bound for a resolution assistant call for resolution-code prediction and answer extraction; invoices bound for a document model call for field extraction. If the records will serve two uses, run both tasks on every arm, because a level that is harmless for classification can be costly for extraction, where the removed span is often the answer itself.

Size the holdout from the decision threshold, not from habit. Before building arms, estimate how many clustered items you need for the interval on a paired difference to be narrower than the smallest gap you said would change the decision. If the owner's records cannot supply that many accounts or threads, say so in the plan and widen the threshold instead of running an experiment that cannot answer its own question.

Decide which version of the holdout each arm is scored on. A deployment-matched score evaluates each arm on a holdout redacted the same way, which reflects how the model will see production inputs if those are also redacted. A reference-target score evaluates every arm on the unredacted holdout inside the enclave, which reflects how much true signal the model learned. Report both; they answer different questions.

Variables to hold fixed so the comparison stays fair

The comparison is only fair if every arm gets the same model, the same compute budget and the same evaluation. Redaction changes token counts, so equal epochs do not mean equal budget; fix training steps and tokens seen instead, and log both.

Variables to hold fixed so the comparison stays fair
VariableRole in the designHow to fix or record it
Base model and checkpointHeld constantOne checkpoint hash for every arm
Training budgetHeld constantSame steps and tokens seen; log actual tokens per arm
HyperparametersHeld constantTuned once on R1 validation data, then frozen for all arms
Random seedsVaried on purposeSame seed list for every arm, at least three per arm
Detector version and thresholdsDefines the armVersion string, entity list and threshold recorded per arm
Surrogate and hash settingsDefines R3 and R4Generator version, salt handling and consistency scope recorded
Task and metricOutcomeFixed before any arm is scored; no switching after a look
Holdout setOutcome populationFrozen IDs, stored separately, hashed for later audit

Metrics: task utility, redaction quality and residual risk

Measure three things for every arm, because a utility number on its own can be bought with a privacy leak. The first is the task metric you care about, such as field-extraction exact match, resolution-code accuracy, classification F1 or retrieval recall at a fixed cutoff. Report utility retention as the arm's score divided by the R0 score on the same holdout.

The second is redaction quality. The Text Anonymization Benchmark (TAB) scores text anonymization with recall as the privacy measure and precision as the utility measure, and counts an identifier as protected only when all its occurrences are masked. Use the same split: recall on a human-labeled audit sample tells you what leaked, precision tells you how much harmless text was removed.

The third is the masked-token share: the fraction of tokens each arm changed. Plot utility retention against masked-token share, with the C1 random-deletion arm on the same chart. If a redaction arm sits well above random deletion at the same token share, the detector is removing the right things; if it sits on or below it, the loss is coming from blunt removal.

  • Primary outcome: task metric on the frozen holdout, per arm, deployment-matched and reference-target.
  • Secondary outcome: utility retention relative to R0, with interval.
  • Redaction precision and recall per entity type on a double-labeled audit sample.
  • Masked-token share per arm and per field.
  • Residual identifier count found by a second detector run on each arm's output.

How to report uncertainty so small gaps are not oversold

Report every arm comparison as a paired difference on the same holdout items, with an interval, and never as two bare scores. Evan Miller's paper on adding error bars to language model evaluations recommends paired differences when two models answer the same questions, and clustered standard errors when questions come in correlated groups. Business records nearly always cluster, by account, ticket thread or document, so cluster at that level.

Seeds matter as much as items. Bouthillier and colleagues, studying variance in machine learning benchmarks, recommend randomizing as many sources of variation as possible, including weight initialization, data sampling and hyperparameter search, and report that these sources move benchmark outcomes. Train every arm on the same seed list and report the spread across seeds alongside the interval across items.

Set the smallest difference that would change a decision before you run anything. If the buyer would accept R3 over R1 only when the loss is below a threshold you name in advance, write that threshold into the pre-registration so the result cannot be reinterpreted after the fact.

Alternative explanations that can fake a redaction effect

Several things besides redaction can move the task metric between arms, and each needs a control in the design rather than a paragraph in the discussion.

Alternative explanations that can fake a redaction effect
Alternative explanationHow it shows upControl in this protocol
Plain token lossEvery arm that removes more text scores lowerC1 random-deletion arm at matched token share
Train and test mismatchArm looks worse only on the reference-target holdoutReport deployment-matched and reference-target scores separately
Label leakage through entitiesLabels mention a product or account that redaction removesAudit labels for redacted spans; redact labels consistently or exclude those items
Surrogate format cuesR3 scores oddly high because fake values carry patternsCheck that surrogate distributions match real ones on a sample
Detector errors correlated with classOne outcome class loses more text than othersReport masked-token share per class and per field
Budget driftArms with fewer tokens effectively train longer per tokenFix steps and tokens seen; log actual counts
Seed luckOne run per arm produces a striking gapSame seed list across arms; report spread

Pre-registration checklist and limits of the design

The Center for Open Science describes preregistration as posting a time-stamped, read-only version of a study plan to a repository before data collection or analysis begins. A private, dated, read-only copy shared between the data owner and the buyer serves the same purpose here: it proves the arms and thresholds were fixed before anyone saw a score.

Checklist · 10 items

  1. Research question in one sentence, naming the task, the model and the records.
  2. Population definition: system, record type, date range, denominator and every exclusion rule.
  3. Arm list with exact operator settings, detector versions, thresholds and entity lists.
  4. Holdout definition: split rule, frozen IDs and where the unredacted copy lives.
  5. Primary metric, secondary metrics and the smallest difference that would change a decision.
  6. Training budget in steps and tokens, frozen hyperparameters and the seed list.
  7. Uncertainty method: paired differences, cluster level and how intervals are computed.
  8. Redaction audit plan: sample size per entity type, two labelers and an adjudication rule.
  9. Privacy controls: who may see R0 and the reference-target holdout, where runs execute, and how outputs are checked before leaving the enclave.
  10. Stopping and deviation rules: what happens if a run fails, and how changes are logged and disclosed.

0 of 10 ticked. Tick each item as you go.

Limits, privacy sign-off and where the result goes next

The result applies to one task, one model and one record population. A different model size, a retrieval use instead of fine-tuning, or records from another system can change the ranking of arms, so treat each run as evidence for that configuration only. Redaction levels also do not map directly onto legal standards; whether any arm meets a particular de-identification requirement is a question for the privacy reviewer and counsel, not for this experiment.

Running the R0 arm needs privacy sign-off before it starts, because it uses unredacted records. Keep R0 training and every reference-target evaluation inside the data owner's controlled environment, export only scores and audit labels, and record who had access.

When the experiment informs a licensing specification, the useful output is the arm the buyer can accept and the evidence behind it. Where SourceX is involved, de-identification falls to SourceX through approved processing routes, and the signed agreement sets what the de-identified dataset may be used for. Buyers set out scope, volume, format and licensing needs, and SourceX sources a dataset from U.S. businesses to match that request instead of holding one in stock.

Frequently asked questions

How do you measure task utility after different levels of data redaction?

Prepare the same records at several redaction levels, train or evaluate the same model with the same budget on each, and score every arm on one frozen holdout. Report paired differences with clustered intervals, and add a random-deletion arm at matched token share so you can separate the effect of removing personal data from the effect of removing text.

Should the holdout set be redacted too?

Report both versions. A holdout redacted like each arm shows performance on production-like inputs, while an unredacted holdout scored inside the data owner's environment shows how much real signal the model learned. They answer different questions, so neither replaces the other.

Why include a random-deletion arm in a redaction experiment?

Redaction removes tokens, and losing tokens alone can lower a task metric. Removing the same number of tokens at random gives a baseline: if a redaction arm scores clearly above it at the same masked share, the detector is removing the right content; if not, the loss is mostly from blunt removal.

Are surrogates better than redaction tags for keeping data useful?

It depends on the task and the model, which is why the protocol runs tags, surrogates and consistent hashes as separate arms on the same records. Surrogates can keep text natural, while consistent pseudonyms keep entity linkage across a thread; only a measured comparison on your own records settles it.

Does this protocol report results?

No. It sets out the design, variables, metrics, uncertainty rules and pre-registration checklist for running the experiment. Any figures that would come from running it depend on your records, task and model, and no results are reported here.

Sources

  • NISTIR 8053 describes the trade-off in which more aggressive de-identification lowers privacy risk but leaves less utility in the data. Source
  • Presidio's anonymizer documents built-in operators including replace, redact, mask, salted hash and encrypt, and its hash operator uses a random salt unless a consistent salt is provided. Source
  • The Text Anonymization Benchmark evaluates anonymization with recall as a privacy measure and precision as a utility measure. Source
  • Miller recommends paired differences for models sharing a question set and clustered standard errors for correlated question groups. Source
  • Bouthillier et al. recommend randomizing sources of variation such as weight initialization, data sampling and hyperparameter optimization. Source
  • The Center for Open Science defines preregistration as a time-stamped, read-only study plan posted before data collection or analysis. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies