Privacy and preparation
Synthetic data privacy utility test: disclosure risk and task value
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
Assess disclosure risk in synthetic data by splitting the real records before generation into a generator training set, an unseen control set and an untouched utility holdout. Run singling out, linkability, inference and copy checks on the synthetic output, read each as excess success on training records over control records, and measure task utility on the holdout against limits set beforehand.
Redaction
Synthetic data privacy
Key takeaways
- Calling data synthetic does not settle whether it discloses its source; research comparing many generators found no reliable privacy advantage over traditional anonymization.
- Split real records by entity into generator training, privacy control and utility holdout sets before generating anything.
- Read every attack as excess success on training records over unseen control records, and report rare segments separately.
- Measure utility on untouched real cases against a real-trained baseline, and disclose how the synthetic set was produced.
- Test results inform the data controller's decision and counsel's view; they are not a legal determination.
How do you assess disclosure risk in synthetic data derived from private records?
Run privacy attacks against the synthetic output and compare how well they work on records the generator trained on versus real records it never saw. The gap between those two success rates is the disclosure risk the generator added. Measure task utility on a separate set of untouched real records in the same run, and decide on both against limits written down before anyone looks at results.
That design does three things a one-off privacy score cannot. It separates leakage from what anyone could guess about the population. It catches the generator that protects privacy by producing useless data. And it fixes the pass criteria in advance, so a disappointing number cannot be quietly renegotiated. The plan below sets out each test, the data it needs and how to read it.
The output is evidence for a decision, not a legal finding. Whether a synthetic dataset is still personal data under a given law depends on that law and on the facts, and counsel decides it. The companion page on whether synthetic data is personal data covers the legal side; this page covers the test plan.
Why does a dataset labeled synthetic still need a disclosure test?
A generator trained on private records can memorize and reproduce parts of them, especially rare combinations. Stadler, Oprisanu and Troncoso evaluated a range of generative models and found that synthetic data did not deliver a better privacy and utility trade-off than traditional anonymization: depending on the model, the output either failed to stop inference attacks or lost the utility it was meant to keep. They also found the outcome hard to predict in advance, which is the practical argument for testing every release.
Regulators frame the question as residual risk. The Article 29 Working Party's opinion on anonymization techniques judges a technique by whether a person can still be singled out, whether records about them can be linked, and whether information about them can be inferred. NIST SP 800-188 lists publishing synthetic data based on identified data as one of several release models for agencies, and recommends governance such as a disclosure review board and re-identification studies to gauge risk. Neither treats the act of generation as proof of anonymity.
Why split the real records before the generator sees any of them?
Make a three-way split of the real data first: a training set for the generator, a privacy control set the generator never sees, and a utility holdout that is also never seen and is not used for tuning. Every later test depends on that split being clean, and it cannot be repaired after generation.
The control set is what makes privacy results interpretable. The Anonymeter tool asks for the original records used for generation, the synthetic output and a control set of original records not used for generation; it compares a main attack on the training records with a control attack on the unseen ones and a naive baseline attack that ignores the synthetic data. SDMetrics' DCR overfitting metric uses the same idea, comparing how close synthetic rows sit to the training data against how close they sit to a holdout.
Split by the entity the privacy question is about, not by row. If a customer has forty tickets, all forty go into one split; otherwise the control set shares people with the training set and the comparison understates the risk. For records that change over time, also hold back the most recent period, so utility is measured on cases the generator could not have imitated.
What does the privacy and task utility evaluation plan contain?
Use this plan as a pre-registration form: fill in the last column with your own limits and sign it before generation. The tool examples are open-source implementations that run each test; they are not the only options. Version 1.0, dated 2026-10-18.
| Test | Question it answers | Data it needs | How to read it (limits set beforehand) | Tool example |
|---|---|---|---|---|
| Exact copy check | Did the generator reproduce whole real rows? | Training set, synthetic set | Any copied row of a real entity is a stop until explained | SDMetrics NewRowSynthesis |
| Distance to closest record | Do synthetic rows sit closer to training rows than to unseen ones? | Training set, control set, synthetic set | Closer to training than to control signals overfitting | SDMetrics DCROverfittingProtection |
| Singling out | Can an attacker isolate one real person from the synthetic data? | Training, control and synthetic sets | Main attack success minus control attack success, against your limit | Anonymeter SinglingOutEvaluator |
| Linkability | Can two partial views of a person be joined through the synthetic data? | Training, control and synthetic sets, split into column groups | Same control-adjusted reading; report outliers separately | Anonymeter LinkabilityEvaluator |
| Attribute inference | Given columns an attacker knows, can a sensitive column be guessed? | Declared known and sensitive columns | Compare against the control and the baseline guess | Anonymeter InferenceEvaluator |
| Membership inference | Can someone tell whether a person was in the training data? | Training and control records, the generator or its output | Attack accuracy near chance passes; report the best attack tried | Shadow-model attacks after Shokri and others |
| Canary exposure | Does a text generator reproduce secrets planted in its training data? | Random canary strings inserted before training | Any canary reproduced or ranked far above chance is a stop | Exposure test after Carlini and others |
| Train on synthetic, test on real | Does a model trained only on synthetic data work on real cases? | Synthetic set, utility holdout | Compare with the same model trained on real training data | Any model of the intended task |
| Augmentation lift | Does adding synthetic data to real data help on real cases? | Real training, synthetic set, utility holdout | Lift on the holdout must beat real-only training | SDMetrics data augmentation metrics |
| Rare case coverage | Which real exceptions does the synthetic data miss or distort? | Utility holdout grouped by segment | Per-segment results; segments below the limit are disclosed or excluded | Grouped evaluation on the task metric |
How do you read the privacy results without fooling yourself?
Read every attack as an excess over the control. If an inference attack guesses a sensitive field correctly for many training records but just as often for control records, the synthetic data revealed population patterns, not individuals. The Anonymeter documentation describes exactly this: the control attack separates what the attacker learns about the population from what it learns about the training records, and the baseline attack is a sanity check that the main attack beats random guessing.
Look at the tails, not just the averages. Disclosure concentrates on unusual people: the only customer in a small region, the account with an extreme value. Stadler and colleagues found that differentially private generators reduced linkage risk for outliers, which is also a reminder of where the risk lives. Report results for the rarest segments separately and set the limit for them explicitly.
Match the attack to the threat. Membership inference, as Shokri and colleagues showed for classifiers, assumes black-box access to a model; it matters when you will release the generator or a model trained on it. If you release only a fixed table, the table-based attacks are the primary evidence. For text generators, planted canaries are cheap insurance: the Secret Sharer work measures how much more readily a model reproduces inserted random sequences than comparable sequences it never saw.
How is task utility measured on real cases the generator never saw?
Utility means performance on the task the synthetic data is for, measured on the untouched real holdout. Train on synthetic, test on real was proposed for evaluating generated medical time series and has become a standard check: train a model on generated data and test it on held-out real data. SDMetrics' augmentation metrics take the complementary view, training on real plus synthetic and evaluating on real validation data.
Always report a real-data comparison trained the same way. A synthetic-trained score means little on its own; what matters is how far it falls from the real-trained score and whether adding synthetic data helps at all. Hold the model, features and tuning budget fixed across arms.
Disclose how the synthetic set was produced, because utility depends on it: the generator and version, the grounding records and their dates, any filtering of generated rows, the share of synthetic rows in the final mixture and where its distribution departs from the real one. Filtering generated rows on the holdout is a common leak; it turns the holdout into a tuning set.
How does the plan read on a synthetic help desk table? (illustrative)
This example is illustrative. The records, segments and percentages are invented to show how the plan reads in practice, and are not from any SourceX supplier.
At a glance
How does the plan read on a synthetic help desk table? (illustrative)
A team wants a synthetic version of a help desk ticket export from a system like Zendesk, with product, plan tier, region, issue category, first-response time, escalation flag and a short issue summary, to train an escalation classifier. They split by customer account: 70% of accounts to generator training, 15% to the privacy control set and 15% to the utility holdout, with the latest quarter forced into the holdout.
- Pre-registered limits: no copied rows; control-adjusted singling out and inference risk below the team's stated limit; synthetic-trained classifier within an agreed margin of the real-trained one on the holdout; every segment reported.
- Exact copy check: zero copied rows. Distance to closest record: synthetic rows no closer to training accounts than to control accounts.
- Inference attack with region and plan tier known and escalation flag sensitive: overall excess over control is small, but for one enterprise tier in one small region, a handful of accounts, the attack recovers the flag far more often for training accounts than for control accounts.
- Utility: the synthetic-trained classifier lands close to the real-trained one overall but performs poorly on the same small segment, which the generator either copied or distorted.
- Free-text summaries: two planted canary strings reappear in generated summaries, so the text column fails regardless of the tabular results.
- Decision: drop the free-text column from this release, coarsen region for the small enterprise segment and regenerate, then rerun the whole plan. Disclose that the segment is coarsened and excluded from utility claims.
When can't the test plan settle the question?
Some outcomes call for a stop or a specialist rather than another run. Stop when the generator was trained on records the team had no right to use for this purpose, since no test result repairs that. Stop when exact copies or canaries appear and their cause is unknown. Bring in a statistical disclosure specialist when the data has many rare combinations, long linked histories per person or free text, where the tabular attacks above understate risk.
Formal privacy methods change the plan but do not remove it. A differentially private generator limits how much any one record can influence the output for the stated parameters, but implementation errors, preprocessing on the full data and a weak privacy budget can undo that, so empirical attacks remain a useful check.
Keep the decision with the data controller. The Anonymeter documentation reports the French data protection authority's view that its risk scores should be used by the controller to decide whether residual re-identification risk is acceptable. Test results inform that judgment; they do not make it.
Why license real records as the reference set?
Every test above depends on real records: a generator training set, a control set and a utility holdout from the same population. Teams that lack the third often discover it only after generation. Licensing a real reference set with terms that cover generation, testing and any synthetic derivatives is cleaner than borrowing one whose terms are silent on derived data.
SourceX looks for a reference set like this among U.S. businesses after a buyer asks rather than keeping one on hand; the request form covers scope, volume, language, format, timing and licensing needs, and a holdout or control requirement can be written there. Delivery waits for the supplying company to approve the scope and terms and sign.
Keep the permissions distinct when scoping such a set. Having access to a help desk export is not the same as a privacy basis for sharing it, and neither is the same as the right to license it for training or synthesis. De-identification reduces privacy risk but does not by itself create a licensing right.
Frequently asked questions
Is synthetic data automatically anonymous?
No. A generator trained on personal records can reproduce or reveal parts of them, and research comparing many generators found no reliable privacy advantage over traditional anonymization. Whether a given synthetic dataset is anonymous depends on test results and on the applicable law, which counsel assesses.
Why do I need a control set of real records?
Because attacks also succeed by guessing population patterns. Comparing attack success on training records with success on unseen control records isolates what the generator leaked about the people it trained on, rather than what anyone could infer about the population.
Can I use the same real holdout for privacy and utility tests?
It is safer to keep two. The utility holdout tends to get reused while tuning the generator or filtering its output, and once that happens it no longer measures unseen data. A separate privacy control set stays clean for the attacks.
Does differential privacy make these tests unnecessary?
It reduces the need but does not remove it. The formal bound covers the stated parameters and a correct implementation, and preprocessing or errors outside the private training step can still leak. Empirical attacks and canaries check the pipeline as built.
How should results be shared with a data buyer or supplier?
Share the pre-registered limits, the split method, each test's result against the control and baseline, per-segment utility, the generation recipe and the date the policy was applied. Note any segment that was coarsened or excluded so the reader knows where the synthetic data does not apply.
Sources
- Stadler, Oprisanu and Troncoso found synthetic data did not provide a better privacy and utility trade-off than traditional anonymization, with outcomes hard to predict, and that differentially private generators reduced linkage risk for outliers. Source
- The Article 29 Working Party's Opinion 05/2014 evaluates anonymization techniques by the risks of singling out, linkability and inference. Source
- NIST SP 800-188 lists publishing synthetic data based on identified data as a release model and recommends a disclosure review board and re-identification studies. Source
- Anonymeter evaluates singling out, linkability and inference using original, synthetic and control data, with control and baseline attacks, and reports the CNIL's view on how controllers should use its scores. Source
- SDMetrics' DCR Overfitting Protection compares how close synthetic data is to real training data against a holdout set. Source
- SDMetrics' NewRowSynthesis measures whether each synthetic row is new or exactly matches a real row. Source
- SDMetrics' data augmentation metrics train on real training data plus synthetic data and evaluate on real validation data. Source
- Shokri and colleagues introduced membership inference attacks that use shadow models and black-box access to tell whether a record was in a model's training set. Source
- The Secret Sharer inserts random canary sequences into training data and measures their exposure relative to non-inserted sequences. Source
- Hyland, Esteban and Ratsch proposed training on synthetic data and testing on real held-out data to evaluate generated medical time series. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.