AI data market
Synthetic dataset buyer disclosure: what to request before you buy
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
Buyers should request a disclosure sheet that names the real seed data and its rights, the generator, version and terms, the generation recipe, every filtering step and its rejection share, the synthetic-to-real mixture, the date of any encoded rules, results on a real holdout the generator never saw, privacy screens, negative controls and known gaps. Missing answers are a reason to narrow or pause the purchase.
Provenance trail
Synthetic dataset buyer
Key takeaways
- Ask for provenance, selection and real-holdout evaluation in writing; a fidelity score alone answers none of them.
- The generator's license and the seed data's rights decide whether you can use the set, so read them before quality results.
- Train on synthetic, test on real results on an isolated holdout are the most direct evidence of what the data teaches.
- A coverage table of real seed, holdout and synthetic counts per category shows thin grounding behind large synthetic volumes.
- Purely simulated data drops the seed and privacy rows but makes the real holdout more important, not less.
What should buyers request in a synthetic dataset disclosure sheet?
Request a disclosure sheet that answers three questions in writing before you buy: what real data and which generator produced these records, what was done to select and mix them, and how they performed against real cases the generator never saw. A sheet that only reports a fidelity score or a row count answers none of these.
In practice that means fields for grounding sources and their rights, the generator's identity, version and terms, the generation recipe, filtering and rejection steps, the ratio of synthetic to real records, the date of any rules or policies the records encode, results on an untouched real holdout, privacy screens, negative controls and known gaps.
The sheet is not a promise of quality from the vendor. It is a record that lets your team rerun the important checks, decide what the data is fit for and compare vendors on the same terms.
Why do generated datasets need a different disclosure than collected data?
Collected data is described by where and how it was recorded. Generated data is a product of three things at once: the real seed records, the generator and its settings, and the filters that decided which outputs to keep. A problem in any one of them passes silently into every record.
Regulators already treat synthetic generation as something to disclose. California's AB 2013 defines synthetic data generation as a process in which seed data are used to create artificial data that have some of the statistical characteristics of the seed data, and requires developers of covered generative AI systems to state whether the system used or continuously uses synthetic data generation in its development. A buyer who will face that question later needs the facts from the vendor now.
Documentation standards point the same way. The Croissant Responsible AI extension from MLCommons lists, among the information compliance teams need, the method used to generate synthetic data where applicable. Research on model collapse adds a technical reason: Shumailov and colleagues reported in Nature that indiscriminate training on model-generated content causes defects in which the tails of the original distribution disappear. Rare cases are exactly what a disclosure sheet must show are still present.
The synthetic dataset fact sheet: provenance and generation fields
Ask the vendor to complete every row, and to write "none" or "not recorded" rather than leaving a row blank. The right-hand column is what should make you push back.
| Field | What an adequate answer contains | Illustrative answer | Weak answer to push back on |
|---|---|---|---|
| Grounding sources | Each real dataset used as seed, prompt context or reference, with record counts and date ranges | Two years of de-identified accounts payable exceptions from three U.S. distributors | Proprietary data or industry knowledge |
| Seed rights | The license or agreement that permits use of the seed data for generation and resale | Seed license permits generation and sublicensing of derived records | We own the generated data, with no word on the seed |
| Generator identity | Model name, exact version, provider and hosting, or simulator and version | Open-weight model at a named version, run on the vendor's own servers | A leading LLM |
| Generator terms | The output and naming terms of the generator's license, quoted | License clause on models trained with its outputs quoted in full | Terms are not relevant because outputs are ours |
| Generation recipe | Prompt templates, parameters, conditioning variables and random seeds, or a summary with access on request | Template set v4, conditioning on exception type and vendor size | Proprietary process |
| Selection and filtering | Each filter, judge or human edit, and the share of outputs rejected at each step | Rule validator removes outputs that break invoice arithmetic; rejection share reported | Quality controlled |
| Mixture | Counts of synthetic, real and edited-real records in the delivered set | Synthetic and real counts listed per exception type | Mostly synthetic |
| Policy and rule date | The date of any rules, tax tables, codes or policies the records encode | Rules as of a named quarter; later changes not reflected | Up to date |
Evaluation fields: how was the data tested against untouched real tasks?
The most useful evaluation is on real records that the generator never saw in any form. Esteban, Hyland and Rätsch proposed the train on synthetic, test on real protocol for this reason: train a model on the generated data and measure it on real held-out data, so the score reflects what the synthetic records actually teach.
Statistical similarity reports are still worth requesting, provided you know what they measure. The open-source SDMetrics library, for example, compares synthetic data with the real data it mimics and produces quality reports on column shapes and column pair trends, along with privacy metrics. Its DCR overfitting protection metric uses distance to the closest record to check whether synthetic rows sit closer to the training data than to a holdout set.
| Field | What an adequate answer contains | Illustrative answer | Weak answer to push back on |
|---|---|---|---|
| Real holdout | Size, source and how it was kept out of seeding, prompting and filter tuning | Separate quarter of real exceptions, never shown to the generator | Validated on real data, with no isolation detail |
| Train synthetic, test real | Task, model, metric and the comparison with a real-trained baseline | Classifier trained on synthetic records scored on the real holdout and compared with a real-trained baseline | Synthetic data performs as well as real |
| Similarity reports | Which report, on which columns or features, with known limits | Column shape and pair trend reports attached for structured fields | A single overall score with no breakdown |
| Rule consistency | Share of records that pass domain rules and how rules were checked | Arithmetic and code validity checked on every record | Records look realistic |
| Privacy screens | Distance-to-closest-record or membership tests against seed data, with method | Distance to closest record compared with a holdout baseline | Synthetic data contains no personal data |
| Negative controls | Tests that should fail, such as a model trained on shuffled labels | Shuffled-label model scores near chance on the holdout | None run |
| Known gaps | Conditions, exceptions or segments under-represented or absent | Cross-border invoices and credit notes not generated | No known limitations |
Which real exceptions does the synthetic mixture fail to represent?
Ask the vendor for a coverage table that lists each exception type or condition that matters for your task, with its count in the real seed data, the real holdout and the delivered synthetic set. Gaps show up immediately: a category with real examples and no synthetic ones, or one that the generator multiplied far beyond its real frequency.
Over-represented categories are as risky as missing ones. A generator asked to balance classes can produce many fluent variations of a rare case from a handful of real seeds, and a model trained on them learns the generator's version of the case rather than the real one. The real holdout counts let you see when a category's synthetic examples rest on very few real records.
Then look for conditions that no seed covered. If the seed data came from one region, one season or one software system, the synthetic set inherits that boundary however many records it contains. The disclosure sheet should say so plainly in its known gaps row.
How do the generator's terms and the seed data's rights affect what you can do?
They decide whether you can use the data at all, so read them before quality. Generator licenses can attach conditions to outputs. Meta's Llama 3.1 license, for example, requires that anyone who uses Llama outputs to create, train or fine-tune an AI model that is distributed or made available include "Llama" at the beginning of that model's name. A dataset built with such a generator can carry that obligation into your model.
Seed rights are a separate question. Real records used to ground generation came from someone, under some agreement, and synthetic outputs do not automatically escape it. De-identification of the seed does not by itself create a right to license what was derived from it. Ask for the seed agreement's permitted uses in writing, and have counsel compare them with how you plan to train, distribute and deploy.
Worked example: reviewing a synthetic invoice-exception set
This example is illustrative; the vendor, counts and findings are invented. A team building a document model for accounts payable is offered a synthetic set of invoice exceptions, such as price mismatches, duplicate invoices and missing purchase orders, generated from a smaller real seed.
The disclosure sheet shows 40,000 synthetic records grounded in 2,500 real exceptions, an open-weight generator at a named version, and a rule validator that rejected about a fifth of outputs (illustrative figures). The real holdout of 800 exceptions came from a later quarter and was never shown to the generator, and a model trained on the synthetic set was compared with one trained on the real seed.
The coverage table exposes the decision point. Duplicate invoices have 30 real seed examples and 6,000 synthetic ones, and the synthetic-trained model does worse on real duplicates than on any other category. The team buys the set for the well-grounded categories, requests real duplicate-invoice records separately, and asks the vendor to cap synthetic examples per real seed in the next version.
The exception: a set produced purely by simulation, with no real seed records, makes the seed rights and privacy rows not applicable. It does not remove the holdout requirement; it makes it more important, because nothing else ties the data to real operations.
When is a disclosure sheet not enough to proceed?
- Stop when the vendor will not name the generator or quote its terms; you cannot assess obligations that may attach to your model.
- Stop when the seed data's rights cannot be shown in writing, regardless of how the outputs were transformed.
- Narrow the purchase to categories with adequate real grounding when the coverage table shows thin seeds behind large synthetic counts.
- Narrow the use to pretraining or augmentation, not evaluation, when no untouched real holdout exists; synthetic evaluation data can share the generator's blind spots.
- Request a privacy specialist when the seed contained personal or sensitive records and the privacy screen is a single summary statistic.
- Request a domain validator when records encode rules, codes or policies that change, and the rule date is old or missing.
When real records are the better purchase
Some gaps cannot be closed by generating more. When the disclosure sheet shows a category resting on a handful of real seeds, or when you need an untouched real holdout to evaluate a synthetic set at all, licensed real records are the direct fix.
When the missing piece is the real reference set itself, SourceX can be asked to find it: each dataset comes from U.S. businesses that match one buyer's request, not from stock, and the request form records the scope, volume, language, format, timing and licensing terms, including a holdout sized to validate a synthetic set.
What does synthetic dataset provenance diligence check, claim by claim?
It checks whether each provenance claim a synthetic data company makes rests on a document that exists today, and sorts every other claim into asserted or unknown. The disclosure sheet above tells a buyer what to ask about one dataset; an investor putting the same questions to a company's whole catalog needs to know which answers would survive a request for the paper behind them.
Open audits show this record-keeping is feasible. The Data Provenance Initiative's metadata for more than 1,800 fine-tuning datasets across 44 collections has a field for the model used to generate each dataset, and its entry for Stanford's Alpaca names OpenAI GPT-3 as the generator beside two license entries, one recorded as OpenAI and one as CC BY-NC 4.0. A company selling generated data should be able to show at least that layering for every set it sells.
Run the checklist below on datasets you pick from the catalog, not on the one the company offers to show:
| Claim the company makes | Document that supports it | What the document leaves open | Mark it |
|---|---|---|---|
| Our seed records were licensed for generation | The seed agreement, with the clause that names generation or derived datasets as a permitted use | Whether that clause also covers selling outputs and models trained on them | Documented with the clause in hand; asserted with a summary |
| The generator's terms allow this use | The terms text in force on the generation dates, archived with the model version | How later changes to the terms, or a court's reading, would apply | Documented if archived; unknown if only a current link exists |
| Outputs were filtered and checked | Pipeline configuration and run logs with rejection counts for each filter | Whether any filter was tuned on the evaluation holdout | Documented with logs; asserted with a slide |
| Quality holds on real data | A holdout manifest with record IDs, split date and the scoring script version | Whether the holdout resembles a particular buyer's population | Documented with the manifest; asserted with one score |
| No personal data survived generation | Privacy screen outputs for the delivered version | Linkage to data outside the screen's scope | Documented for the screen that ran; unknown beyond it |
| Every dataset is built this way | The same records for a random sample of the catalog | Datasets outside the sample | Documented only for what was sampled |
Why does a synthetic dataset still need a data card, and what would a good one say?
A synthetic dataset needs a card because the card is where its lineage, generation settings and limits become checkable by someone outside the team that built it. A card does not make the data good. It makes the builder's claims testable, and testable claims are the ones a buyer can price and an investor can underwrite.
Two widely used schemas already have the slots. Hugging Face's dataset card metadata lists machine-generated among the example values for its annotations_creators field and carries a source_datasets list, and its card template asks who produced the source data, meaning the people or systems that created it, and how records were selected, filtered and normalized.
The Data & Trust Alliance's Data Provenance Standards go further on lineage. Their generation-method codes include machine-generated synthetic and generative data, a sub-provenance element describes the other sources a dataset was assembled from, and the intended-use codes include AI synthetic data generation, so a seed set's own record can list feeding a generator among the purposes it was created for.
The card below is hypothetical, for a synthetic warranty-claim set, with every entry invented to show the level of detail to expect:
| Card field (schema) | Illustrative entry | What it lets a reader check |
|---|---|---|
| annotations_creators (Hugging Face); generation method (DPS) | machine-generated; Machine generated/MLOps Generative | That labels and text came from a model, so no human-label claim applies |
| source_datasets; sub-provenance | Two licensed seed sets, each with its own metadata record and issue date | Each seed's rights record, not just its name |
| Intended use on each seed record | AI synthetic data generation listed on both seed records | That seeding a generator was a stated purpose of the seed data |
| Who are the source data producers? | Warranty desk staff at two manufacturers for the seeds; an open-weight model at a pinned revision for the generated text | Which content is human-made and which is model output |
| Data collection and processing | Template set version, filters with rejection counts, deduplication threshold | Whether the pipeline can be rerun and audited |
| License | The dataset license plus the generator's terms, archived by date | Every license layer a buyer inherits, as in the Alpaca record |
| Bias, risks and limitations | Claim types missing from the seeds; real holdout score with its date | What the set cannot teach, in its builder's own words |
What can documentation prove to an investor, and where does its value stop?
Documentation proves that diligence is possible, not that the data is worth buying. A complete card on a set with weak holdout results describes a weak asset precisely, while a strong set with no card still carries rights risk that a buyer will discount or refuse.
Three things could make documentation worth less than this section argues: buyers who price on benchmark scores and never read cards; schemas that stay optional, so depth varies from card to card; and generator terms that change after generation, which an archived copy records but cannot undo. Ask any company whose datasets feed generators, SourceX included, for the provenance record of a dataset you choose, the share of its agreements that name synthetic generation as a permitted use, and one card that admits a limitation the company would rather not have.
For SourceX, the seed end of such a chain begins with the business supplying records. That business settles the scope and terms and signs first; only then does anything change hands, and the signed agreement governs what SourceX may use, sublicense and distribute from the de-identified dataset. Whether a given agreement permits synthetic generation is a term of that deal. No SourceX cards or deal figures are reported here; investors can ask the company for its materials through the investor page.
Methods note: the checklist and the illustrative card were built on October 27, 2026 from the Data Provenance Standards specification, Hugging Face's dataset card template and metadata reference, and the Data Provenance Initiative's repository, each read from its source repository that day. No company's records were reviewed, and the card describes no real dataset.
This article is for information only and is not an offer to sell or a solicitation of an offer to buy securities. Any offering by SourceX is made only to verified accredited investors through its offering documents.
Sources for this section
- The Data Provenance Initiative audited 44 collections spanning more than 1,800 fine-tuning datasets and records the model used to generate each dataset where applicable. Source
- The Data Provenance Collection's Alpaca entry lists OpenAI GPT-3 as the generating model and two license entries, OpenAI and CC BY-NC 4.0. Source
- Hugging Face dataset card metadata lists machine-generated as an example annotations_creators value and includes a source_datasets list. Source
- Hugging Face's dataset card template asks who produced the source data, people or systems, and how data was selected, filtered and normalized. Source
- The Data & Trust Alliance Data Provenance Standards define generation-method codes for machine-generated synthetic and generative data, a sub-provenance element and an intended-use code for AI synthetic data generation. Source
Frequently asked questions
What is a synthetic dataset disclosure sheet?
It is a document the vendor completes describing how a generated dataset was made and tested: seed data and rights, generator and terms, recipe, filtering, mixture, evaluation on real held-out data, privacy screens and known gaps. It lets buyers compare vendors and rerun key checks.
Is a high fidelity score enough to trust synthetic data?
No. Similarity reports show whether distributions resemble the seed, not whether a model trained on the data performs on real cases. Ask for train on synthetic, test on real results on a holdout the generator never saw.
Do synthetic datasets have to be disclosed under California law?
California's AB 2013 requires developers of covered generative AI systems to state whether synthetic data generation was used in development. Whether and how it applies to your system is a question for counsel, but buyers will need the facts from their vendors either way.
Can synthetic data carry obligations from the model that generated it?
Yes, depending on the generator's license. Some model licenses attach conditions to models trained with their outputs, such as naming requirements, so the generator's terms belong on the disclosure sheet.
Is synthetic data free of personal data?
Not automatically. Generated records can sit very close to seed records, so ask for privacy screens such as distance to closest record against a holdout baseline, and have a privacy specialist review results when the seed contained personal data.
Sources
- California AB 2013 defines synthetic data generation and requires developers to state whether a covered generative AI system used or continuously uses it in development. Source
- The Croissant Responsible AI specification lists the method used to generate synthetic data among information relevant to compliance. Source
- Shumailov and colleagues report that indiscriminate use of model-generated content in training causes defects in which the tails of the original distribution disappear. Source
- Esteban, Hyland and Rätsch proposed train on synthetic, test on real (TSTR) as an evaluation for generated data. Source
- SDMetrics evaluates synthetic data by comparing it with the real data it mimics, with quality reports on column shapes and column pair trends and privacy metrics. Source
- The SDMetrics DCR overfitting protection metric uses distance to closest record to check whether synthetic data was overfit to the real data, compared with a holdout set. Source
- The Llama 3.1 license requires models created or trained with Llama outputs that are distributed or made available to include Llama at the beginning of the model name. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.