Deal economics
Synthetic versus licensed data economics: the verifier decides
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
Synthetic data replaces licensed data when three tests pass: a cheap verifier can check every generated example, the correct answer does not depend on real outcomes that no record states, and the generator's terms and source rights allow the use. Constrained-format tasks often pass all three; domain judgment and rare exceptions usually fail the second; held-out evaluation must stay real by design. Price licensed data where the tests fail.
Value drivers
Synthetic
Key takeaways
- Synthetic data substitutes for licensed data when a cheap verifier exists, the answer does not depend on real outcomes, and the generator's terms allow the use.
- Constrained-format tasks usually pass all three tests; domain judgment and rare exceptions need real outcomes; held-out evaluation stays real by design.
- Compare inputs at a matched human review budget and score constraint fidelity, coverage and downstream utility separately.
- Generation is cheap; validation and permissions decide the economics, and generator terms change between model versions.
- Licensed records keep pricing power as outcome labels, held-out evaluation, generator seeds and evidence of rare exceptions.
Does synthetic data replace licensed data?
In some tasks it already does, and an investor who assumes every real record carries a premium will overpay for the wrong supply. Synthetic data replaces licensed data where a cheap check can confirm each generated example, where the right answer does not depend on what actually happened inside a business, and where the generator's terms allow the use. Where any of the three fails, synthetic data complements licensed records instead of replacing them.
The substitution is not hypothetical. NVIDIA's Nemotron training FAQ says its models use synthetic data in both pre-training and post-training, and that in Nemotron 3 Ultra quality-filtered and synthetic web crawl together make up about 49 percent of the first pre-training phase's tokens and 38 percent of the second. Google DeepMind's AlphaGeometry, published in Nature in 2024 under the title Solving Olympiad Geometry without Human Demonstrations, trains its language model on synthetic problems.
Neither fact makes licensed business data worthless. Both move the burden of proof: a licensed-data thesis has to name the tasks where generation cannot do the job and explain why. The rest of this page builds that test and applies it to synthetic versus licensed data economics.
Three tests that decide substitution: verifier, real outcome, rights
Run three tests for each task, in order: can every generated example be checked cheaply, does the correct answer depend on real outcomes that no record states, and do the generator's terms and the source records' rights permit the use. Synthetic data substitutes only when the first passes, the second does not bind and the third is clear.
The verifier test explains the strongest synthetic results. AlphaGeometry's repository pairs the language model with DDAR, a symbolic theorem prover, and in geometry a deduction engine can check each step of a proof mechanically. NVIDIA's FAQ describes the same habit at scale: synthetic tool-calling data is screened by turn-level and trajectory-level model judges paired with rule-based verification, and new synthetic sets are validated by training small models on them to check that they learn something. Where no such check exists, every generated example needs human review and the cost advantage narrows.
The real-outcome test asks whether the label lives outside the text. Whether a disputed invoice was paid, a warranty claim upheld or a customer renewed is a fact about the world after the record was written. A generator can imitate the record; it cannot observe the outcome, so a model trained only on imitations learns plausible decisions rather than correct ones.
The rights test is the one investors skip. Stanford's Alpaca project generated 52,000 instruction-following examples for less than $500, then released them under a non-commercial license and said models trained on them should not be used outside research. Meta's Llama 2 license barred using the model's outputs to improve any other large language model; the Llama 3.1 license, dated July 23, 2024, permits it on condition that a distributed model's name begins with Llama. Cheap synthetic data can carry expensive terms, and those terms change between versions.
The synthetic substitution matrix: four tasks compared
Applied to concrete tasks, the three tests sort substitution into three verdicts: substitute, complement or keep real. The Synthetic Substitution Matrix below is this page's original asset. It is an editorial framework built from the sources cited here, not a measured result, and the task examples are illustrative.
Two patterns stand out. Substitution is strongest where the truth sits inside the artifact, as with a schema, a proof or a passing test. It is weakest where the truth arrives later and outside the record, which describes most of the operating decisions that business records capture. The partly cells are where the uncertainty lives: a verdict there can flip with a better judge or a narrower task definition, so read complement as a range rather than a point.
The held-out row is not a verdict on synthetic quality. Anthropic's evaluation guidance notes that production monitoring catches issues synthetic evals miss, and that converting user-reported failures into test cases keeps a suite tied to actual usage. An evaluation built from generated tasks measures agreement with the generator; only untouched real work measures transfer.
| Task (illustrative) | Cheap verifier? | Depends on real outcomes? | Rights constraint to check | Verdict |
|---|---|---|---|---|
| Constrained format: turn an invoice into a fixed schema | Yes: schema and arithmetic checks | No: the answer is in the document | Generator license terms for commercial use | Substitute, with a small real sample to confirm coverage |
| Domain judgment: approve or deny a warranty claim under policy | Partly: rule checks, not judgment | Yes: whether the decision held | Rights in any records used as generator seeds | Complement: real decisions and outcomes anchor, synthetic widens coverage |
| Rare exception: a claim that breaks two rules at once | Partly: invariants can be checked | Yes: which exceptions actually occur | Derivative-use terms in the source license | Complement: real cases show which exceptions exist |
| Held-out evaluation: score a model on real work | Beside the point: the test measures transfer to reality | Yes, by design | Evaluation-only license scope | Keep real: a generated test grades the generator |
Comparing public, synthetic, licensed and hybrid inputs at one review budget
Compare the four input types at a matched human review budget, not a matched row count, and score them on separate measures. A generator can produce far more rows than a team can review, so a row-count match flatters synthetic data, while a budget match shows what each review hour buys.
At a glance
Comparing public, synthetic, licensed and hybrid inputs at one review budget
Score three things separately on an untouched real test set. Constraint fidelity asks whether outputs obey the task's hard rules. Coverage asks whether the inputs include the cases and exceptions that occur in real work. Downstream utility asks whether a model trained on the inputs performs better on held-out real tasks. A synthetic set can score perfectly on fidelity and poorly on coverage, which is exactly the pattern a single blended metric hides.
Agreement between reviewers belongs in the budget. Anthropic describes a good evaluation task as one where two domain experts would independently reach the same verdict; tasks that fail that test cost more to review and produce noisier comparisons. For data derived from sensitive records, NIST's SDNist package shows the reporting shape to copy: it scores a de-identified dataset against its original counterpart and reports utility and privacy together. The full protocol for real-synthetic mixture experiments is on a separate SourceX page linked below; this page uses only its logic.
- Public: openly available data for the task, with its license recorded.
- Synthetic: generated data, with the generator, prompts, seeds and terms recorded.
- Licensed: real business records approved for the use, prepared to the same standard.
- Hybrid: licensed records as seeds or anchors plus synthetic expansion, with every row tagged by origin.
Pricing the comparison: validation and permissions are the real costs
Once generation is cheap, the commercial comparison turns on validation and permissions, not production. Alpaca's 52,000 examples cost less than $500 to generate; the license that came with them limited what anyone could build on them. An investment model that prices only generation misses the two lines that decide margin.
Read the table as a ratio, not a ranking. A synthetic-heavy business wins where the validation line is automatic and the permissions line is clean; a licensed-data business wins where validation needs real outcomes that no generator can supply.
| Cost line | Synthetic input | Licensed input | What to ask |
|---|---|---|---|
| Production | Generation compute and prompt design | Sourcing, rights review, preparation and delivery | What does one accepted example cost, not one generated row? |
| Validation | Automatic where a verifier exists; human review where it does not | Quality review against the buyer's acceptance test | How many review hours per accepted example? |
| Permissions | Generator terms, seed-record rights, naming or use limits | The license scope signed by the record owner | Which terms restrict commercial or competitive use? |
| Refresh | Regenerate when the generator or the policy changes | Relicense or extend when the business changes | How often does the task drift, and who pays to follow it? |
| Failure exposure | Plausible but wrong examples that pass weak checks | Defects, gaps and residual privacy risk | What happens when a defect reaches a customer's model? |
Where licensed data keeps its pricing power
Licensed records keep pricing power in four roles the substitution tests cannot remove: outcome labels, held-out evaluation, seeds for generation and evidence of rare exceptions. A company whose licensed-data revenue sits in format tasks should expect pressure; one whose supply sits in these four roles has a structural argument.
At a glance
Where licensed data keeps its pricing power
OpenAI's GDPval benchmark shows the evaluation role. Its 1,320 tasks across 44 occupations were each based on a real work product and vetted by professionals averaging more than 14 years of experience, and OpenAI says the benchmark is limited to one-shot evaluations that do not capture building context or improving through drafts. Real work defined the tasks; generating them would have defeated the purpose.
The testable form of the thesis follows. A licensed-data supplier's defensibility rises with the share of its supply that carries real outcomes or serves as untouched evaluation, and falls with the share a verifier can check. An investor can estimate that share in diligence by sampling a supplier's datasets and placing each on the matrix, and the company should be able to do the same for its own pipeline and show its work.
- Outcome labels: what happened after the decision, which only the business recorded.
- Held-out evaluation: untouched real work that tests transfer, kept away from every generator.
- Seeds and anchors: real cases that ground a generator in the actual case mix.
- Rare exceptions: proof of which unusual combinations occur and how they were resolved.
What could make the substitution matrix wrong
Four developments would move tasks from complement to substitute, and an investor should track each one. Naming them is the test of a licensed-data thesis: a company that cannot say which one would hurt it most has not stress-tested its own supply.
Licensed data carries risks of its own: rights that do not cover the intended use, de-identification that removes the signal, and preparation costs that rise with record complexity. A fair comparison applies the same scrutiny to both sides of the matrix.
- Verifiers improve. NVIDIA already screens synthetic tool-calling data with model judges paired with rule checks; if model judges become reliable on judgment tasks, the verifier test passes for more rows.
- Outcomes get simulated. Agent environments that model consequences could produce outcome-like labels without real records, weakening the second test for some workflows.
- Generator terms loosen. Meta moved from barring the use of Llama 2 outputs to improve other models to permitting it for Llama 3.1 under a naming condition; looser terms remove the rights test as a barrier.
- Buyers accept synthetic evaluation. If procurement teams treat generated test sets as good enough, the keep-real row shrinks, whatever its statistical merit.
Questions to ask a data company about synthetic substitution, SourceX included
Ask where on the matrix a company's supply sits, and ask for evidence measured on untouched real work. Ask SourceX each of these too.
SourceX is one application of this question. Its work is converting business records that companies choose to license into AI datasets, which spans supplier sourcing and vetting, rights review, contract work, data preparation, and the delivery and payment at the end; it keeps no shelf of ready-made datasets, sourcing each one when a buyer asks. It has not published substitution studies or commercial metrics, and investors receive company and commercial figures from it directly. Accredited investors are the audience for its Rule 506(c) raise; the investor overview is available on request, and a pointed diligence question, such as which matrix rows its sourcing targets, is the best thing to bring.
Checklist · 6 items
- What share of your demand comes from format tasks that a generator could take over?
- For judgment tasks, what real outcome does each record carry, and how is it verified?
- Which evaluation sets are kept away from every generator, and who controls access?
- Which terms govern any generator used on your data, and on data derived from it?
- At a matched review budget, how did licensed, synthetic and hybrid inputs score on real held-out work?
- Which verifier improvement would remove the need to license a category you supply?
0 of 6 ticked. Tick each item as you go.
Methods note: how the matrix was built and what it leaves out
The three tests, the Synthetic Substitution Matrix and the cost table are editorial frameworks written for this article on 2026-10-26. Evidence comes from NVIDIA's Nemotron training FAQ, Google DeepMind's AlphaGeometry repository, Stanford's Alpaca repository, Meta's Llama 2 and Llama 3.1 licenses, NIST's SDNist package, Anthropic's evaluation guidance and OpenAI's GDPval announcement, each checked on 2026-10-26. Task examples are illustrative.
Not measured: the cost of any synthetic or licensed dataset, head-to-head performance of the four input types on any task, the share of AI training spend in each matrix row and the pace at which verifiers improve. Each would need a preregistered study with dated, approved evidence, untouched real evaluation and published negative results; SourceX has published none.
Frequently asked questions
Is synthetic data better than real data?
Neither is better in general. Synthetic data is cheaper and easier to share, and it works well where every example can be checked automatically, as with schemas, proofs or code tests. Real data is better where the correct answer depends on what actually happened, and it is required for testing whether a model works on real work.
What are the disadvantages of synthetic data?
It reflects only what the generator and its seeds already contain, so rare real exceptions and true outcomes are missing unless real records supply them. It can carry restrictive terms: Stanford's Alpaca data was released for non-commercial use only. An evaluation built from generated tasks grades agreement with the generator, not performance on real work.
Can another company's model be used to generate training data commercially?
It depends on that model's license. Llama 2's license barred using its outputs to improve other large language models, while Llama 3.1's permits it on condition that a distributed model's name begins with Llama. Read the current terms of the exact model version, and have counsel review any commercial use.
Will synthetic data make licensed business data worthless?
Not across the board. It is already replacing real data in tasks with cheap verifiers, which pressures suppliers whose value sits there. Records that carry real outcomes, rare exceptions or untouched evaluation work keep a role no generator fills today, unless verifiers or simulators improve enough to change the matrix.
Does SourceX publish synthetic-versus-licensed results?
No. Substitution studies and commercial metrics are not public; SourceX provides company and commercial metrics to investors directly, starting from a request at sourcex.si/investors. Only verified accredited investors are eligible for any offering.
Sources
- NVIDIA's Nemotron training FAQ says synthetic data is used in both pre-training and post-training, that quality-filtered and synthetic web crawl make up about 49% of Phase 1 and 38% of Phase 2 tokens in Ultra, that synthetic sets are validated through small-scale training ablations, and that synthetic tool-calling data is screened by turn- and trajectory-level LLM judges paired with rule-based verification. Source
- Google DeepMind's AlphaGeometry repository reproduces the theorem provers from the Nature 2024 paper Solving Olympiad Geometry without Human Demonstrations, whose language model is trained on synthetic problems. Source
- Stanford's Alpaca repository says its instruction-following dataset of 52K examples was obtained for less than $500, is licensed CC BY NC 4.0 for non-commercial use only, and that models trained on it should not be used outside research. Source
- The Llama 2 Community License says licensees will not use Llama Materials or any output to improve any other large language model, excluding Llama 2 and its derivatives. Source
- The Llama 3.1 Community License (release date July 23, 2024) allows using outputs to create, train or improve an AI model that is distributed, provided Llama appears at the beginning of the model's name. Source
- NIST's SDNist provides benchmark data and evaluation metrics for de-identified data generators, evaluating a de-identified dataset against its original counterpart and reporting utility and privacy. Source
- Anthropic's guidance on agent evaluations says a good task is one where two domain experts would independently reach the same verdict, that production monitoring catches issues synthetic evals miss, and that converting user-reported failures into test cases keeps a suite tied to actual usage. Source
- OpenAI's GDPval spans 1,320 tasks across 44 occupations, each based on a real work product and vetted by professionals averaging more than 14 years of experience, and is limited to one-shot evaluations. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.