Skip to content

AI data market

Next wave AI training data: five inputs, five purchase triggers

By SourceX Editorial · Reviewed by Noah Loul ·

9 sections

Short answer

The next wave of AI training data is less about bulk pretraining text and more about four later inputs: post-training examples, retrieval corpora, evaluation sets and deployment feedback. Each answers a different buyer decision and is bought again for a different reason: a new capability, a regression, stale facts, contamination or a wider rollout. Investors should test demand stage by stage, against cheaper substitutes such as synthetic data.

Value drivers

Next wave AI training data

This page covers next wave AI training data: five inputs, five purchase triggers.Illustrative

Key takeaways

  • Pretraining inputs are increasingly public, synthetic or first-party; the scarce evidence sits in post-training, retrieval, evaluation and deployment feedback.
  • Each of the five learning inputs has its own budget owner and its own repeat purchase trigger, so demand for one says little about another.
  • Contamination and new capability are the triggers a buyer is least able to satisfy from its own logs, which makes them the strongest signals of outside demand.
  • Reinforcement learning on checkable answers, distillation, better retrieval methods and first-party data each remove demand at one stage, not all five.
  • Measure demand as written requests by stage and dated cohort, followed to scored samples and paid commitments, not as AI capital spending.

Where is AI training data demand moving after pretraining?

Toward the stages after pretraining, where data has to carry evidence a web crawl cannot: expert decisions, current facts, graded outcomes and real user corrections. Pretraining still involves the largest volumes, counted in trillions of tokens where fine-tuning is counted in millions of examples, but its inputs are increasingly public, synthetic or produced inside the developer's own products.

Meta's model cards show the shift within one model family. Llama 3, released April 18, 2024, was pretrained on over 15 trillion tokens from publicly available sources, and its fine-tuning data included over 10 million human-annotated examples. The Llama 3.1 card, dated July 23, 2024, lists about 15 trillion pretraining tokens and over 25 million synthetically generated fine-tuning examples. The Llama 4 card, dated April 5, 2025, says its pretraining drew on publicly available data, licensed data and information from Meta's products and services, including people's interactions with Meta AI.

Read together, the cards describe a market where bulk text is established and the differentiated inputs sit later in the pipeline: annotated decisions, licensed material and feedback from deployment. The investor question is no longer whether AI needs data, but which of five learning inputs a buyer will pay for again, and why.

The AI Learning Inputs Map: task, evidence, buyer and trigger by stage

Each learning input answers a different buyer decision, so demand for one stage says little about demand for another. The map separates them by the task the data teaches or tests, the evidence it must carry, the team that usually holds the budget and the event that brings a buyer back.

Post-training evidence takes concrete forms. Anthropic's public helpfulness and harmlessness data, for example, is a file of chosen and rejected response pairs, part of it gathered during an iterated online process with live models. That is a different product from a pile of documents, with a different buyer, and the map treats it that way.

The budget owners are typical patterns drawn from how model developers describe their pipelines, not observed SourceX buyers; the methods note at the end says what was not measured.

The AI Learning Inputs Map: task, evidence, buyer and trigger by stage
StageWhat the data teaches or testsEvidence it must carryTypical budget ownerRepeat purchase trigger
PretrainingGeneral language, code and world knowledgeScale, breadth and lawful sourcingFoundation model training teamA new base model run
Post-trainingHow to perform a task: instructions, preferences, rewarded outcomesExpert demonstrations, ranked responses or checkable resultsFine-tuning, alignment or applied AI product teamA new capability or a failure to fix
RetrievalNothing permanent: it supplies facts at answer timeCurrent, versioned documents with dates and contextProduct or knowledge management ownerStale facts or a new document set
EvaluationWhether the model can do the job, and whether it regressedCases with known correct outcomes, graders and severityEvaluation, safety or procurement ownerContamination, regression or a new model to compare
Deployment feedbackWhat went wrong in real useLogged corrections, overrides and outcomes, with consent and contextProduct owner of the deployed systemA wider rollout to new users, tasks or markets
A five-stage flow from pretraining to post-training, retrieval, evaluation and deployment feedback, each card naming its repeat purchase trigger, with a loop from deployment feedback back to post-training and evaluation.
The AI Learning Inputs Map: five learning inputs and the event that brings a buyer back to each.

One claims workflow, five kinds of evidence (Illustrative)

Tracing one business workflow through the map shows why the stages are separate purchases. Illustrative example: an AI developer builds an assistant for property insurance adjusters, from first notice of loss to payment. The example describes no real company or deal.

Five inputs, five owners, five reasons to buy. A supplier strong in one can be irrelevant to another: years of closed claims are valuable for evaluation and post-training but say nothing about this month's endorsements, and a well-kept policy library helps retrieval but teaches no judgment.

  • Pretraining: general language and some insurance vocabulary, already available from public text; nothing here is specific to the insurer.
  • Post-training: adjuster decisions with their reasons, such as why a water claim was covered under one policy form and denied under another, plus ranked versions of draft letters.
  • Retrieval: policy wordings, endorsements and state rules by version and effective date, so the assistant cites the form in force on the date of loss.
  • Evaluation: closed claims with known correct outcomes, graded by severity, kept out of training and refreshed when policy forms change.
  • Deployment feedback: adjusters' overrides of the assistant's suggestions, logged with the corrected outcome, under consent and contract terms that allow that use.

Which triggers bring buyers back? Comparing five repeat-purchase events

Repeat purchase, not first purchase, is what makes a data market investable, and the five triggers differ in how much they depend on data from outside the buyer. The right-hand column of the table below matters most: a trigger the buyer can satisfy from its own logs creates little outside demand, however often it fires.

OpenAI's own Evals framework makes the self-supply point plainly: it lets teams build private evals from their own data without exposing it, and its documentation calls creating high quality evals one of the most impactful things a team building with language models can do. Regression testing and deployment expansion therefore lean on the buyer's own records. Contamination and new capability are different: once a test set has leaked, or a task is new, fresh unseen cases have to come from somewhere else.

Which triggers bring buyers back? Comparing five repeat-purchase events
TriggerWhat changesStage it hitsPublic evidence, datedCan the buyer self-supply?
New capabilityA model is asked to do work it was not trained or tested onPost-training, evaluationOpenAI GDPval (2025): 1,320 tasks across 44 occupations, written by professionals averaging about 14 years of experiencePartly: experts and synthetic data can seed it
RegressionA model update breaks behavior that used to workEvaluationAnthropic's agent evaluation guidance: run multiple trials per task because outputs varyMostly, from its own failure logs
FreshnessFacts, documents or rules change after the training cutoffRetrievalMeta model cards: pretraining cutoff December 2023 for Llama 3.1, August 2024 for Llama 4Often, from its own documents
ContaminationTest cases leak into training data and scores stop meaning muchEvaluationOpenAI stopped reporting SWE-bench Verified after models reproduced original fixesHardly: unseen cases must come from outside the training pipeline
Deployment expansionA system moves to new users, tasks or marketsDeployment feedback, evaluationOpenAI Evals: private evals built from a team's own dataYes, once deployed

When better models or public data remove the need for a paid dataset

Often, and an honest thesis names those cases first. Four public examples show demand at one stage shrinking because a method, a model or the buyer's own data did the job.

Each counterexample hits one stage, not all five. That is the point of mapping them: a thesis built on evaluation data is not refuted by synthetic instruction data, while a thesis built on human demonstrations should worry about it a great deal.

  • Reinforcement learning on checkable answers. DeepSeek's 2025 R1 release reports that DeepSeek-R1-Zero developed its reasoning through large-scale reinforcement learning without supervised fine-tuning as a first step, though the full R1 pipeline still added cold-start data. Where answers can be checked automatically, demand for human demonstrations falls.
  • Distillation and synthetic data. The same release says reasoning patterns from the large model were distilled into smaller open models fine-tuned on samples it generated, and the Llama 3.1 card lists over 25 million synthetic fine-tuning examples. Model outputs now stand in for part of what annotators once supplied.
  • Better retrieval methods. Anthropic's cookbook reports that contextual embeddings, which add relevant context to each chunk before indexing, reduced the top-20-chunk retrieval failure rate by 35 percent on average across the data sources it tested. A method change, not a new corpus, closed much of the gap.
  • First-party data. Llama 4's pretraining included information from Meta's own products and services, including interactions with Meta AI. A buyer that runs a large product can supply much of its own feedback and many of its own evaluation cases.

How to measure demand by stage without counting AI spending

Count written buyer requests by stage and dated cohort, then follow each one to a scored sample and a paid commitment; broad AI capital spending says nothing about which input gets bought. The unit is one written request from one budget owner for one task.

Steps

How to measure demand by stage without counting AI spending

Interview budget owners separately from the request log: ask which trigger caused the request and what they would have used instead. Report every rate with its denominator, its cohort dates and the share still open at the measurement date. SourceX has not published results for this design; it shares company and commercial metrics with investors directly.

Checklist · 5 items

  1. Request: a dated written request naming the task, stage, volume and acceptance test. Denominator: all requests received in the cohort month.
  2. Scored sample: a sample delivered and scored against the buyer's test. Denominator: requests in the same cohort.
  3. Paid commitment: a signed order or license. Denominator: requests in the cohort, reported beside the share still open, which is censored rather than counted as lost.
  4. Repeat: a second paid commitment from the same budget owner, tagged with its trigger. Denominator: budget owners whose first commitment is at least 12 months old.
  5. Independence: the number of distinct budget owners, not companies, since one developer can hold several budgets for different stages.

0 of 5 ticked. Tick each item as you go.

What could make the next-wave thesis wrong

Five developments could shrink outside demand for later-stage inputs. Name them before the upside; each comes with evidence an investor can track.

  • Synthetic and self-generated data keep improving. The Llama 3.1 card's 25 million synthetic fine-tuning examples and DeepSeek's distilled models show model outputs already replacing part of human-made post-training data.
  • Checkable rewards spread to more tasks. Reinforcement learning without supervised fine-tuning worked for DeepSeek-R1-Zero where answers can be verified; every new domain with automatic checks needs fewer demonstrations.
  • Platforms supply themselves. Llama 4's pretraining drew on Meta's own products, including interactions with Meta AI; developers with large deployments may never need to buy feedback data.
  • Evaluation sets perish. SWE-bench Verified, introduced on August 13, 2024 as a 500-problem subset, lost its role once OpenAI found frontier models could reproduce original fixes for some tasks. That creates refresh demand, and it also makes any single evaluation set a wasting asset.
  • Announcements outrun budgets. A model release, a benchmark or a partnership page is not a purchase order, and capability progress may not translate into commercial data spending, which is why the measurement design counts requests and commitments rather than headlines.

Questions to ask any company selling learning inputs, SourceX included

Ask which stage the company actually serves and what evidence shows repeat purchase there, using definitions you set rather than ones it chooses.

Put SourceX through the same questions. Its role is to help companies convert records they select into datasets licensed for AI: it qualifies and sources supply, reviews rights, handles contracting, prepares the data, delivers it and manages payment. For buyers, it finds datasets among U.S. businesses on request instead of keeping an inventory. Under Rule 506(c), SourceX is raising from accredited investors, and its investors page takes requests for materials along with any of the questions below.

Checklist · 6 items

  1. Which of the five stages do your requests come from, and who assigns a request to a stage?
  2. What share of paid commitments came from repeat budget owners, and which trigger caused each repeat?
  3. How many distinct budget owners issued requests in each cohort, and how many reached a scored sample?
  4. Which requests could the buyer have met with synthetic data, its own logs or a better retrieval method, and how do you know?
  5. For evaluation data, how do you keep delivered cases out of training sets, and what happens to a set once it leaks?
  6. What would make your largest stage unnecessary within two model generations?

0 of 6 ticked. Tick each item as you go.

Methods note: evidence behind the learning inputs map and purchase triggers

The AI Learning Inputs Map, the trigger comparison and the demand measurement design are this article's own editorial frameworks, assembled on 2026-10-26. They draw on Meta's Llama 3, 3.1 and 4 model cards, DeepSeek's R1 release, OpenAI's GDPval and SWE-bench Verified pages and its Evals framework, the SWE-bench project, and Anthropic's evaluation guidance, cookbook and published preference data, all checked on 2026-10-26. The claims workflow is illustrative and describes no real company or deal.

Not measured: the size of any stage's market, request or commitment counts at SourceX or any other company, prices, and how often each trigger fires in practice. Budget owners in the map are typical patterns, not observed buyers, and the self-supply column is a judgment from the cited evidence rather than a survey.

Frequently asked questions

What is post-training data?

Post-training data teaches a pretrained model to perform tasks well: instruction examples, ranked responses for preference training and tasks with checkable outcomes for reinforcement learning. Meta's Llama 3 card, for example, lists over 10 million human-annotated fine-tuning examples. It is far smaller than pretraining data and much more specific to the task.

Is evaluation data a separate market from training data?

It has a different buyer and a different trigger. Evaluation data must stay out of training, carry known correct outcomes and be refreshed when it leaks or the task changes, while training data is consumed once learned. No public figure sizes either market, so treat any sizing claim as something to verify.

Will synthetic data replace licensed business data?

At some stages it already replaces part of it: the Llama 3.1 card lists over 25 million synthetic fine-tuning examples. It is weakest where the data must record what actually happened, such as real outcomes, current documents or user corrections. Test each stage separately rather than the market as a whole.

What is deployment feedback data?

Records of how a deployed AI system performed in real use: user corrections, overrides, ratings and the outcomes that followed. It is valuable for finding failures and building evaluation cases, but its use depends on consent, contracts and privacy terms, and large platforms often collect their own.

How should an investor test claims about AI data demand?

Ask for written requests by stage and dated cohort, with denominators, the share still open, and conversion to scored samples and paid commitments. Ask which trigger caused each repeat purchase and what substitute the buyer considered. Treat model releases and partnership announcements as context, not demand.

Sources

  • Meta's Llama 3 model card (released April 18, 2024) says the model was pretrained on over 15 trillion tokens from publicly available sources and fine-tuned with over 10 million human-annotated examples. Source
  • Meta's Llama 3.1 model card (July 23, 2024) lists about 15 trillion pretraining tokens, over 25 million synthetically generated fine-tuning examples and a December 2023 pretraining cutoff. Source
  • Meta's Llama 4 model card (April 5, 2025) says pretraining used publicly available data, licensed data and information from Meta's products and services, including interactions with Meta AI, with an August 2024 cutoff. Source
  • Anthropic's published helpfulness and harmlessness preference data consists of chosen and rejected response pairs, part of it sampled during an iterated online process. Source
  • OpenAI's GDPval spans 1,320 tasks across 44 occupations, written by professionals averaging about 14 years of experience, with future versions planned to be more interactive. Source
  • Anthropic's agent evaluation guidance defines tasks, trials and graders and recommends multiple trials per task because model outputs vary between runs. Source
  • OpenAI stopped reporting SWE-bench Verified after finding that frontier models could reproduce original human-written fixes or problem details for some tasks. Source
  • SWE-bench Verified was introduced on August 13, 2024 as a subset of 500 problems that software engineers confirmed are solvable. Source
  • OpenAI's Evals framework supports private evals built from a team's own data and says creating high quality evals is one of the most impactful things teams building with LLMs can do. Source
  • DeepSeek reports that R1-Zero was trained with large-scale reinforcement learning without supervised fine-tuning as a first step, that R1 added cold-start data, and that R1's reasoning was distilled into smaller open models. Source
  • Anthropic's cookbook reports that contextual embeddings reduced the top-20-chunk retrieval failure rate by 35 percent on average across the data sources tested. Source

Request SourceX investor materials

SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.

See if your company qualifies