AI data market
AI data freshness versus novelty: a five-rung test of buyer spending
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Buyers pay for freshness and novelty for different jobs. Freshness, meaning records created recently, is bought as ongoing access for products that must answer about the present, as OpenAI's 2024 Reddit deal shows. Novelty, meaning information absent from existing corpora, is bought to teach or test what models have not seen; GSM1k's unseen problems exposed accuracy drops of up to 8%. Check which property the contract prices before valuing either.
Value drivers
AI data freshness
Key takeaways
- Freshness is about when a record was created; novelty is about whether its information is already in a model's training data, so a ten-year-old claims file can be novel and stale at once.
- Public deal language splits the two: OpenAI's Reddit partnership stresses real-time content on recent topics, while its data partnerships page asks for data not easily accessible online.
- In evaluation, freshness is mostly a route to novelty: LiveBench uses recent sources to limit contamination, and GSM1k's unseen problems exposed drops of up to 8%.
- Freshness revenue can recur because operations keep producing records; novelty is used up for a model once that model trains on it.
- Announcements are the weakest rung of evidence; only written specifications, prices and renewals show which property a buyer pays for, and almost none of that is public.
Freshness or novelty: which one are AI data buyers paying for?
Both, for different jobs, and the contract usually shows which. Freshness means a record was created recently. Novelty means the information it carries is absent from the data a model has already learned from. A buyer pays for freshness when its product must answer questions about the present, and for novelty when a model must learn, or be tested on, something it has not seen.
The two properties are independent. A support ticket from last week is fresh, yet if it repeats an answer already published in a help center it teaches a model almost nothing. A decade-old claims file was never posted anywhere, so it can be new to every model while being stale by any calendar. Most private operating records sit in that second group: too old to count as recent, yet unseen because nobody ever published them.
For an investor the question is commercial, not semantic. Revenue built on freshness can recur, because a running business keeps producing records, but it competes with live feeds and search. Revenue built on novelty is harder to substitute, but a given model can learn a given fact for the first time only once. The sections below separate the two with a five-rung demand ladder, a six-test evidence matrix, the strongest counter-argument, the risks and a measurement design.
AI data freshness versus novelty: two properties that decay for different reasons
Freshness decays with the calendar; novelty decays when material is absorbed by a model or published. That difference sets how long a dataset keeps its value and what a buyer would pay to replace it.
The calendar effect is measurable. In Mind the Gap, a 2021 study by Angeliki Lazaridou and colleagues, Transformer-XL language models did worse when predicting text written after their training period, and the shortfall grew as the test period moved further away. A larger model alone did not close the gap, while models that kept learning from new information reduced it. That is the technical case for paying for recency in domains that change quickly.
Novelty wears off another way. Once a dataset enters a model's training mix, its contents stop being new to that model, and once material leaks online, it stops being new to every model later trained on the web. Evaluation shows the cost. Hugh Zhang and colleagues wrote GSM1k, a set of grade-school math problems built to mirror the public GSM8k benchmark, and in its current version the paper finds some models losing up to 8% accuracy on the fresh set, several model families showing signs of systematic overfitting and frontier models showing little. The authors kept GSM1k private so it would not leak the same way.
The two properties meet in evaluation. A question written after a model's training cutoff is unlikely to sit in its training data, so benchmark builders use dates as a stand-in for unseen material. LiveBench, released in 2024, draws questions from recent math competitions, arXiv papers, news articles and datasets, scores them against objective answers and adds new questions monthly to limit contamination. In that design, freshness is the means and novelty is the goal.
What does public deal language say each buyer wants?
Published deal language separates the two properties more clearly than most commentary does, but it sits on the lowest rung of evidence: it shows what a buyer wants, not what it paid for each property.
Freshness shows up as access. When OpenAI announced its Reddit partnership in May 2024, it said it would use Reddit's Data API, which delivers real-time, structured and unique content, so that its tools could better understand and showcase Reddit content, especially on recent topics. That is a feed bought to keep a product current, not an archive bought once.
Novelty shows up as absence. OpenAI's data partnerships page, published in November 2023, goes after large-scale datasets not yet within easy public reach online, favors data that expresses human intention, and excludes sensitive personal information and information that belongs to a third party. GDPval, the evaluation OpenAI published in 2025, pushes the idea further: 1,320 tasks across 44 occupations in the nine industries that contribute most to U.S. GDP, written and vetted by professionals with an average of 14 years of experience. Expert work products of that kind exist in no public corpus until someone commissions them.
Saturation keeps novelty demand alive. Anthropic's January 2026 guide to agent evaluations says capability evals should start at a low pass rate, and that suites with high pass rates graduate into regression tests. Its system card for Claude Opus 4.6, published in February 2026, reports that the model saturated all of the company's current cyber evaluations, that those benchmarks could no longer track capability progress, and that harder evaluations had become a priority. Each saturated suite is, in effect, a request for material no model has seen.
The freshness-novelty ladder: what each rung of buying evidence reveals
Rank every demand claim by how close it sits to money, and record at each rung whether the buyer's own terms point to freshness or to novelty. The rung matters more than the count: one renewal keyed to new records says more than ten partnership announcements.
Read each rung for its tell, the term or test that names the property being paid for. Public sources reach the first rung and touch the fourth and fifth only in aggregate; everything else sits in buyers' specifications and contracts:
| Rung | Freshness tell | Novelty tell | Public evidence, dated | What it cannot show |
|---|---|---|---|---|
| 1. Public statement | Real-time access, recent topics, ongoing feeds | Not easily accessible online, held out, never published | OpenAI and Reddit, May 2024; OpenAI data partnerships page, November 2023 | Whether anyone paid for that property |
| 2. Written specification | A creation-date window or a delivery cadence | An overlap or contamination test against named corpora | None public; specifications stay with buyers | Whether the records pass the test |
| 3. Sample evaluation | Better scores on questions written after the model's cutoff | Better scores, or exposed overfitting, on unseen lookalike items | Published methods only: LiveBench and GSM1k, both 2024 | Whether a budget follows the result |
| 4. Paid commitment | Price tied to a term, a feed or a refresh schedule | Price tied to exclusivity, confidentiality or a held-out set | Reddit's annual report for fiscal 2025: medium-term content licenses, value concentrated in two partners | Which property set the price |
| 5. Repeat purchase | Refresh orders for new periods | New held-out sets ordered as old ones saturate | Anthropic's Claude Opus 4.6 system card, February 2026: saturated cyber evaluations, harder ones prioritized | Whether repeat orders keep their price |
Six tests that tell recent records from unseen ones
The procurement evidence matrix below uses six tests to separate the two properties, and none of them works alone. A dataset can pass the freshness test and fail the novelty test, which is exactly the case in which a buyer ends up paying a feed price for records sold as unique.
Ask which of these tests a seller or buyer has actually run, on what sample and on what date, before accepting either label:
| Test | Reading for freshness | Reading for novelty | Main weakness |
|---|---|---|---|
| Record creation dates against the buyer model's training cutoff | Share of records created after the cutoff | Silent: recent records can repeat public content | The cutoff is the developer's own statement |
| Overlap scan against public corpora and benchmarks | Silent: overlap says nothing about age | Low exact and near-duplicate match against named corpora | Covers only the corpora scanned; paraphrase slips through |
| Matched held-out set, in the style of GSM1k | Score gap between items written before and after the cutoff | Score gap between public items and unseen lookalikes | Needs a matched twin set large enough to read |
| Temporal drift test | Scores fall as the test period moves past training | Silent | The rate of decay differs by domain |
| Contract terms | Cadence, term length and refresh clauses | Exclusivity, confidentiality and no-publication clauses | Rarely public; read under a confidentiality agreement |
| Repeat order history | Renewals keyed to new periods | New held-out sets ordered after saturation | Needs dated cohorts over several years |
Could announcements and capability gains stand in for real data spending?
Partly, yes, and this is the strongest objection to any demand thesis built from public material. Most of the evidence above sits on the first rung, and capability progress and partnership news can rise for years while commercial data budgets stay narrow.
The clearest public window into paid licensing shows how narrow. Reddit's Form 10-K for fiscal 2025 says the company has negotiated content licensing agreements with a number of partners that are medium-term in length, yet that, to date, "substantially all of the contract value associated with our licensing revenue is derived from two of our partners." Several named relationships, little spread of money: an investor who counts partnerships as demand would overstate it.
Two cheaper routes also absorb demand before it reaches a supplier. Benchmark builders can manufacture freshness from public sources, as LiveBench does with recent competitions, papers and news. Synthetic data can stand in for some novelty: the Phi-4 technical report, published in December 2024, describes synthetic data used throughout training and making up the bulk of the pretraining mix, seeded by curated web content, licensed books and code.
What would settle the objection is evidence from rungs two to five, by dated cohort: written specifications that name a date window or an overlap test, sample results, prices and renewals. That evidence is private, which is why the measurement design below matters more than any headline.
What could break a freshness or novelty thesis
Five developments could shrink the premium for fresh or novel records. Name them before the upside; each has a public marker an investor can watch.
- Retrieval could absorb freshness. The 2020 retrieval-augmented generation paper by Patrick Lewis and colleagues named updating a model's world knowledge as an open problem for knowledge stored in its parameters, and paired the generator with a searchable document index instead. Recent material served through retrieval can reach users without a training run, so freshness budgets may flow to feeds and search rather than to training licenses.
- Synthetic data could absorb novelty for some tasks. Where generated data passes a buyer's acceptance test, as the Phi-4 report argues for its own training mix, the premium for unseen real records narrows to the tasks generation cannot reach.
- Buyer concentration could cap both. Reddit's annual report for fiscal 2025 shows two partners behind substantially all of its licensing contract value and cautions that its agreements might not be renewed, or might be renewed on worse terms.
- Novelty gets used up. A model learns a dataset's contents once, which is why the GSM1k authors withheld their problems from public release. A supplier selling novelty needs new material, new buyers or new tasks, not repeat sales of the same records to the same model.
- Measurement is immature. An overlap scan covers only the corpora it checks, and a training cutoff is a claim by the model's developer, so a freshness or novelty label can rest on weaker tests than it suggests.
How to measure paid demand for each property: an interview design
Interview independent budget owners and code each data purchase by the property its terms price, rung by rung and cohort by cohort. SourceX has not run this study and no results are published; the design is laid out so that anyone can run it and compare.
At a glance
How to measure paid demand for each property: an interview design
The coding rule does most of the work. A purchase counts as freshness-driven when its specification, price or renewal depends on when records were created, and as novelty-driven when it depends on absence from named corpora, exclusivity or confidentiality. When both apply it is coded as both; when neither is written down it is coded as unstated rather than guessed. The full design:
Checklist · 6 items
- Question: for each outside data purchase or request in the last four quarters, what did the specification, the price and any renewal depend on?
- Population: data procurement, post-training and evaluation leads who control a data budget at model developers and at enterprises building AI products, with no commercial tie to the interviewer.
- Unit and cohort: one purchase decision, assigned to the quarter of its first written request and followed through evaluation, commitment and renewal.
- Variables: highest rung reached, property coded from the terms, domain, delivery form (feed, archive or held-out set), and whether retrieval, synthetic data or a public benchmark was tried first.
- Denominators and uncertainty: every rate names its denominator; samples under 30 organizations are reported as counts with ranges, not percentages; a second coder re-codes a sample and the agreement rate is published.
- Limits: only decisions backed by a dated document count, since people can describe purchases they never made; no reported cell may identify a single buyer.
0 of 6 ticked. Tick each item as you go.
Questions for a supplier of fresh or novel data, SourceX included
Ask which property each contract prices, and for the evidence from rungs two to five behind the answer. The same questions apply to SourceX.
Take SourceX as the worked case. By its own description it turns selected business records into licensed AI datasets, carrying each one from sourcing and qualification through rights review, contracting, preparation and delivery to payment. Nothing sits in stock: datasets are sourced from U.S. businesses when a buyer asks, with the buyer stating scope, volume, language, format, timing and licensing needs. Its general guide is that buyers seek consistent history of typically two years or more and a volume in the tens of thousands of records or more in one system, a profile weighted toward depth and unpublished history rather than recency, though every dataset is assessed deal by deal.
SourceX has not published demand, deal or commercial metrics, including any split between freshness and novelty; its company and commercial metrics are handed to investors directly. SourceX's Rule 506(c) raise is aimed at accredited investors, who can request materials through the investor page and bring any of these questions:
- For each active contract, does the price or the renewal depend on record dates, on absence from public corpora, or on neither?
- Which overlap scans have been run, against which corpora, and on what date?
- How many buyers reached a paid commitment, and how many of those renewed or ordered a refresh?
- Which datasets lost value after a buyer trained on them, and how was the loss measured?
- How many requests were withdrawn because retrieval, synthetic data or a public benchmark proved good enough?
- Which buyer holds the largest share of contract value, and what happens if it does not renew?
Methods note: sources behind the freshness-novelty ladder and tests
The freshness-novelty ladder, the six-test evidence matrix and the interview design are editorial constructs devised for this article on 2026-10-26. Their public evidence comes from OpenAI's Reddit partnership announcement (May 2024), data partnerships page (November 2023) and GDPval (2025); Anthropic's agent evaluation guide (January 2026) and Claude Opus 4.6 system card (February 2026); the GSM1k, LiveBench, Mind the Gap, retrieval-augmented generation and Phi-4 papers; and Reddit's Form 10-K for fiscal 2025. Every rung assignment is the author's reading of the cited text.
Not measured: prices for fresh or novel data, the share of AI data spending on either property, renewal rates, overlap rates for any real dataset, and any SourceX demand or deal data. None of these can be measured from outside without buyer and supplier records that nobody has published.
Frequently asked questions
What is the difference between data freshness and data novelty for AI?
Freshness is about when a record was created; novelty is about whether its information already appears in the data a model learned from. A record can be fresh and familiar, such as a new post repeating public facts, or old and novel, such as a never-published claims file. Buyers price the two differently because they decay for different reasons.
Do AI developers pay more for recent data?
No public source shows a price premium for recency on its own. Deal language shows recent material being bought as ongoing access, as in OpenAI's 2024 Reddit partnership, but filings do not split contract value by freshness. Treat any claimed recency premium as unverified until a contract term or a renewal shows it.
Why does benchmark contamination matter to data buyers?
A contaminated benchmark inflates scores, so a buyer cannot separate what its model learned from what it memorized. GSM1k's unseen problems exposed accuracy drops of up to 8% for some models against the public benchmark they mirrored. Unseen evaluation material is therefore worth paying for, and worth keeping private.
Can old business records still be novel to an AI model?
Yes. Novelty depends on whether the information is absent from training corpora, not on its age, so records that were never published can stay novel for decades. An overlap scan against named public corpora is the way to check, and its result covers only the corpora scanned.
Does SourceX publish demand data split by freshness or novelty?
No. SourceX publishes no demand, deal or commercial metrics, split or otherwise; investors obtain company and commercial metrics from it directly. The interview design in this article shows how such a split could be measured, and any offering by SourceX is limited to verified accredited investors.
Sources
- OpenAI's May 2024 Reddit partnership announcement says OpenAI will access Reddit's Data API, which provides real-time, structured and unique content, to better understand and showcase Reddit content, especially on recent topics. Source
- OpenAI's data partnerships page (November 2023) seeks large-scale datasets not already easily accessible online, favors data expressing human intention, and is not seeking sensitive or personal information or information belonging to a third party. Source
- OpenAI's GDPval spans 44 occupations in the nine industries contributing most to U.S. GDP, with 1,320 specialized tasks (220 in the open gold set) crafted and vetted by professionals averaging 14 years of experience. Source
- Anthropic's January 2026 agent evaluation guide says capability evals should start at a low pass rate and that capability evals with high pass rates can graduate into a regression suite. Source
- Anthropic's Claude Opus 4.6 system card (February 2026) reports the model saturated all current cyber evaluations, so current benchmarks can no longer track capability progression, and that harder evaluations are being prioritized. Source
- The current version of the GSM1k paper (Zhang et al.) reports accuracy drops of up to 8% on newly written grade-school math problems, several model families showing systematic overfitting, frontier models minimal signs, and the set withheld from public release. Source
- LiveBench (2024) uses frequently updated questions from recent sources such as math competitions, arXiv papers, news articles and datasets, scores them against objective ground truth, and adds and updates questions monthly to limit contamination. Source
- Mind the Gap (Lazaridou et al., 2021) finds Transformer-XL models perform worse predicting text from beyond their training period, increasingly so over time; larger models alone do not fix it, while continual updating mitigates it. Source
- Reddit's Form 10-K for fiscal 2025 says it has content licensing agreements with a number of partners that are medium-term in length, that substantially all licensing contract value comes from two partners, and that agreements may not be renewed or may be renewed on less favorable terms. Source
- The 2020 retrieval-augmented generation paper (Lewis et al.) names updating world knowledge as an open problem for parametric models and pairs a generator with a non-parametric memory, a dense vector index searched by a retriever. Source
- The Phi-4 technical report (December 2024) says synthetic data is incorporated throughout training and makes up the bulk of the pretraining data, with curated web content, licensed books and code used as seeds. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.