Skip to content

AI data market

Which AI failures create dataset demand? Six rungs to paid data

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

A public AI failure signals real dataset demand only when it repeats on a sound task, new data is the plausible fix, and an independent budget owner pays for that fix. Many fail earlier: in 2026 OpenAI reported flawed tests or descriptions in at least 59.4 percent of the SWE-bench Verified problems it audited from those models often failed. Climb all six rungs, from disclosure to paid commitment, before backing supply.

Value drivers

AI failures dataset demand

This page covers which AI failures create dataset demand? Six rungs to paid data.Illustrative

Key takeaways

  • Most public failures stop short of a budget: they turn out to be broken tasks, single-run noise, or problems a scoring or tooling change fixes more cheaply than data.
  • Repeatability is measured with pass^k, the chance that all k trials succeed; in tau-bench's original airline domain one agent fell from 0.460 at k=1 to 0.225 at k=4.
  • A failure is data-addressable only when examples, environments or evaluation tasks would plausibly fix it; OpenAI traces hallucination partly to scoring that rewards guessing.
  • Requests, paid evaluations, signed commitments and renewals prove different things; count each by dated cohort and by independent budget owner.
  • Paid demand can still be narrow: Reddit's FY2024 10-K attributes substantially all of the contract value of its licensing revenue to two partners.

When does a public AI failure become dataset demand?

Only when four things hold at once: the failure is real rather than an artifact of a broken test, it repeats, more or better data is the plausible fix, and someone with a budget pays for that fix. A failure that misses any of the four is a research finding, not a market.

The investor decision is narrow. A company argues that a well-publicized model weakness, such as agents that mishandle refunds or models that invent answers, creates demand for the data it supplies. The useful response is a verdict on that specific claim: which rung of evidence the failure has reached, and what would move it to the next.

Evidence that supports a yes is specific and dated: a versioned task with a reference solution, trial-level failure rates across more than one model release, a written request that names the failure, and a paid evaluation or contract tied to it. Capability headlines, leaderboard screenshots and statements of interest do not count on their own. The sections below turn those tests into a six-rung ladder, a sorting list for interventions and a procurement evidence matrix.

The failure-to-budget ladder: six rungs from disclosure to payment

Score a failure by the highest rung it clears without skipping one. Each rung has a test, a dated public example of how failures fall off it, and a condition that sends the claim back down; a failure that skips rung 2 or 3 tends to collapse later, after money has been spent.

The first four rungs can be checked from public sources. The last two need procurement evidence that only buyers and suppliers hold, which is why a pitch that stops at rung 4 is a hypothesis about demand rather than proof of it. The figure shows where the public examples in this article drop off.

The failure-to-budget ladder: six rungs from disclosure to payment
RungTest to clear itPublic example, datedFalls back when
1. DisclosedA dated, versioned public record names the failure at task levelSWE-bench maintainers, August 2024: Verified launched as a 500-problem subsetThe source is an anecdote, a screenshot or an undated leaderboard
2. Sound taskThe task has a reference solution, and two experts would give it the same verdictOpenAI, 2026: at least 59.4 percent of audited, often-failed Verified problems had flawed tests or descriptionsThe grader rejects correct answers or the specification is ambiguous
3. RepeatableThe failure holds across repeated trials and across a model releasetau-bench README: one airline agent scored 0.460 at pass^1 and 0.225 at pass^4The evidence is one run or one model version
4. Data-addressableExamples, environments or evaluation tasks are the plausible fixOpenAI, 2025: hallucination traced partly to accuracy-only scoring that rewards guessingA scoring, tooling, retrieval or policy change fixes it more cheaply
5. OwnedAn independent budget owner names the failure in a dated written requestNot public: requests sit with buyers and suppliersInterest is verbal, or every request traces to one buyer
6. PaidAn evaluation converts into a signed, paid commitment, ideally renewedReddit FY2024 10-K: substantially all licensing contract value from two partnersPilots never convert, or payment rests on one or two buyers
A six-rung ladder from disclosed failure to paid commitment, with exits showing broken tasks, single-run noise and scoring problems dropping off before the top two rungs, which need procurement evidence.
The failure-to-budget ladder. Rungs 1 to 4 can be checked from public sources; rungs 5 and 6 need buyer evidence. Editorial framework, not measured data.

Is the failure real, or is the benchmark broken?

Check the task before the model. Anthropic's January 2026 guide to agent evaluations says that with frontier models, a 0 percent pass rate across many trials is most often a sign of a broken task rather than an incapable agent, and it recommends a reference solution for every task to prove the task can be solved and the graders work.

SWE-bench Verified shows how far this reaches. OpenAI built it in 2024 by having annotators screen 1,699 randomly drawn SWE-bench problems for underspecified issues and overly strict tests, keeping 500. In 2026 OpenAI said it no longer evaluates on the set: auditing the 27.6 percent of problems that models often failed, it found that at least 59.4 percent had flawed tests or descriptions, including tests that rejected functionally correct fixes, and it reported signs that models had seen some problems during training. Most of the hardest-looking failures it audited were defects in the test, not gaps in the model.

Agent benchmarks get revised for the same reason. The original tau-bench repository now warns that its airline and retail tasks are outdated; the later tau3-bench release added more than 75 task fixes, and its 2.0 English runs are not comparable with the 1.x leaderboard. Anthropic also describes Claude Opus 4.5 'failing' a tau2-bench flight task by finding a policy loophole that served the user better.

The decision rule: a failure counted on an unversioned or since-corrected task drops to rung 1. Ask which version the failure was measured on, whether that task has been fixed since, and whether a reference solution passes the grader.

Does the failure repeat across trials and releases?

A demand signal needs a failure that survives a rerun. Anthropic defines pass^k as the probability that all k trials of a task succeed and gives the arithmetic: an agent that succeeds 75 percent of the time per trial passes three trials in a row only about 42 percent of the time.

Public leaderboards show the same gap. In the original tau-bench airline domain, the tool-calling agent built on claude-3-5-sonnet-20241022 scored 0.460 at pass^1 but 0.225 at pass^4, according to the benchmark's README. For a customer-facing workflow that has to work every time, the second number is the one a buyer feels, and the distance between the two is where reliability data could earn a budget.

Repetition across releases matters as much as repetition across trials. Anthropic's January 2026 guide describes SWE-bench Verified scores climbing from 30 percent to above 80 percent within a year, close to saturation. A failure that one release closes is a weak base for a supply business, because the buyer's reason to pay leaves with the next model.

The decision rule: rung 3 needs trial counts, pass^k or an equivalent consistency measure, and the same failure on at least two model versions. A claim that rests on one pass@1 number has not cleared it.

Would data fix it, or would a scoring, tooling or policy change?

Only some failures are data problems, and the cheapest fix usually wins the budget. Before counting a failure as demand, name the intervention a buyer would try first and ask whether new examples, environments or evaluation tasks would beat it on the same test.

Hallucination is the clearest public counterexample. OpenAI's 2025 paper Why Language Models Hallucinate argues that training and evaluation reward guessing over acknowledging uncertainty, and proposes changing how existing benchmarks are scored rather than adding more hallucination tests. Where that diagnosis holds, money flows to evaluation design, not to a bigger corpus.

When data is the plausible fix, the unit decides the size of the purchase. OpenAI's GDPval, released in 2025, covers 1,320 tasks across 44 occupations in the nine sectors that each contribute more than 5 percent of U.S. GDP, written by professionals averaging 14 years of experience and graded blind by peers; its unit is the expert task, not the token. Anthropic tells teams that 20 to 50 simple tasks drawn from real failures are a good start for an evaluation. A real, data-addressable failure can support a small, high-value evaluation purchase long before it supports a large training purchase.

Sort the failure before granting rung 4:

  • Missing coverage of real cases, such as exception paths in claims handling or dispatch: data-addressable through examples or environments.
  • Unreliable multi-step behavior on tasks the model sometimes solves: data-addressable through trajectories and repeated-trial evaluation sets.
  • Scoring incentives, such as confident guessing: fixed mainly by evaluation design.
  • Tool, retrieval or permission errors: fixed mainly by engineering.
  • Unwritten or ambiguous business rules: fixed first by writing the rule, then by examples that follow it.

Requests, evaluations and paid commitments: the procurement evidence matrix

Each kind of buyer evidence proves something different, so count it by dated cohort and by independent budget owner. A request shows someone named the failure; a paid evaluation shows they tested a fix; only a signed commitment shows they valued it, and a renewal shows the value outlived a model release.

Public statements sit below all of them. OpenAI's Data Partnerships page, published in 2023, asks for large-scale datasets that are not easily accessible online, with a stated preference for data expressing human intention, for example long-form writing, or conversations, instead of disconnected snippets. That is one buyer's stated preference: useful for direction, not a purchase order.

Paid evidence can still be narrow. Reddit's FY2024 Form 10-K states that "to date, substantially all of the contract value associated with our licensing revenue is derived from two of our partners," and cautions that those agreements might lapse or be renewed on worse terms. A supplier whose paid commitments come from one or two budget owners has a concentration problem, not yet a market.

Read each signal against its limits:

Requests, evaluations and paid commitments: the procurement evidence matrix
EvidenceWhat it showsWhat it does not showHow to verifyCohort rule
Public failure or data askA capability gap or a buyer preference existsThat anyone will pay, or how muchDated source, task version, named publisherDate of publication
Written requestA budget owner named the failure and a data needA budget, a decision date or a testDated document naming task, unit and intended useQuarter the request arrived
Paid evaluation or pilotThe buyer spent money testing a data fixThat the fix worked or will scaleSigned pilot terms and the acceptance test usedQuarter the pilot was signed
Signed commitmentThe buyer valued the fix enough to contractRepeat demand or a broad marketExecuted agreement, unit bought, count of distinct buyersQuarter of signature
Renewal or expansionThe value outlived at least one model releaseThat other buyers share the needRenewal terms compared with the originalQuarter of renewal, linked to the first cohort

A budget-owner interview design for the top two rungs

Rungs 5 and 6 can only be measured by asking the people who control spend, so the design should be fixed before the first interview. SourceX has not published results from this design; it is set out here so any reader can run it, or ask a company whether it already has.

At a glance

A budget-owner interview design for the top two rungs

  • Question: which public failure categories moved independent budget owners from request to paid evaluation to signed commitment, and how quickly?
  • Population: people who approve spending on training or evaluation data at model developers and enterprise AI teams. Independent means not employed by or invested in the supplier being assessed, and counted once even after changing employer.
  • Selection rule: a sampling frame fixed in advance, with inbound and outbound contacts recorded separately, because inbound contacts already lean toward buying.
  • Unit and cohort: one failure category per budget owner, assigned to the quarter in which that owner first named it in writing.
  • Variables: highest rung reached, the intervention tried first, the unit bought (tasks, records or tokens), the dates of request, evaluation and commitment, and renewal status.
  • Metrics and denominators: the share of each cohort reaching each rung, with the number interviewed in that cohort as the denominator; time between rungs reported as a median, with open cases kept and marked as censored rather than dropped.
  • Uncertainty and limits: small counts reported with intervals, not bare percentages; self-reported spend checked against documents where the owner allows; no extrapolation from interviewed owners to the whole market.

What could make the failure-to-demand thesis wrong

Five developments could cut the link between visible failures and data budgets, and each can be tracked from public sources. Ask a company which of them would hurt it most; one with no answer has not tested its own story.

  • Capability progress closes failures first. METR's time-horizon analysis reports that AI agent time horizons have been doubling about every 7 months; failures that one release fixes never reach rung 6.
  • Labs fix failures without buying data. If more failures look the way hallucination looked to OpenAI's 2025 authors, a scoring problem more than a data gap, budgets go to evaluation design and engineering.
  • Announcements outrun spending. Public data asks and benchmark launches commit no budget, so a market inferred from them can stay hypothetical for years.
  • Benchmarks churn. Corrections such as tau3-bench's 75-plus task fixes can erase failures counted on earlier versions, along with the demand story built on them.
  • Paid demand stays concentrated. Reddit's reliance on two partners for substantially all of its licensing contract value, as its FY2024 10-K reports, could repeat for failure-driven data; a supplier facing a few buyers carries renewal and pricing risk however real the failure is.

Diligence questions for any company claiming to fix public failures, SourceX included

Ask for evidence at the rung where the company claims its advantage, using your definitions rather than its own. A credible answer arrives with task versions, trial counts, dated cohorts and denominators.

SourceX is a labeled application of this test. It sources datasets for AI buyers on request from U.S. businesses instead of holding stock, and buyers state scope, volume, language, format, timing and licensing needs up front, so in this ladder its demand evidence begins at rung 5, a written request, rather than at a public failure. On the supply side it carries the licensing work for the companies that provide records: qualification and rights review, then contracting, preparation, delivery and payment.

Request, evaluation and commitment counts from SourceX are unpublished, and its company and commercial metrics go straight to investors. SourceX's raise falls under Rule 506(c), which confines it to accredited investors; those who request materials can bring any question below, such as how the company assigns requests to dated cohorts.

  • Which public failures does your data address, measured on which benchmark version, and has that task been corrected since?
  • What pass^k or trial counts show the failure repeats, and on how many model versions?
  • What did buyers try before buying data, and did the data beat that alternative on the same test?
  • How many independent budget owners reached written request, paid evaluation and signed commitment in each of the last four quarterly cohorts?
  • Which unit did each buyer pay for: tasks, records or tokens?
  • If the next model release closed the failure, which commitments would survive it?

Methods note: how the ladder and matrix were built

The failure-to-budget ladder, the intervention sorting list, the procurement evidence matrix and the interview design are frameworks of the author's own, drafted for this article on 2026-10-26. Public examples come from Anthropic's January 2026 evaluation guide, the SWE-bench repository and OpenAI's SWE-bench Verified posts (2024 and 2026), the tau-bench and tau2-bench repositories, OpenAI's 2025 hallucination paper and GDPval release, OpenAI's Data Partnerships page, METR's time-horizon repository and Reddit's FY2024 Form 10-K.

Not measured: how many public failures reach each rung, conversion between rungs at SourceX or any other company, budget sizes, prices, or the split of spending between evaluation and training data. Those need procurement records nobody has published; the interview design above describes how to collect them.

Frequently asked questions

Do public benchmark failures mean AI labs will buy more data?

Not on their own. A benchmark failure shows a capability gap on one task version; it does not show that the task is sound, that data is the right fix or that anyone has budgeted for it. Treat it as rung 1 of six and look for repeated-trial evidence and dated buyer requests before counting it as demand.

What is pass^k, and why does it matter for data demand?

pass^k is the probability that all k trials of a task succeed, as Anthropic defines it in its January 2026 evaluation guide. It falls as k rises, so it exposes inconsistency that a single pass@1 score hides. Consistent failure across trials is the kind a buyer may pay to fix; one bad run is not.

How small can the data purchase behind a real failure be?

Smaller than many pitches imply. Anthropic says 20 to 50 simple tasks drawn from real failures are a good start for an evaluation set, so the first purchase a real failure triggers may be a few dozen expert tasks rather than a training corpus. A larger training purchase needs evidence that the fix tested in evaluation scales.

Which model failures are least likely to create data spending?

Failures caused by broken tasks, one-off runs, scoring incentives, tool or retrieval errors, and unwritten business rules. Each has a cheaper fix than new data: repair the test, rerun it, change the scoring, fix the integration or write the rule down. OpenAI's 2025 hallucination paper traces one widely discussed failure partly to scoring.

Has SourceX published data on which failures lead to paid demand?

No. SourceX has published no request, evaluation or commitment results, and this article reports none; it sets out the interview design instead. Investors can ask for the company's materials at sourcex.si/investors, where any offering is limited to verified accredited investors.

Sources

  • Anthropic's January 2026 evaluation guide defines tasks, trials and pass^k, says a 0 percent pass rate across many trials usually signals a broken task, recommends reference solutions and 20 to 50 tasks from real failures to start, and notes SWE-bench Verified nearing saturation above 80 percent. Source
  • SWE-bench Verified was introduced on August 13, 2024 as a subset of 500 problems that software engineers confirmed are solvable. Source
  • OpenAI's annotators screened 1,699 random SWE-bench problems for underspecified issues and overly specific tests and kept 500. Source
  • In 2026 OpenAI audited the 27.6 percent of SWE-bench Verified that models often failed, found flawed tests or descriptions in at least 59.4 percent of audited problems, reported contamination signs and stopped evaluating on the set. Source
  • The original tau-bench README marks its tasks as outdated and reports an airline agent at 0.460 pass^1 and 0.225 pass^4. Source
  • The tau3-bench release added more than 75 task fixes, and 2.0 English runs are not comparable with the 1.x leaderboard. Source
  • OpenAI's 2025 paper argues training and evaluation reward guessing over acknowledging uncertainty and proposes rescoring existing benchmarks. Source
  • GDPval covers 1,320 tasks across 44 occupations in nine sectors that each contribute over 5 percent of U.S. GDP, written by experts averaging 14 years and graded blind. Source
  • OpenAI's Data Partnerships page asks for large-scale datasets not easily accessible online and prefers data that expresses human intention. Source
  • Reddit's FY2024 Form 10-K says substantially all of the contract value associated with its licensing revenue is derived from two of its partners and flags renewal risk. Source
  • METR's time-horizon analysis reports AI agent time horizons doubling approximately every 7 months. Source

Request SourceX investor materials

SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.

See if your company qualifies