Skip to content

Leadership and readiness

Even weaker models have winning demos: the limits of AI data demo evidence

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

A single demo shows that a system produced one good output under conditions the presenter chose; it cannot show a rate, a direction or a cause. In HELM Lite's public item-level results, the weaker model in 99.7% of 7,466 clearly ranked pairs still answered items the stronger one missed. Ask instead for a fixed test set, disagreement counts in both directions, a same-budget control without the data, and a failure log.

Checklist

AI data demo evidence

This page covers even weaker models have winning demos: the limits of AI data demo evidence.Illustrative

Key takeaways

  • A demo is a selected success: it shows possibility under chosen conditions, never a success rate or a direction against a baseline.
  • In HELM Lite's item-level results, a model 10 or more points worse still won a median 27 GSM8K problems, 34 MedQA questions and 180 LegalBench items that the better model missed.
  • Attempts are cheap to hide: Codex solved 28.8% of HumanEval problems with one sample, while 100 samples per problem solved 70.2% of the paper's problems.
  • Credit the data only when a control changes the data alone: same model, prompt, attempts, compute and grader.
  • Underwrite with fixes and breaks counted in both directions and a failure log with frequencies, not a highlight reel.

What does one demo actually prove about a data business?

It proves possibility, not performance. A demo shows that a system produced one good output, on an input the presenter picked, under settings the presenter controlled. It has no denominator, so it cannot tell an investor how often the output is good, whether a rival system does better on the same input, or whether the supplier's data caused the result.

That gap matters more for a data business than for a software vendor. The usual pitch runs: a model trained or tuned on our records handled this hard case, so our records are valuable. Each link in that sentence is a separate claim. The case may be typical or rare. The baseline may fail it or pass it. The improvement may come from the data, or from a newer model, a longer prompt or more attempts at inference.

This article replaces the highlight with evidence an investor can check: a public recount of how often a weaker model beats a stronger one on individual items, the mechanics that make demos easy to stage, a worksheet for underwriting demo-based claims and a checklist for rerunning them.

Can a single item show which model is better? A recount of 12,195 model pairs

No. In public item-level results, two models disagree on enough items that a presenter can find a showcase in either direction, even when one model is clearly better overall.

We tested this on HELM Lite, the capability leaderboard from Stanford's Center for Research on Foundation Models, which publishes each model's result on each test item. From release v1.13.0 we took three right-or-wrong scenarios: GSM8K (1,000 grade-school math word problems), MedQA (503 test-split questions from US medical licensing exams) and LegalBench (2,047 items from the five legal tasks HELM Lite chose out of 162). Every pair of models that ran a scenario was compared item by item: 4,005 pairs across 90 models on GSM8K and 4,095 pairs across 91 models on each of the other two.

We counted a pair as clearly ranked when the better model led by at least 10 percentage points and an exact McNemar test on the items where the two disagreed gave p below 0.001. In 7,443 of the 7,466 clearly ranked pairs (99.7%), the weaker model still answered at least one item correctly that the stronger model got wrong, and in 6,973 (93.4%) it did so on 10 or more. The medians were 27 of 1,000 GSM8K problems, 34 of 503 MedQA questions and 180 of 2,047 LegalBench items.

Close races are coin flips at the item level: within 5 points, the weaker model won a median 44.0% to 45.6% of the items on which the two disagreed. Counterexamples thin out only at extreme gaps; the widest GSM8K pair, 95.6% against 2.8%, had none. The table gives the median number of items the weaker model won, by accuracy gap, with the number of pairs in parentheses.

Can a single item show which model is better? A recount of 12,195 model pairs
Accuracy gap between the two modelsGSM8K, 1,000 problemsMedQA, 503 questionsLegalBench, 2,047 itemsWeaker model's median share of disagreements (GSM8K / MedQA / LegalBench)
Under 5 points94 (631 pairs)55 (763 pairs)247 (1,238 pairs)44.0% / 44.8% / 45.6%
5 to under 10 points78.5 (474 pairs)45 (693 pairs)210 (901 pairs)33.3% / 35.2% / 37.3%
10 to under 20 points58 (909 pairs)37 (1,159 pairs)206.5 (1,060 pairs)21.8% / 25.2% / 29.6%
20 to under 40 points38 (998 pairs)33 (1,111 pairs)152 (867 pairs)10.1% / 16.0% / 18.2%
40 points or more10 (989 pairs)29 (348 pairs)71 (25 pairs)1.7% / 9.5% / 6.8%
A grid of three benchmarks by five accuracy-gap bands showing the median number of items the weaker model answered correctly that the stronger model missed, from 247 LegalBench items under a 5-point gap down to 10 GSM8K problems at gaps of 40 points or more.
Median items won by the weaker model in each pair, HELM Lite v1.13.0 item-level results, recounted 2026-10-27.

Attempts, staging and fluency: how a demo is built to succeed

Three ordinary practices make a demo look better than the system behind it, and none of them requires bad faith.

Attempts come first. The 2021 Codex paper, Evaluating Large Language Models Trained on Code, reported that its model solved 28.8% of HumanEval programming problems with a single sample, and that drawing 100 samples per problem solved 70.2% of the paper's problems. A presenter who records several tries and keeps the best is running the 100-sample test while implying the one-sample test. Ask how many attempts preceded the clip, and what a single attempt scores on the same inputs.

Staging comes second. Google's developer blog post on how the interactions in its Hands on with Gemini video were prompted describes still frames and text prompts, including worked examples given to the model before a cup-shuffling test. A polished video does not show which scaffolding sat behind it; only the written prompts and inputs do.

Fluency comes third. In The False Promise of Imitating Proprietary LLMs (2023), Gudibande and co-authors found that models tuned on a stronger model's outputs copied its style far better than its factuality. Crowd raters had trouble telling the imitators from the original, while targeted automatic tests showed the imitation data closed little to none of the gap on tasks it did not heavily cover. A fluent live demo is exactly the kind of evaluation that style wins.

Was it the data, or a newer model, prompt or budget?

Treat a demo of a data effect as unattributed until a control changes the data and nothing else. The strongest alternative explanation is rarely fraud; it is that something besides the data changed between the before and after clips.

Even neutral evaluators tune the setup per model. In HELM Lite's run specifications, 20 of the 90 GSM8K runs used a different prompt wrapper from the most common one, such as a preamble asking the model to read the examples carefully and follow their format, and 41 used a different stop-sequence setting. A supplier comparing its own before and after runs faces the same pull, with more at stake.

The control that settles attribution is easy to state: the same base model and version, the same prompt and tool access, the same attempts and compute per task, the same grader and the same frozen items, with the supplier's data as the only difference. Then report both directions, the items the data-trained system fixed and the items it broke. A company that shows only the first number has shown half of a paired comparison.

Inference budget deserves its own line because it can buy a demo outright. The Codex gap between one sample and 100 came from sampling alone, with no change to training data. A claim that data improved results should hold at matched attempts and matched cost per task.

Benchmark underwriting worksheet for demo-based claims

Use the worksheet when a company's evidence for its data is a demo, a case study or a set of curated examples. Each line names what to request and what passing looks like. A line the company cannot fill moves the claim from evidence to anecdote, which is a diligence finding, not a reason for hostility:

Benchmark underwriting worksheet for demo-based claims
LineWhat to requestPasses whenIf it fails
Item populationHow test items were drawn, from what source and by whom, plus the full item listItems were sampled by a written rule before any results were seenTreat the demo as one hand-picked case
DenominatorItems attempted, passed and failed, including the demo itemThe demo item appears inside a full score with its countNo rate can be inferred from the evidence
Attempts and budgetAttempts per item, sampling settings and compute or cost per item for both armsMatched across arms, with a single-attempt score also reportedThe gain may have been bought with retries
Control armThe same model version, prompt, tools and grader without the supplier's dataOnly the data differs between armsAttribution to the data is unproven
Both-direction countsItems the data arm fixed and items it broke, with an exact McNemar testFixes clearly exceed breaks and the test is reportedA highlight from one side says nothing about direction
GradingWritten grading rules, who graded and whether graders knew the armRules fixed in advance and graders blind to the armScores may follow expectations
LeakageOverlap checks between the supplier's records and the test itemsOverlap measured, then removed or reportedThe gain may be memorization
Failure logFailure types with examples and frequencies on the same itemsFailures are counted, not only illustratedThe company does not know where its data breaks

Replication checklist: rerun the claim without the presenter

A claim that passes the worksheet should also survive a rerun by someone who did not make it. The checklist assumes the investor's team or a hired evaluator works from frozen files:

At a glance

Replication checklist: rerun the claim without the presenter

Checklist · 8 items

  1. Receive the test items, prompts, grader and both systems' outputs as files with hashes, before seeing any summary slide.
  2. Rerun a random sample of items with a seed you choose and write down, then compare your item results with the company's.
  3. Recount fixes and breaks yourself from item-level outputs instead of accepting a net score.
  4. Run the control arm at the company's attempt count and again at one attempt per item.
  5. Re-grade a sample of items without knowing which arm produced them, and compare with the company's grades.
  6. Pull five failures at random from the log and confirm each matches its label.
  7. Ask when the demo item entered the test set and how many candidates were tried before it was chosen.
  8. Before the rerun, record the result that would make you discount the claim, so the threshold cannot drift afterward.

0 of 8 ticked. Tick each item as you go.

What a failure analysis should show besides the wins

A failure analysis turns a highlight reel into a map of where the system breaks, and it is the part of the evidence a demo hides by construction. It should name failure types, count them on the same items as the successes and show an example of each.

For a data business, the useful failure types are those the data could plausibly fix or cause: cases the supplier's records do not cover, cases where the records teach an outdated rule, cases where the data-trained arm breaks something the baseline handled, and cases no arm passes. The HELM Lite recount explains why the third type needs its own column. In every gap band the weaker system won some items, so a model improved by data will still lose specific cases to its baseline, and a well-run company knows which ones.

Ask for counts, not only a taxonomy. A failure log without frequencies is another kind of demo.

Risks for the investor, and where this test is too strict

Discounting demos can mislead too, and relying on them carries its own risk. Weigh these before using the worksheet as a veto:

  • Existence is sometimes the real claim. When a capability has never been shown at all, one verified success under disclosed conditions is news; the worksheet then asks only whether inputs, prompts, attempts and edits were disclosed.
  • Frozen test sets can be gamed too. Leaked or overused benchmarks inflate scores, so a test set beats a demo only after overlap with the training records has been checked.
  • Untested capability claims are a regulatory exposure. The FTC's complaint against DoNotPay alleged the company did not test whether its chatbot's output matched the level of a human lawyer; a company that markets from demos carries that kind of risk, which counts against the investment rather than for the demo.
  • Academic items are not business tasks. HELM Lite's items are math, medicine and law questions; longer business workflows with partial credit may show more item-level disagreement or less, and the recount does not measure which.
  • Full evaluation costs money. An early company may not have a frozen test set yet; the fair request is the smallest test that could fail, not a large benchmark.

Questions for any data company that leads with a demo, SourceX included

Put these in writing, and apply them to SourceX as readily as to anyone else.

SourceX licenses records that companies choose to make available for AI training and evaluation, and it handles the work in between: finding and qualifying suppliers, reviewing rights, contracting, preparing and delivering the data, and paying the supplier. This article reports no SourceX dataset outcomes, and the company gives investors its company and commercial metrics directly rather than in articles. SourceX is raising capital under Rule 506(c), from accredited investors alone; a diligence team can request materials at sourcex.si/investors and send any line of the worksheet with the request.

  • How was the demo item chosen, and how many candidates were tried before it?
  • What does the same model score on the same items without your data, at the same attempts and cost?
  • How many items did your data fix, and how many did it break?
  • Who wrote the grader, and did graders know which system produced each answer?
  • Which failure type is most common, and what share of the test items does it cover?
  • Can we receive the items, outputs and grader as files and rerun them ourselves?

Methods note: how the HELM Lite pairs were counted

Data: HELM Lite release v1.13.0 in the public crfm-helm-public bucket. The release's run map (releases/v1.13.0/runs_to_run_suites.json) points each run to a suite between v1.0.0 and v1.13.0; per-item files were retrieved on 2026-10-27 and listed with SHA-256 hashes. Each item was scored with the scenario's HELM main metric on its main split, as given in the release schema: final_number_exact_match for GSM8K and quasi_exact_match for MedQA and LegalBench, all on the test split, using trial 0 and unperturbed rows only. LegalBench's five subsets were pooled, with each item keyed by subset and instance id.

Pairs: every unordered pair of models with the scenario, compared on the items both were scored on. The weaker model is the one with lower accuracy on those shared items; exact ties (4 GSM8K, 21 MedQA and 4 LegalBench pairs) sit outside the gap bands. Clearly ranked means a gap of 10 points or more and a two-sided exact McNemar p below 0.001. Prompt variation compares each run's prompt-text fields and decoding fields with the most common combination across models. No step uses randomness, and the scripts are kept for reviewers.

Not measured: business tasks, repeated attempts per item (HELM Lite scores one attempt), the reason behind each disagreement (some reflect answer-format failures rather than knowledge), and any comparison that changes data alone. The worksheet and checklist are editorial tools drawn up on 2026-10-27 and report no SourceX results.

Frequently asked questions

Is a product demo ever good evidence for investors?

Yes, when the claim is that something can be done at all and the conditions are disclosed: inputs, prompts, attempts and edits. It is weak evidence for any claim about how often, how much better or why. For those, ask for a scored test set with a baseline run under matched settings.

What is a discordant item in a model comparison?

It is an item one system gets right and the other gets wrong. Items both pass or both fail say nothing about which system is better, so a paired comparison rests on the two discordant counts. An exact McNemar test asks whether the split between them is lopsided enough to rule out chance.

How many test items should a data company show?

Enough that the items where its system and the baseline disagree can support a test. The count of disagreements matters more than the size of the test set, because agreements carry no information about direction. Ask for both discordant counts and the test result, not only accuracy.

Can a demo be honest and still misleading?

Yes. Every frame can be real while the selection, the number of attempts or the prompt scaffolding is left out. In HELM Lite's public results, the median model trailing by 10 or more points still got 27 to 180 items right that the better model missed, depending on the benchmark, so an honest clip can point the wrong way.

Does SourceX publish demos or benchmark results for its datasets?

This article reports none. SourceX shares company and commercial metrics with investors directly, and materials can be requested at sourcex.si/investors. Any offering it makes is limited to accredited investors whose status has been verified.

Sources

  • Stanford CRFM introduced HELM Lite in December 2023 as a lightweight, broad capabilities benchmark; it includes GSM8K, MedQA and five of LegalBench's 162 tasks, with at most 1,000 instances per scenario. Source
  • The HELM Lite leaderboard lists release v1.13.0. Source
  • HELM Lite release v1.13.0 maps each run (model and scenario) to the suite that holds its per-item results. Source
  • The v1.13.0 schema names each scenario's main metric and split (final_number_exact_match for GSM8K, quasi_exact_match for MedQA and LegalBench, test split) and describes the item sources. Source
  • HELM publishes per-item results for every run in per_instance_stats.json, including the main metric for each test instance. Source
  • The Codex paper reports that its model solved 28.8% of HumanEval problems with one sample, and that drawing 100 samples per problem solved 70.2% of its problems. Source
  • Google's developer blog explains how the Hands on with Gemini demo was prompted, using still frames and text prompts, including worked examples before a cup-shuffling test. Source
  • Gudibande et al. (2023) found imitation models mimic a proprietary model's style but not its factuality; crowd evaluations struggled to separate them, and targeted evaluations showed little gap closed on tasks not heavily covered by imitation data. Source
  • The FTC's complaint alleged DoNotPay did not test whether its AI chatbot's output was equal to the level of a human lawyer. Source

Request SourceX investor materials

SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.

See if your company qualifies