Skip to content

AI data market

Post-training data task selection: how to pick data that fixes a specific model failure

By SourceX Editorial · Updated

10 sections

Short answer

To select training data for a specific model failure, start from the failure, not from a catalog. Measure the error cohort on an untouched test slice, confirm that new examples can fix it, choose a selection rule that fits your candidate pool, then compare the selected examples against a random draw of the same size and check other cohorts for regressions. Buy more only where that comparison shows a gain.

Value drivers

Post training data task

This page covers post-training data task selection: how to pick data that fixes a specific model failure.Illustrative

Key takeaways

  • Spend the first post-training data budget on a failure that is frequent or severe, measurable on held-out cases, and caused by missing examples rather than by policy or tooling.
  • A selection rule is a written, repeatable procedure: retrieval by similarity, distribution matching, gradient influence or expert curation, each with different inputs and blind spots.
  • The control that matters is a random draw of the same size from the same pool, trained with the same settings, so the result separates selection from volume.
  • A fix only counts if the cohorts the model already handled, its safety behavior and its output format hold steady.
  • Every selected example should carry its source, the rule version that picked it and its license scope, so the result can be reproduced and the purchase defended.

Which failure should get the first data budget?

Give the first budget to the failure that scores well on four tests at once: it happens often or costs a lot when it happens, you can measure it on cases the model never trained on, new examples plausibly fix it, and a supplier could actually hold records like it. A failure that passes only one of those tests usually belongs in a prompt, a guardrail or a product decision instead.

Write each candidate failure as a cohort, meaning a set of inputs defined by an observable condition, such as support tickets that mention two orders, or invoices with a credit memo attached. A symptom like the model is wrong about refunds cannot be counted, sampled or bought against. Once the cohort is defined, count how many production cases fall into it over a fixed window and how many of those the model got wrong.

Rank the cohorts that pass on a single page and fund one at a time. Running several targeted purchases in parallel feels faster, but when two selected sets land in the same training run, nobody can say which one moved which metric, and the next budget request rests on a guess.

Which failure should get the first data budget?
Triage testQuestion to answerPasses whenFails when
Frequency or severityHow often does the cohort appear, and what does one error cost the user?The cohort is a visible share of traffic, or a single error triggers a refund, escalation or compliance reviewThe cohort is rare and errors are cheap to correct by hand
MeasurabilityCan you assemble held-out cases with a known right answer?A reviewer can label the correct output for each case in minutesCorrectness depends on judgment nobody has written down
Data-fixabilityWould more examples of this pattern change the behavior?Training data has few or no cases with this conditionThe model sees the right information but a tool, retrieval step or policy blocks it
SourceabilityDo real businesses hold records with this condition and its outcome?The condition exists in a system of record such as a ticketing, ERP or claims systemOnly synthetic or adversarial prompts would contain it
ReversibilityIf the fix fails, can you roll back cheaply?Training runs are short and versionedA failed run would delay a release or a safety review

Can more data fix this failure at all?

Only some failures are data problems. Missing coverage, where the training set has few cases with the condition, and shallow reasoning on a recurring pattern respond to examples. Conflicting labels, an unwritten policy, a broken retrieval index or a tool returning the wrong field do not, and buying data for them adds cost without moving the metric.

A quick check is to give the model the missing information directly, for example by pasting the policy clause or the correct retrieved document into the prompt for a sample of failing cases. If accuracy on that sample jumps, the gap is in context or tooling. If it stays low, the model lacks the pattern and examples may help.

Research on image classifiers points the same way. Singla and colleagues started from a handful of failure samples, pulled visually similar examples from a large, noisily labeled pool and trained on those, aiming to improve the failure distribution while keeping performance on the original test set. The method works because the failure is a coverage gap that similar examples can fill, which is the condition to confirm before you buy.

The failure-to-dataset requirements mapping worksheet

Fill in one worksheet per failure cohort before you draw a single training example. The right-hand column is an illustrative entry for an insurance intake assistant; it shows the level of detail each field needs, not a real result. Fields marked as decided before training must not change after the first run, or the comparison stops meaning anything.

At a glance

The failure-to-dataset requirements mapping worksheet

The failure-to-dataset requirements mapping worksheet
FieldWhat to writeIllustrative entryOwner
Cohort definitionObservable input condition plus the wrong behaviorSubmissions listing two or more policies; model extracts only the first policy numberEvaluation lead
Measured error, untouched sliceCases, errors and the date range of the held-out slice, decided before trainingIllustrative: 400 held-out cases from the last quarter, 31 percent extraction errorsEvaluation lead
Root-cause testResult of the context-injection check from the previous sectionPasting the full document still leaves most errors, so coverage is the likely gapApplied researcher
Record type and candidate poolSystem, record and outcome fields a supplier must hold, and how many candidates exist before selectionClosed intake records with every policy number keyed by a human processor; illustrative 20,000 candidates from own logs plus a supplier sampleData sourcing lead
Selection rule and versionThe procedure and its parameters, frozen before trainingEmbedding similarity to 50 seed failures, top 1,000 by cosine score, rule v1Applied researcher
Matched-budget controlSame count and token budget drawn at random from the same pool1,000 random records, same training settings, three seeds per armApplied researcher
Success thresholdThe gain on the cohort that would justify buying more, set in advanceIllustrative: error rate falls by at least a third against the random arm on every seedEvaluation lead
Regression guardsCohorts, safety tests and format checks that must not worsenSingle-policy submissions, refusal tests, JSON schema validityEvaluation lead
Lineage fieldsPer-example source, rule version and license scopeRecord ID, source system, rule v1, license allows fine-tuning onlyData governance
Buy decision and review triggerBuy, narrow, change rule or stop, with the reason, plus what reopens the worksheetBuy more of the selected kind if the threshold holds on all seeds; reopen on new policy forms or a model upgradeBudget owner

Which selection rule suits the candidate pool you have?

Pick the rule by what you already have: a few failure seeds, a target sample that looks like the cohort, a model you can compute gradients on, or expert reviewers with time. Each rule is a bet about what makes an example useful, so write the bet down next to the rule.

The published methods differ mainly in their inputs. LESS selects influential instruction-tuning data for a target capability by running a short warmup training, building a gradient datastore and scoring candidates against target examples; its example scripts keep 5 percent of the pool. DSIR selects raw text distributed like a target dataset by matching feature distributions such as n-gram frequencies, and is built to run on a single CPU node at very large scale. LIMA showed that 1,000 carefully curated prompts and responses could fine-tune a 65B LLaMa model, which makes the case for expert curation when the pool is small and quality varies.

Whatever rule you choose, freeze it before training: the seed set, the embedding or scoring model, the cutoff and any filters for length, language or duplicates. Then deduplicate the selected set against the held-out slice. Similarity retrieval in particular tends to pull near-copies of the very cases you plan to test on, which inflates the measured fix without improving the model on new inputs.

Which selection rule suits the candidate pool you have?
Selection ruleInputs it needsGood fit whenWatch for
Similarity retrievalA few dozen seed failures and an embedding modelThe cohort has a recognizable surface patternNear-duplicates of the seeds and of the test slice
Distribution matching (DSIR-style)A target sample that resembles the cohortThe pool is huge and mostly irrelevantMatching surface features while missing the outcome field
Gradient influence (LESS-style)Model access, a warmup run and target examplesYou can afford the compute and the pool is instruction dataScores that depend on the warmup checkpoint
Expert curationReviewers with domain judgment and written criteriaThe pool is small and quality varies widelyReviewer drift; keep criteria and a second-rater sample
Random drawNothing beyond the poolAlways, as the control armTreating it as a baseline once and never rerunning it

How do you show that selection, not volume, fixed the failure?

Run at least two arms with the same number of examples and the same token budget: the selected set and a random draw from the same pool. Train both with identical settings, repeat each over several random seeds, and score both on the untouched slice. If the selected arm beats the random arm on the cohort and the gap holds across seeds, selection did the work.

Small comparisons mislead in a specific way. Card and colleagues found underpowered experiments common in NLP and showed that significant results from small samples tend to overstate, or even reverse, the true effect. So size the held-out slice for the effect you set as your threshold, report the per-seed results rather than only the mean, and show an interval around the difference.

Two optional arms sharpen the answer. A no-new-data arm, which retrains with the same settings on the existing mix, tells you how much of any gain comes from the run itself. A larger random arm tells you whether simply buying more of everything would have reached the same place at a different volume. The pre-purchase evaluation guide linked below covers arm design for one supplier sample in more depth.

Which regressions must the comparison rule out?

A targeted set fixes nothing if it breaks what already worked. Define the regression guards in the worksheet before training and treat any guard that worsens beyond its own threshold as a failed run, even when the target cohort improved.

At a glance

Which regressions must the comparison rule out

Safety behavior is the guard teams forget most often. Qi and colleagues showed that fine-tuning an aligned model even on a benign, utility-oriented dataset can erode its safety alignment, so a narrow business fine-tune needs its own refusal and harmful-request checks rather than an assumption that clean data is safe.

  • Neighbor cohorts: the inputs that look like the target but should be handled differently, such as single-policy submissions next to multi-policy ones.
  • Previously fixed failures: every cohort that an earlier data purchase repaired, rerun with its original slice.
  • Safety and refusal: the harmful-request and over-refusal sets your release process already uses.
  • Format and schema: the share of outputs that pass your JSON schema or field validator.
  • Length and tone: response length and style drift, which selected sets with a uniform style can introduce.
  • General capability: a small fixed sample of broad tasks, so a narrow gain does not hide a broad loss.

Would the gain survive a different supplier's records?

The assumption that most often flips the decision is that the candidate pool looks like what a supplier will deliver at volume. If the pilot pool came from your own logs and the purchase comes from another company's system, field names, outcome codes and writing style can differ enough to erase the gain. Test the rule on a supplier sample before committing.

Three other assumptions deserve a line each in the decision note: that the cohort's share of traffic will stay roughly where it is, that the held-out slice represents production rather than the easiest cases, and that a gain at one volume continues at a larger one. Show uncertainty as a range for the cohort gain across seeds, the regression guard results with their own ranges, and a plain statement of which assumption would reverse the recommendation if it proved wrong.

Write the decision note so that someone outside the team could repeat it: the cohort definition, the rule version, the arms, the seeds, the threshold set in advance, the result per seed and the guards. A note that only reports the best seed or the mean gain invites a second purchase that the evidence does not support.

Worked example (illustrative): multi-policy insurance submissions

All figures here are illustrative and describe no real model or dataset. An intake assistant extracts policy numbers from broker submissions. The team defines the cohort as submissions listing two or more policies and assembles 400 held-out cases from one quarter, where the model misses at least one policy number in 31 percent of them.

Pasting the full submission into the prompt barely helps, so the team treats the gap as coverage. It builds a pool of 20,000 candidate intake records, selects 1,000 by similarity to 50 seed failures, and draws 1,000 at random as the control. Both arms train with the same settings over three seeds.

The selected arm cuts cohort errors by more than a third on every seed; the random arm moves them only slightly. Single-policy accuracy and schema validity hold, but one seed shows a small rise in refusals on routine requests, so the team adds 100 ordinary single-policy examples to the selected set and reruns that guard before approving more purchases of multi-policy records.

The exception is worth noting. Had the random arm matched the selected arm, the conclusion would have been that volume, not selection, drove the gain, and the cheaper decision would have been to buy general intake records instead of a narrowly specified slice.

What lineage has to travel with each selected example?

Each selected example should carry enough lineage to rebuild the training set and to show where it came from. Without it, a later reviewer cannot tell which rule version picked which record, whether the license allows the use, or whether a test case leaked into training.

Dataset documentation standards already name most of these fields. The Croissant RAI vocabulary from MLCommons covers who created a dataset and why, which sources were used, versioning with timestamps, how the data was processed and the provenance and lineage of derived datasets. Record the same facts per example: source system and record ID, collection window, selection rule and version, selection score, license scope and the deduplication check against the held-out slice.

Lineage also protects the next purchase. When a supplier later offers more records of the same kind, the team can apply the identical rule version to the new pool, compare the selection scores with the first batch and spot a drift in record type or outcome coding before any training run is spent on it.

From a measured cohort to a sourcing request

A filled-in worksheet is also the clearest data request you can send. It states the record type, the outcome field, the volume band suggested by your matched-budget run, and the license scope you need, which is what lets a supplier say yes or no quickly.

Datasets for a targeted fix like this are not held in stock: SourceX sources them for AI teams from U.S. businesses on request, and its request form asks for the scope, volume, language, format, timing and licensing details your worksheet already records. On the supply side it works with U.S., Canadian and EU companies that had at least 50 full-time employees at their peak, not counting contractors, and it can consider companies in the UK, Australia, New Zealand, India, Singapore, South Africa, Latin America, the UAE and Saudi Arabia when the records on offer are at least 90 percent English, measured on the dataset itself. Delivery waits until the supplying business has approved the scope and terms and signed its agreement.

Which of six causes is behind the failure, and what should you try before buying data?

Buy data only when the failure traces to learned judgment, or to domain knowledge that cannot be supplied at run time. The other causes have cheaper fixes, and each can be tested in about a day on the same failing cases the cohort was built from.

The context check earlier on this page separates missing context from missing coverage. Two more tests close the remaining gaps. First, give a qualified person exactly the inputs, tools and access the model had: if they cannot finish the task either, the gap sits in context, tooling or permissions, and no training record will close it. Second, inspect the task itself. Anthropic's engineering team wrote on January 9, 2026 that a 0 percent pass rate across many trials, such as 0 percent at pass@100, is most often a sign of a broken task rather than an incapable agent.

OpenAI's guide to optimizing LLM accuracy draws the remaining line: knowledge that was never in training, is out of date or is proprietary is a context problem for retrieval or prompting, while fine-tuning on examples is a lever for consistency. Write the outcome of each test into the worksheet's root-cause field. A request that cannot name its cause in one row of the table below is not ready to send.

Which of six causes is behind the failure, and what should you try before buying data?
CauseOne-day test on the failing casesPoints to this cause whenTry firstData that could help
Broken task or graderTwo reviewers regrade 20 failures against the written criteria and reference answerReviewers disagree, or mark many failures as correctRewrite the criteria and reference answersNone until the test itself is fixed
Missing contextPaste the needed document or record into the prompt, as in the context check aboveAccuracy recovers once the information is presentRetrieval, input or prompt changeNone for training; at most a reference corpus to retrieve from
ToolingReplay the failures with the correct tool output substitutedErrors disappear when the tool returns the right fieldFix the tool, schema or parserNone
PermissionsGive a qualified person the same access the model had, and read the access logsThe person fails too, or the logs show denied or missing accessChange the access, or route the step to a personNone; this is an access decision, not a data gap
Missing knowledgeAsk the domain questions directly, without the task around themThe model lacks facts a professional in the role would knowRetrieval over reference materialReference documents, only if they cannot be reached at run time
Learned judgmentRun the cases with correct context, tools and access all in placeErrors persist although the model had everything a professional hadA few worked examples in the prompt, as a cheap probeRecords of how professionals decided similar cases, with the outcome

How do you price being wrong and rank failures when the rates are uncertain?

Rank each cohort that passed triage by expected weekly cost: errors per week, times a severity weight, times the chance a data remedy works, each carried as a low and a high value. The weights belong to whoever absorbs the errors; the ones below are placeholders, not industry values.

Cost per error is rarely one number. scikit-learn's cost-sensitive learning example, checked on October 26, 2026, uses a credit cost matrix in which accepting a bad credit costs, on average, five times as much as rejecting a good one, and a fraud case in which the cost of an error depends on each transaction's amount. Anthropic's guide to success criteria sets a target that 90 percent of errors cause inconvenience rather than egregious error, notes that a team still has to define those words, and adds that citation accuracy matters more in a medical app than in a casual chatbot.

Have the owner of each consequence, such as claims operations or compliance, sign the four classes below. In the illustrative table every figure is invented, and ranges multiply low by low and high by high. The multi-policy cohort ranks first under the placeholder weights, but its range overlaps the endorsement cohort's, and a class 4 weight of 400 reverses the order.

That sensitivity is the finding: the purchase should rest on one signed weight, or on another month of counts until the ranges separate, never on midpoints alone. A broad category such as more insurance documents never enters the ranking, because it has no error count, severity class or remedy chance with which to beat a repeated, costly, data-fixable failure.

  • Class 1, cosmetic: corrected in under a minute, and nothing downstream changes. Placeholder weight 1.
  • Class 2, rework: a step is redone or a ticket reopens before a customer is affected. Placeholder weight 5.
  • Class 3, customer-visible: a wrong answer, delay or refund reaches a customer. Placeholder weight 25.
  • Class 4, financial or regulated: money moves to the wrong place, a regulated decision is affected or a safety rule breaks. Placeholder weight 100, and every class 4 cohort also gets its own review.
How do you price being wrong and rank failures when the rates are uncertain?
Cohort (illustrative)Errors per week and main classData-remedy chanceWeekly cost units, placeholder weightsWeekly cost units, class 4 weight 400
Submissions listing two or more policies50 to 80, class 2 rework0.6 to 0.8, coverage gap confirmed150 to 320, rank 1150 to 320, rank 2
Mid-term endorsements that change the policy number2 to 4, class 4 misrouted claim payment0.3 to 0.5, judgment with few examples60 to 200, rank 2240 to 800, rank 1
Broker name formatting100 to 150, class 1 cosmetic0.4 to 0.6, inconsistent conventions40 to 90, rank 340 to 90, rank 3
Low-quality scanned attachments30 to 60, class 2 reworkNear zero, the OCR step failsNot ranked: fix the OCR stepNot ranked: fix the OCR step

What belongs in a dataset specification written from a product incident?

Copy seven lines from the incident record into the specification: the task inputs, the behavior a professional produced, the condition that defines repeats, the cases deliberately excluded and why, what makes a delivered record unusable, the acceptance test, and what the incident review ruled out. Exclusions, rejection grounds and ruled-out causes are the lines most requests drop, and without them a supplier can only send look-alikes.

Anthropic's eval guidance of January 9, 2026 recommends testing both the cases where a behavior should occur and the cases where it should not, and keeping a reference solution for each task. Both habits carry over: exclusions are the should-not cases, and the processor's corrected output is the reference solution. The table continues the endorsement cohort from the ranking above.

Keep the first evidence packet small. The same Anthropic post calls 20 to 50 tasks drawn from real failures a good start for an eval set; that is enough to show the failure repeats and the reference answers can be graded, not to measure a modest gain, which needs the sizing described in the matched-budget comparison above.

Write every rejection line so two reviewers reach the same verdict. On OpenAI's GDPval benchmark, whose tasks come from professionals averaging 14 years of experience and are graded blind by experts in the same occupations, human expert graders agreed with each other 71 percent of the time (Patwardhan and colleagues, 2025). Narrow yes-or-no record tests leave less room for disagreement than those open-ended deliverables; if two reviewers split on a sample, rewrite the line before a supplier sees it.

Send the finished specification with the data request. SourceX keeps no inventory and goes looking among U.S. businesses only once a buyer asks, so the specification is the only thing a candidate supplier can screen its records against.

What belongs in a dataset specification written from a product incident?
Specification lineCopy from the incident recordIllustrative entryRejection or test it enables
Task inputsThe exact documents and fields the model receivedBroker submission plus an endorsement form that changes the policy number mid-termRecords without the endorsement form are rejected
Desired behaviorThe output the processor produced when correcting the caseCurrent policy number extracted, superseded number flaggedGives the acceptance test a reference answer
Inclusion conditionThe observable condition shared by the incident and its repeatsEndorsement effective before the claim or renewal dateDefines what the supplier searches for
Exclusions, with reasonsCases the review decided this fix should not coverCancellations and reinstatements, which follow a separate rule; lines of business the product does not writeStops look-alike records from padding the volume
Record-level rejectionWhat makes a delivered record unusableNo final policy number keyed by a person; missing effective date; near-duplicate of a held-out caseLets the buyer reject records on stated grounds instead of disputing a whole delivery
Acceptance testHeld-out repeats with reference outputs and two graders30 endorsement cases from after the incident, never shared with any supplierMeasures the fix on cases no supplier saw
Negative result keptCauses the review tested and ruled outRetrieval did return the endorsement; the model read it and still chose the older numberShows why data is the remedy, and reopens the specification if that stops being true

Is a rare case worth buying, or only unusual?

A rare case is worth buying when it occurs in the population the model will actually serve, the model fails on it, and an error there sits in a high severity class. An unusual record that the deployment never sees is irrelevant, however scarce it is.

Anthropic's testing documentation tells teams to design evaluations that mirror the real-world task distribution, and lists irrelevant or nonexistent input data among the edge cases to test. An irrelevant record belongs in a test of how the model declines or routes it, not in training. Run this coverage test on your own logs before asking for a supplier sample:

  • Define the deployment population: the channels, products, form versions and regions the model will serve next quarter.
  • Count the rare condition there over a fixed window. Zero cases in n still allows a rate of about 3 in n at 95 percent confidence, since solving (1 - p) to the power n = 0.05 gives roughly 3 divided by n; zero in 2,000 leaves room for about 0.15 percent, a reason to keep watching rather than to buy.
  • Confirm the model fails on the cases that do appear, and record their severity class from the rubric above.
  • Check that candidate records share the deployment's condition, not only its rarity: same product, same form version, a process still in use.
  • Rejection example (illustrative): a supplier sample offers thousands of handwritten, faxed submissions from a broker channel the insurer closed two years ago. The model fails on them, but current traffic contains none and the form is retired, so the sample is rejected with the reason recorded. Mid-term endorsements occur every week in current traffic and sit in class 4, so they pass.

Sources for this section

  • Anthropic's engineering post (January 9, 2026) says 20 to 50 tasks drawn from real failures is a good start, that a 0 percent pass rate across many trials most often signals a broken task, and recommends testing cases where a behavior should and should not occur and keeping a reference solution per task. Source
  • Anthropic's success-criteria guide gives a target that 90 percent of errors cause inconvenience rather than egregious error, notes that a team would define those terms, and says citation accuracy may be critical for medical apps but less so for casual chatbots. Source
  • Anthropic's testing guide says to design evals that mirror the real-world task distribution and lists irrelevant or nonexistent input data among example edge cases. Source
  • OpenAI's accuracy guide treats knowledge missing from training, out of date or proprietary as a context problem, and describes fine-tuning on examples as a way to improve consistency. Source
  • scikit-learn's cost-sensitive learning example uses a credit cost matrix where misclassifying a bad credit as good is five times as costly as the reverse, and a fraud case where the business cost depends on each transaction's amount. Source
  • GDPval tasks were built by industry professionals averaging 14 years of experience and are graded by experts from the same occupations who blindly compare model and human deliverables. Source
  • The GDPval paper reports human expert inter-rater agreement of 71 percent on its gold subset, with an experimental automated grader at 66 percent. Source

Frequently asked questions

What is post-training data task selection?

It is the practice of choosing which tasks and examples go into fine-tuning or preference training based on a measured need, usually a specific failure. It covers which failure to target, which rule picks the examples, and how the choice is tested against a same-size random draw.

How many examples does a failure-targeted fine-tune need?

There is no fixed number. LIMA showed that 1,000 carefully curated examples could shape a large model's behavior, but a narrow business failure may need fewer or more depending on how varied the cohort is. Run the matched-budget comparison at two sizes and let the difference guide the purchase.

Should the failing cases themselves go into training?

Keep the measured failures out of training and use them as the held-out slice or as seeds for retrieval. If they enter training, the test can no longer show whether the model learned the pattern or memorized the cases.

Can an LLM judge select the training examples instead of gradients or embeddings?

Yes, a judge prompted with written criteria is a form of curation and can be the selection rule. Treat the judge prompt and model version as part of the rule, check a sample against human ratings, and still compare against a random draw of the same size.

What if the selected set fixes the failure but hurts another cohort?

Treat it as a failed run. Common repairs are mixing in examples of the neighbor cohort, lowering the share of selected data in the mix, or tightening the selection rule, then rerunning every guard before you buy more.

Sources

  • LESS selects influential instruction-tuning data for a target capability using warmup training and a gradient datastore; its example scripts keep 5 percent of the pool. Source
  • DSIR selects data distributed like a target dataset by matching a feature space such as n-gram frequencies. Source
  • LIMA fine-tuned a 65B LLaMa model on 1,000 carefully curated prompts and responses. Source
  • Data-centric debugging retrieves examples similar to a few failure samples from a noisy pool to improve the failure distribution while keeping original test performance. Source
  • Underpowered experiments are common in NLP, and significant results from small samples tend to overstate or reverse the true effect. Source
  • Fine-tuning an aligned model on a benign, utility-oriented dataset can degrade its safety alignment. Source
  • The Croissant RAI vocabulary documents data sources, versioning with timestamps, processing, and the provenance and lineage of derived datasets. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies