Skip to content

AI uses for records

Dataset expansion regression test: set loss budgets before you train

By SourceX Editorial · Updated

9 sections

Short answer

A dataset expansion regression test checks whether a corpus that lifts your target task also erodes capabilities you already depend on. Before training, freeze the protected evaluation sets, give each a loss budget (a non-inferiority margin), match compute across arms and write the decision rules. Expand only if the target gain clears its preset threshold and every protected set stays inside its budget.

Value drivers

Dataset expansion regression

This page covers dataset expansion regression test: set loss budgets before you train.Illustrative

Key takeaways

  • Regression risk belongs to the expansion, not to the corpus alone: the same records can help at one dose and do damage at another.
  • Published studies have measured forgetting in 1B to 7B models during continual instruction tuning and safety erosion from fine-tuning on benign data.
  • A loss budget is a non-inferiority margin written before training; a protected set passes only when its lower confidence bound stays above it.
  • Matching compute and adding a replay arm let a regression be blamed on the new records rather than on extra training steps.
  • The expansion decision grid maps every combination of target and protected results to expand, mitigate, extend, hold, stop or reject before anyone sees a score.

What does a dataset expansion regression test decide?

It decides whether adding a new corpus to your training mix is worth what it may cost elsewhere: expand at the tested dose, expand with a mitigation, gather more evidence once, or reject. The test compares the current recipe with the expanded one on two kinds of evaluation at the same time, the target task the corpus was bought to improve and a protected set of capabilities that must not get worse.

Anthropic's engineering guide to agent evaluations draws the same line. Capability evals start with low pass rates and give a team a hill to climb; regression evals ask whether the system still handles everything it used to and should pass at close to 100 percent (Anthropic, January 2026). A dataset expansion needs both, scored on the same checkpoints, because a gain on the first can hide a loss on the second.

Useful evidence has four parts: paired, item-level scores for each arm; more than one training seed per arm; a loss budget for every protected set, fixed before training; and a written rule that maps each combination of results to a decision. Without the budget, any loss can be explained away after the fact, and the result becomes a negotiation rather than a test.

This is a pre-expansion experiment, not the continuous regression suite that guards a deployed system. The two share test cases, but the expansion test runs once per candidate corpus and ends in a training or purchase decision.

How can a corpus that lifts one task damage another?

Through forgetting and interference: updating weights toward new records moves them away from what earlier training encoded, and the damage lands on capabilities nobody was measuring. The risk rises when the new corpus is narrow, large relative to the rest of the mix, or formatted differently from what the model has seen before.

The research points the same way. Luo and colleagues found catastrophic forgetting generally present in models from 1 billion to 7 billion parameters during continual instruction tuning, across domain knowledge, reasoning and reading comprehension; the larger models in that range forgot more, and mixing in general instruction data reduced the loss (arXiv 2308.08747). Biderman and colleagues found that full fine-tuning on code forgot substantially more of the base model's abilities than any LoRA setting they tried, while LoRA also learned less of the target (arXiv 2405.09673, 2024).

Safety behavior is part of what can be lost. Qi and colleagues reported that fine-tuning aligned models on benign, utility-oriented datasets such as Alpaca and Dolly degraded their safety alignment, though less than adversarial data did, and that 10 adversarially designed examples costing under $0.20 through a fine-tuning API removed GPT-3.5 Turbo's guardrails (arXiv 2310.03693, ICLR 2024). A purchased corpus does not need many bad records to matter. NVIDIA's Nemotron FAQ adds that supervised fine-tuning on agentic trajectories often degrades performance outside the target domain, and advises against training on a narrow domain alone.

In business terms: a claims corpus that sharpens triage decisions can loosen the JSON schema a downstream system parses, make Spanish replies worse because every new record is English, or teach the model to answer questions it used to escalate. None of those losses shows up on the claims test.

Which capabilities belong in the protected set?

Everything the current model does that someone depends on and the new corpus could plausibly disturb, grouped into five classes and frozen as versioned test sets before training starts. Items in these sets are never used for training, mixture tuning or checkpoint selection, or they stop being able to detect forgetting.

Start with tasks already in production. Anthropic's guide describes capability evals with high pass rates graduating into a regression suite that runs continuously, so a team's own shipped tasks are the first source of protected items. Then add the other classes in proportion to what a failure would cost; a protected set nobody would act on only adds noise to the decision.

The five classes, with examples for a customer support model, are set out below.

Which capabilities belong in the protected set?
Protected classWhat it catchesExample items for a support modelHow the budget is usually set
Shipped tasksLosses on work the model already does wellRefund eligibility answers and order lookups that pass todayTight margin, set from the cost of one extra failure per hundred cases
Safety and escalationHarmful compliance and lost refusals, plus new over-refusalRequests to bypass identity checks; legitimate requests that must still be answeredHard limit on the highest severity tier, a margin on the rest
Output contractFormats that downstream systems rejectJSON schema validity, tool-call arguments, required fieldsNear-zero margin, because a parse failure stops the pipeline
General skillsErosion of reasoning, knowledge and instruction followingMulti-step policy questions, length and language constraintsWider margin, weighed against the size of the target gain
Adjacent populationsDamage outside the corpus's own coverageSpanish tickets, enterprise accounts, rare product linesA margin per segment, with a minimum item count per segment

Loss budgets: how much decline will you accept for the gain?

Write a loss budget for every protected set before training: the largest decline you would accept in exchange for the target gain, in the set's own units. A protected set passes when the lower bound of the 95 percent interval on the paired difference between the expanded arm and the current arm stays above minus the budget.

The idea comes from non-inferiority trials in medicine, where a new treatment has to be shown to be not unacceptably worse than an existing one. The FDA's guidance separates the full effect of the comparator, called M1, from the smaller margin actually used, called M2, which is the portion of the comparator's benefit judged important to preserve, and notes that choosing the smaller value is usual and generally desirable (FDA, Non-Inferiority Clinical Trials to Establish Effectiveness, November 2016). The translation for models is direct: the budget records what the business can afford to lose, decided before the data can argue for itself.

Each protected set then has three possible outcomes. Within budget: the lower bound sits above minus the budget. Breach: the point estimate itself is at or below minus the budget. Inconclusive: the point estimate is within budget but the lower bound is not, which means the test lacks the precision to clear the set either way.

Precision comes from pairing and repetition. Anthropic's statistical guidance recommends analyzing question-level paired differences when two models answer the same items, clustering standard errors when items come in related groups, since clustered errors on popular evals can be more than three times the naive ones, and resampling answers several times per question to cut within-question noise (Anthropic, November 2024). Treat training seeds the same way: one seed per arm cannot separate a corpus effect from a lucky initialization.

Safety gets a hard limit on top of its margin. Any new response in the highest severity tier, on an item the current model handled safely, stops the expansion for review whatever the averages say.

The preregistered protocol: arms, doses and stop rules

Hold the base checkpoint, training recipe, compute and evaluation fixed, and vary only the corpus and how much of it enters the mix. Three arms answer the question; a fourth is worth its cost when the corpus is large or the planned dose is high.

Compute matters because an expansion usually adds training steps, and extra steps can cause forgetting on their own. Replacing tokens instead of adding them keeps the comparison clean, and it follows NVIDIA's advice for narrow-domain fine-tuning: take a small slice from every other domain and reassign it to the new one, so the overall mixture stays close to the original (NVIDIA Nemotron FAQ). For continued pretraining, Ibrahim and colleagues found that learning rate re-warming, re-decaying and replay of earlier data together matched full retraining on all the data in their experiments (arXiv 2403.08763, 2024), which makes a replay arm the natural mitigation to test.

Fill in the protocol and have the decision owner sign it before the first training run:

The preregistered protocol: arms, doses and stop rules
ElementFixed settingWhy it is fixed
ArmsA0 current mix; A1 current mix with the new corpus at the planned dose; A2 as A1 plus a replay slice of the current mix; optional A3 at half doseSeparates the corpus effect from the mitigation and shows whether damage scales with dose
Compute and stepsSame token count and step count in every arm; new records replace tokens rather than add themExtra training alone can cause forgetting
SeedsAt least three training seeds per arm, shared across arms where the framework allowsSeed-to-seed spread is the noise floor the effect has to clear
Evaluation setsTarget set and every protected set frozen, hashed and deduplicated against the new corpusA protected item seen in training cannot detect forgetting
GradersGrader model, prompt, rubric and version pinned; deterministic checks wherever possibleA grader change can move scores without any model change
ThresholdsMinimum useful target gain, a loss budget per protected set and a safety hard limitFixed before results, so no number is chosen to fit them
Stop ruleStop every arm if the first completed seed of A1 crosses the safety hard limitSpares compute on an expansion that cannot pass
Inconclusive ruleIf a decisive interval is too wide, add seeds or protected items once, then decideOne extension, not repeated looks until a result appears

Reading the result: the expansion decision grid

Find the row for the target result and the column for the worst protected result, and take the action in that cell. The target clears when its lower bound is above zero and its point estimate reaches the preset minimum useful gain; it shows no useful gain when even its upper bound falls short of that minimum; anything else is inconclusive.

The columns read the worst protected set, not the average across sets, because an average lets a gain in general skills hide a broken output contract. The safety row overrides everything else.

An illustrative run shows how the grid works. A support-ticket model gains 4.1 points on the target set in arm A1 (95 percent interval 2.0 to 6.2) against a minimum useful gain of 2.0, so the target clears. JSON schema validity falls 1.8 points (interval minus 3.1 to minus 0.5) against a budget of 1.0, a breach. In the replay arm A2 the target gain is 3.6 points (1.5 to 5.7) and schema validity moves minus 0.2 (minus 0.9 to plus 0.5), inside its budget, with every other protected set also inside. The grid says expand with replay at the tested dose. These numbers are invented to illustrate the rules; they are not measured results.

Reading the result: the expansion decision grid
Target resultEvery protected set within budgetWorst protected set inconclusiveAny protected set in breach
Clears the minimum useful gainExpand at the tested doseHold: add items or seeds to the inconclusive set once, then decideMitigate: expand only if the replay or half-dose arm is within budget and still clears; otherwise reject this dose
InconclusiveExtend once with more seeds or target items; do not expand on the point estimateStop: the pilot is underpowered on both axes; redesign before spending moreReject this dose; a doubtful gain does not pay for a measured loss
No useful gainReject: the corpus does not do what it was bought forRejectReject, and record which capability broke for the supplier conversation
Safety hard limit crossedReject and escalate, whatever the gainReject and escalateReject and escalate
A three-by-three grid of target results against the worst protected result, with a safety bar beneath, where an illustrative run moves from the breach cell to the expand cell once a replay arm is added.
The expansion decision grid. Illustrative: the run markers are examples, not measured results.

Could the gain or the regression be an artifact?

Yes, in both directions, so try to make each result disappear before acting on it. A positive target result may vanish under a stronger baseline, a proper holdout or repeated trials, and a regression may come from the setup rather than from the records.

Run these checks before the decision owner reads the grid:

  • A stronger baseline. Compare A1 with the best you could do without the corpus, such as a tuned prompt, retrieval over the same records or the strongest current model, not only with A0. If that rival already reaches the minimum useful gain, the corpus has not earned any loss budget.
  • A proper holdout. Deduplicate the target and protected sets against the new corpus with exact and near-duplicate matching; target items that appear in the corpus inflate the gain, and protected items that appear in it hide forgetting.
  • Repeated trials. Anthropic runs multiple trials per task because model outputs vary between runs; report every seed and trial, never the best one.
  • Extra steps. If A1 trained longer than A0, the regression may belong to the schedule rather than the records; the matched-compute rule exists to rule this out.
  • Grader drift. Re-grade a sample of A0 outputs with the pinned grader at the end of the study; if A0's score moves, the grader moved.
  • Dose. A loss at the planned dose may vanish at half dose, and a pass at a small dose may not hold at full volume; the optional A3 arm shows which way the curve bends.

What the test cannot tell you, and the evidence to collect next

It cannot protect capabilities you did not list, and it measures one model at one scale. A protected set is only as good as the inventory behind it, so a capability nobody wrote down can still be lost without any alarm.

Scale is the second limit. NVIDIA says it generally runs data ablations on models under 10 billion parameters and that results seem to generalize better for models above about 3 billion, which it describes as an observation rather than a firm threshold (NVIDIA Nemotron FAQ). Treat a small-model regression test as a screen, and repeat the protected-set check at the size you will deploy.

Later training can change the picture too. NVIDIA reports that a small regression on unrelated domains, introduced by a light supervised warmup in its Ultra model, was recovered by later on-policy distillation, while noting that it did not test that step as a general cure for forgetting. Run the protected sets again after every later training stage, and keep them in the production regression suite once the expansion ships.

The next evidence worth collecting is a dose curve with at least three points, the protected-set results at deployment scale, and a comparison of the shipped model against A0 on samples of live traffic in the first weeks after release.

Putting protected sets into a data request

Put the target task, the acceptance test and the protected classes you will run into the request itself, so a supplier can say early whether its records can arrive in slices that make the dose and replay arms possible. A corpus delivered as one undivided block can only be tested at full dose.

Characteristics worth asking for include stable record identifiers for deduplication against your evaluation sets, segment and language fields so adjacent populations can be checked, date ranges that allow a time-based holdout, and a format close to the one the model already sees. None of these makes a corpus pass; they make the test possible.

Share the classes and the rules with a supplier, not the protected items themselves. A supplier who sees the items could shape the corpus around them, and protected items have to stay out of every training and selection step.

Datasets reach buyers through SourceX by request rather than from a catalog in stock: it looks for U.S. businesses whose records fit the need a buyer has described, covering scope, volume, language, format, timing and licensing terms in the request form. From there it handles sourcing and qualification, rights review, contracting, data preparation and delivery. A request that states the intended use and an acceptance test with protected sets is the most precise way to describe what a corpus has to do and what it must not undo.

Frequently asked questions

What is a dataset expansion regression test?

It is a controlled comparison between your current training recipe and the same recipe with a new corpus added, scored on the target task and on protected evaluation sets for capabilities you already rely on. Each protected set has a loss budget fixed before training. The expansion goes ahead only when the target gain clears its threshold and no protected set breaches its budget.

How is a loss budget different from a CI regression threshold?

A CI threshold usually blocks any drop beyond noise on a deployed system. A loss budget is a non-inferiority margin chosen for one expansion decision: it states how much decline on a protected set is acceptable in exchange for a specific target gain. It is judged on the lower confidence bound of a paired difference, not on the point estimate alone.

Does LoRA avoid regressions when a corpus is added?

It tends to reduce them, not remove them. Biderman and colleagues found that LoRA forgot less than full fine-tuning on code but also learned less, and that on math a high-rank LoRA forgot nearly as much as full fine-tuning. Test the adapter setting you will ship against the same protected sets.

How many training seeds does the test need?

Three per arm is a practical minimum for a screen, because one seed cannot separate the corpus effect from a lucky run. Score items in pairs across arms so each seed carries more information. If a decisive interval is still too wide, add seeds once under the inconclusive rule written before training.

What if only safety regresses?

Stop and escalate if the regression reaches the hard limit on the highest severity tier, whatever the target gain. Below that limit, treat safety like any other protected set with its own budget, and test a mitigation arm that mixes safety data back in before deciding.

Should a data supplier see the protected sets?

No. Share the protected classes and the decision rules, not the items. A supplier who sees protected items could filter or shape the corpus around them, which would stop those items from detecting forgetting.

Sources

  • Anthropic's guide separates capability evals from regression evals, which should pass at close to 100 percent; capability evals with high pass rates can graduate into a regression suite; models' outputs vary between runs, so multiple trials are run. Source
  • Anthropic recommends paired differences on shared questions, clustered standard errors (over three times naive ones on popular evals) and resampling answers to reduce within-question variance. Source
  • NVIDIA's Nemotron FAQ advises against fine-tuning on a narrow domain alone, recommends reassigning a small slice of every domain to keep the mixture close to the original, and notes SFT on agentic trajectories often degrades out-of-domain performance. Source
  • Luo et al. observed catastrophic forgetting in 1B-7B models during continual instruction tuning across domain knowledge, reasoning and reading comprehension, more severe at larger scale in that range, reduced by general instruction data. Source
  • Qi et al. found fine-tuning on benign datasets such as Alpaca and Dolly degrades safety alignment to a lesser extent, and 10 adversarial examples costing under $0.20 jailbroke GPT-3.5 Turbo. Source
  • Biderman et al. found full fine-tuning on code forgot substantially more than any LoRA configuration, LoRA learned less, and a rank-256 LoRA on math forgot nearly as much as full fine-tuning. Source
  • Ibrahim et al. found that learning rate re-warming, re-decaying and replay of previous data matched fully retraining from scratch on all data in continued pretraining experiments. Source
  • The FDA's 2016 non-inferiority guidance distinguishes M1, the assumed effect of the active control, from M2, the smaller margin preserving a clinically relevant portion of that effect. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies