Skip to content

AI uses for records

World model failure attribution: was the prediction or the record wrong?

By SourceX Editorial · Updated

10 sections

Short answer

To separate model prediction errors from errors in source records, treat each miss as unattributed until the record passes ordered checks: time alignment, units, sensor health, physical possibility, episode completeness, unlogged interventions and an independent second source. Misses that pass go to two domain adjudicators. Report counts for model error, each source-error type and undetermined, and rerun them with the key assumptions changed.

Video and sensors

World model failure

This page covers world model failure attribution: was the prediction or the record wrong.Illustrative

Key takeaways

  • A world-model miss is unattributed until the record behind it passes integrity checks; only then can it count as a model error.
  • Run cheap checks first: clock offsets, units and sensor health settle many misses before any expert is involved.
  • Impossible states are record errors by definition, so keep a constraint table for the operation and test every logged state against it.
  • Use two independent domain adjudicators and report how often they agree.
  • Always report an undetermined count, and show how attribution shifts when the time tolerance or source of truth changes.

How do you separate model prediction errors from errors in source records?

Treat every world-model miss as unattributed until the record behind it passes a fixed set of integrity checks. Count a miss as a model error only when the logged state is time-aligned, comes from a working and calibrated sensor, is physically possible, has no unlogged intervention or missing event around it, and is confirmed by a second, independent source or by an expert adjudicator.

Misses that fail a check go to a named source category instead: source record error, missing observation, impossible or inconsistent state, ground-truth label error, or undetermined. The undetermined bucket matters most. Folding it into model error makes the model look worse than it is; folding it into source error hides real model weaknesses.

The worksheet below turns that rule into a record per miss, so the attribution can be audited, rerun when an assumption changes, and used to decide whether the fix is more training, better data or a different evaluation set.

Why is failure attribution harder for world models than for classifiers?

A classifier is wrong about one label at one moment. A world model predicts how a scene or system evolves after an action, so its target is a future state recorded by some operational system, and that recording can be late, partial or wrong in ways a single label rarely is.

Ha and Schmidhuber's World Models paper described a model that learns a compressed spatial and temporal representation of an environment and predicts what comes next. When the environment is a real warehouse, plant or fleet instead of a simulator, "what comes next" is read from scanners, controllers, maintenance tickets and video, each with its own clock and its own failure modes.

Label error is not a hypothetical. Northcutt, Athalye and Mueller studied the test sets of 10 widely used datasets and estimated a lower bound of 3.3% label errors on average, with about half of algorithmically flagged candidates confirmed by human review. Oakden-Rayner and colleagues showed a second trap: aggregate scores can hide poor performance on important subsets nobody labeled. Both point to the same discipline. Check the record and the slice before blaming the model.

What should the failure attribution worksheet include?

Fill one row set per miss. The worksheet records what the model predicted, what the record says happened, the evidence for and against the record, and who decided the attribution and how confident they were.

Keep the inputs as raw as possible. Copy the logged values and timestamps exactly, including units and time zones, and link to the source record rather than paraphrasing it, so a second reviewer can repeat the check without asking.

What should the failure attribution worksheet include?
FieldWhat to recordEvidence sourceIllustrative entry
Case and episodeCase ID, episode ID, step index and model versionEvaluation run logCase 0412, episode 88, step 31, model build of 3 October
Predicted stateThe model's predicted next state or event, with any confidence it reportsModel outputPallet at dock door 6 within 90 seconds
Logged stateThe state the record shows, with units, time zone and source systemWarehouse management system scanPallet scanned at staging lane B after 4 minutes
Time baseClock used by each stream and the measured offset between themDevice clock settings, sync logsScanner clock ahead of video by about 2 seconds
Sensor provenanceWhich device produced the observation, its status and calibrationDevice registry, calibration recordsOverhead camera 14; uncalibrated; firmware changed in September
State constraint checkWhether the logged state was physically and operationally possibleLayout, capacity and rule tablesLane B full at that time; second pallet could not fit
Interventions and eventsAny manual override, stop or out-of-band action near the stepSupervisor log, maintenance ticketsForklift reassigned by radio; no system entry
Coverage and occlusionWhether the relevant area was visible or instrumentedCamera map, video reviewDoor 6 occluded by trailer for 40 seconds
Second-source evidenceAn independent record of the consequenceShipping manifest, carrier pickup scanCarrier pickup scan shows pallet left from door 6
Attribution and confidenceFinal category, confidence level and the adjudicatorsAdjudication recordSource record error, high confidence, two reviewers agreed

Which checks come before blaming the model?

Run the checks in order, from cheapest to most expensive, and stop at the first one that fails. A miss caused by a clock offset does not need a domain expert's time, and the order makes the work predictable.

Robotics conventions help with the first two checks. In ROS, the standard message header carries a timestamp and a coordinate frame for each observation, and the CameraInfo message says its header timestamp should be the time the image was acquired and that an uncalibrated camera should leave its calibration matrices zeroed. The RLDS episode format separates a terminal step from a truncated episode and allows an invalid flag for incomplete episodes. If your records carry these fields, use them; if they do not, the worksheet's time base and sensor rows have to reconstruct them.

Which checks come before blaming the model?
CheckQuestionPass conditionIf it fails, attribute to
Time alignmentDo the prediction and the logged state refer to the same moment?Offset measured and within the agreed toleranceSource record error (timing)
Units and encodingAre units, coordinate frames and codes the same on both sides?Units and frames match after documented conversionSource record error (encoding)
Sensor healthWas the device working, calibrated and on known firmware?Status and calibration records exist for the periodSource record error (sensor)
Physical possibilityCould the logged state exist given layout, capacity and rules?No constraint violatedImpossible or inconsistent state
Episode completenessDid the record end naturally or get truncated?Terminal step present or truncation markedMissing observation
InterventionsDid a person or system act outside the logged actions?No override, stop or manual move near the stepMissing observation (unlogged intervention)
Ground truth labelIs the outcome label itself right?Label agrees with a second source or adjudicatorGround-truth label error
Second sourceDoes an independent record confirm the consequence?Two sources agreeUndetermined if no second source exists

Could source error, missing observations or impossible states explain the result?

Often, yes, and operational systems produce these errors in recognizable ways. A handheld scanner that batches uploads when it regains Wi-Fi stamps events minutes late. A maintenance work order closed at the end of a shift records the paperwork time, not the repair time. A controller counter that rolls over produces a sudden drop that looks like a reversal. A supervisor who moves a pallet by radio instruction leaves no system event at all.

Impossible states are the easiest to prove and the most useful to collect. Two objects in one slot, a conveyor running while its stop is engaged, or a vehicle appearing at two depots at once are record errors by definition. Keep a list of the constraints your operation enforces, such as lane capacity, interlock rules and travel times, and check every logged state against it before review.

Missing observations are harder because absence leaves no trace. Look for gaps: steps with no events for longer than the task normally allows, camera views blocked during the step, or episodes that end without the expected final event. Mark these as missing observation rather than guessing what happened inside the gap.

How should expert adjudication be set up?

Send to adjudication only the misses that pass the automatic checks or that the checks cannot settle. Each case goes to two reviewers who know the operation, such as a shift supervisor and a process engineer, working from the worksheet and the raw records without being told which attribution the model team expects.

Each reviewer picks a category and a confidence level and writes one sentence of reasoning that cites a record. Disagreements go to a third reviewer, and the record keeps all three answers. Report how often the first two agreed; a low agreement rate is itself a finding about how ambiguous the records are.

Automated screens can choose which cases to review first. The open-source cleanlab library, for example, uses confident learning on a model's predicted probabilities to flag likely label issues. Treat those flags as candidates for review, not as decided errors, which is consistent with the Northcutt study's finding that only about half of flagged candidates were confirmed.

Which matching assumptions can flip an attribution, and how should the doubt be reported?

Two assumptions usually decide the outcome: the time tolerance used to match a prediction to a logged state, and which system is treated as the source of truth when two disagree. Change either one and a set of misses can move between model error and source error.

Rerun the attribution with the tolerance halved and doubled, and with the alternative source of truth, and report how many cases change category. If many move, the evaluation is measuring the records as much as the model, and the next step is a cleaner evaluation slice rather than more training.

Show results as counts per category with the undetermined count always visible, not as a single corrected accuracy number. NIST's AI Risk Management Framework places measurement as one of four core functions alongside govern, map and manage; a transparent attribution table is the measurement record that the other functions rely on.

Worked scenario: a pallet-movement model scored against warehouse records

This scenario is illustrative; the counts are invented to show the method. A team evaluates a world model that predicts where pallets will be a few minutes after a dispatch instruction, using warehouse scans as ground truth. The raw evaluation shows 200 misses out of 1,000 predictions.

The automatic checks attribute 46 misses to clock offsets between handheld scanners and the video recorder, 18 to logged states that violate lane capacity, and 22 to episodes truncated before the final scan. That leaves 114 misses for review. Adjudicators find 21 unlogged radio reassignments and 9 wrong ground-truth scans, confirm 61 as model errors, and mark 23 as undetermined because no second source exists.

The team reports 61 model errors, 116 source-related misses and 23 undetermined, not a single adjusted score. Doubling the time tolerance moves 7 cases from model error to source error, which the team reports alongside the main table. The exception: in one aisle the cameras were occluded for most of the evaluation period, so every miss there is reported as undetermined and the aisle is excluded from the comparison until coverage is fixed.

What can the worksheet not settle, and when should you escalate?

  • It cannot prove causation. Two agreeing records show what happened, not why the model got it wrong; that needs ablations or targeted test slices.
  • It does not fix a biased evaluation set. If the logged episodes over-represent quiet periods, attribution will be accurate and still unrepresentative; check slices as well as cases.
  • Escalate to the operating domain expert when constraint tables are incomplete or when the operation changed layout, equipment or rules during the period.
  • Escalate to the data owner when a pattern of source errors points to a system defect, such as a scanner model that always uploads late, because the fix belongs upstream.
  • Escalate to rights reviewers before sending raw records to outside adjudicators; adjudication use may not be covered by the original data terms.

Requesting records that make attribution possible

Attribution depends on records that carry their own context: clocks, device identifiers, calibration status, intervention logs and a second source for outcomes. When you source operational data for world-model evaluation, ask for those fields explicitly, along with the constraint tables that define impossible states.

Records with that context attached can be requested through SourceX, which has nothing in stock and instead looks for U.S. businesses able to supply each request. Listing the attribution fields above in the request's scope, format and licensing needs turns them into requirements rather than nice-to-haves.

Frequently asked questions

What is failure attribution for a world model?

It is the process of deciding, for each wrong prediction, whether the model erred or the record it was scored against was wrong, incomplete or ambiguous. The result is a count per category, including an undetermined category, rather than a single accuracy figure.

How do you tell a model error from a label error?

Check the label against an independent second source or an expert adjudicator before scoring the miss. If the label disagrees with the second source, it is a ground-truth label error; if both sources agree with each other and not with the model, it is a model error.

What should happen to misses that cannot be attributed?

Report them as undetermined, with the reason, such as no second source or an occluded view. Keeping them visible stops the evaluation from overstating either the model's weakness or the data's quality.

How many adjudicators does each case need?

Two independent reviewers who know the operation, with a third to break disagreements, is a workable default. Report the agreement rate between the first two, because frequent disagreement shows the records are ambiguous.

Can confident learning replace expert adjudication?

No. Confident learning tools such as cleanlab rank likely label issues, which helps choose which cases to review first, but flagged items are candidates rather than confirmed errors. Domain review still decides the attribution.

Sources

  • Ha and Schmidhuber describe a world model that learns a compressed spatial and temporal representation of an environment. Source
  • Northcutt, Athalye and Mueller estimated a lower bound of 3.3% label errors on average across the test sets of 10 datasets, and about half of flagged candidates were confirmed by humans. Source
  • Oakden-Rayner and colleagues describe hidden stratification, where models perform poorly on important subsets not identified during training or testing. Source
  • The ROS standard header carries a two-integer timestamp and a coordinate frame ID for stamped data. Source
  • The ROS CameraInfo message says the header timestamp should be image acquisition time and that an uncalibrated camera should leave its calibration matrices zeroed. Source
  • RLDS separates terminal steps from truncated episodes and proposes an invalid flag for incomplete episodes. Source
  • The cleanlab library uses confident learning on model predicted probabilities to detect label issues. Source
  • NIST's AI Risk Management Framework core is composed of four functions: govern, map, measure and manage. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies