Skip to content

Systems and records

World model dataset freshness: what a factory layout change made stale

By SourceX Editorial · Updated

10 sections

Short answer

After a factory layout change, a training example is stale when what it teaches depended on something that moved: map coordinates, routes, travel times, reachable zones, station order or equipment. Log each change in a register with its time, area and map version, join every example to it, then keep, relabel, keep as labeled history or retire the example. Local physics such as a grasp usually survives; routes and timings usually do not.

Video and sensors

World model dataset

This page covers world model dataset freshness: what a factory layout change made stale.Illustrative

Key takeaways

  • Staleness after a layout change is about dependency, not age: an example is stale if its state, action or consequence relied on what moved.
  • Record each change once, with time, area, map version and what changed, then join examples to it.
  • Every example gets one of four verdicts: current, relabel, keep as labeled history, or retire.
  • Rule out recalibrated sensors, moved cameras and missing events before blaming the layout.
  • Plants already keep change records in work orders, layout drawings, routing versions and robot maps; use them.

How do you identify stale physical AI examples after a factory layout changes?

List what changed, then test every example for dependence on it. An example is stale when the state it starts from, the action it shows or the consequence it records would be different under the new layout. An example that does not touch the changed area or process stays valid, however old it is.

This matters because world models learn transitions: given this scene and this action, what happens next. Some are trained heavily on robot interaction video; the V-JEPA 2 authors, for example, post-trained an action-conditioned model on under 62 hours of unlabeled robot video. With that little interaction data, a block of examples showing travel along an aisle that no longer exists can teach the wrong physics of the site.

Evaluation suffers too. The WILDS benchmark authors found that standard training gives substantially lower performance out of distribution than in distribution on real-world shifts. If old and new layouts are mixed without labels, a test split can look healthy while the model is quietly being graded on a site that no longer exists.

What makes an example stale rather than just old?

An example is stale when its prediction target depends on a fact the change altered. Sort examples by the kind of fact they depend on; that tells you which ones to check first and which are almost always safe.

Tag dependencies once, when examples are created or imported, rather than after each change. A short list of dependency tags per example, taken from this table, turns every future layout change into a join instead of a fresh review of the whole dataset.

What makes an example stale rather than just old?
DependencyExampleStale ifUsually survives if
Absolute positionForklift pose in the site map frameThe map was rebuilt or the area was re-surveyedCoordinates can be transformed exactly to the new map
Route or pathTravel from receiving to a rackAisles, doors or one-way rules changedBoth endpoints and the path between them are untouched
TimingCycle time at a station, queue waitStation order, staffing or conveyor speed changedOnly areas upstream of a buffer changed
ReachabilityWhether an arm can reach a binFixture, bin or table position changedThe cell was moved as one rigid unit
Process sequenceScan, then label, then sealRouting or work instructions changedThe change only renamed steps
Local physicsGrasp, push, pour, tape a boxTooling, material or product changedOnly the location of the work changed

The spatial and process change impact register

Keep one register entry per change, written when the change happens or reconstructed from records as soon as possible afterwards. The register is the join key between the site's history and the dataset.

At a glance

The spatial and process change impact register

The spatial and process change impact register
FieldWhat it holdsTypical sourceIllustrative value
change_id and typeIdentifier; layout, equipment, routing, staffing, software or sensorChange request or work orderCHG-031, layout
effective_from and toWhen the change took effect and, for temporary changes, when it endedWork order close date, shift logNight shift start, open-ended
areaZones or cells affected, as polygons in a named frameLayout drawing revisionZones P2 and P3, aisles 7 and 8
map_version_before and afterRobot or site map identifiers on either side of the changeFleet manager or mapping toolMap v14 to v15
what_changedPlain description plus the facts it alters: positions, paths, timings, reach, sequenceEngineering notesPacking cell moved; aisles 7 and 8 merged
sensors_touchedCameras moved or recalibrated, scanners added or removedMaintenance and calibration recordsCamera C4 remounted
affected_dependenciesWhich dependency types from the staleness table applyAnalyst reviewPosition, route, timing
verdict_ruleDefault verdict for examples that touch the area, and exceptionsAnalyst and domain expertRetire routes through aisle 7; relabel poses
confirmed_byOperating-side person who confirmed the descriptionSite contactShift engineering lead

Joining examples to changes by time, place and map version

Join on three keys at once: the example's time window against the change's effective dates, its spatial footprint against the change's area, and the map version it was recorded under. Time alone over-marks examples from unaffected areas; place alone misses examples whose route merely crossed the changed zone.

Robot frames make the map version essential. The ROS convention in REP 105 says a robot's pose in the map frame can jump discretely whenever localization updates, while the odometry frame stays continuous but drifts. A rebuilt map means positions recorded in the old map frame are not directly comparable with new ones, so store the map identifier with every pose rather than assuming one global frame.

The same REP warns that retention of data in the odometry frame should be tuned so old or distant data is discarded before error accumulates, and that many problems come from assuming a static environment. A layout change is exactly a broken static-environment assumption, recorded once in the register instead of rediscovered in every training run.

Compute each example's spatial footprint from what it actually covers: the full trajectory for a mobile robot, the reachable workspace for an arm, and the camera's field of view for fixed video. A trajectory that starts and ends outside a changed zone but crosses it midway is affected; a fixed camera whose view includes the changed aisle is affected even if the task it recorded happened elsewhere in the frame, because the background a world model learns from has changed.

Four verdicts for an affected example

Give each affected example one verdict, applied by rule from the register and checked by sampling. Deleting everything near the change is simple, but it throws away local physics that is still true and leaves the dataset thinner in exactly the area the model now has to learn.

Check the rule on a sample before applying it to the whole set. Pull a few dozen examples from each verdict, have someone who knows the site look at them against the new layout, and record how many the rule got wrong. If the rule misclassifies more than the team is willing to tolerate, refine the area polygons or the dependency tags and run the join again, rather than hand-fixing individual examples that the next change will break again.

Four verdicts for an affected example
VerdictWhen to use itWhat to doEvidence to keep
CurrentNo dependency on anything the change alteredKeep as isJoin result showing no overlap
RelabelThe dependency can be corrected exactly, such as a known map transformTransform coordinates or update labels, keep the originalTransform parameters and version
Keep as labeled historyValid for the old layout and useful for learning how layouts differTag with a layout regime and exclude from current-site evaluationRegime tag and the reason for keeping
RetireDepends on a route, timing or reach that no longer exists and cannot be correctedRemove from training and evaluation, keep in an archiveRegister entry and retirement date

Could sensor error, missing events or impossible states explain the shift instead?

Check for those before blaming the layout. A drop in model performance or a change in data statistics around a layout date is a signal to investigate, not proof that the layout caused it.

Compare statistics before and after the change date. TensorFlow Data Validation, for example, can detect drift between different days of training data by comparing dataset statistics against thresholds set in a schema. A drift alarm in the changed zone that lines up with a register entry points to the layout; an alarm elsewhere, or one without a matching entry, needs a different explanation.

  • Sensor change: a camera remounted or recalibrated during the same work can shift every pose and image statistic on its own.
  • Clock or time zone change: an alarm at the change date may be a timestamp shift from new equipment.
  • Missing events: a new conveyor or scanner may not yet log to the same system, so events seem to vanish.
  • Impossible states: travel times shorter than the new path allows usually mean old-map positions joined to new-map routes.
  • Unrecorded informal changes: pallets parked in an aisle for weeks change routes without any work order.

Where a plant already records its changes

Most of the register can be rebuilt from records the site already keeps; the work is joining them. Ask for these alongside the operational data rather than reconstructing changes from the footage.

Some sites are required to keep formal change records. OSHA's process safety management standard, for example, requires written procedures to manage changes to process chemicals, technology, equipment and procedures, and to facilities that affect a covered process. That standard covers only certain highly hazardous chemical processes, but many other plants run similar management-of-change programs voluntarily.

  • Maintenance work orders and project tickets in the plant's maintenance system: dates, assets and areas.
  • Layout drawing revisions: before and after geometry for the register's area field.
  • Warehouse location master and slotting changes in the warehouse management system: which bins and racks moved.
  • Routing and work instruction versions in the manufacturing execution system: step order and station assignment.
  • Robot fleet map versions and keep-out zone edits: the map_version fields.
  • Calibration logs for cameras and scanners: the sensors_touched field.

Worked example: a packing cell moves and two aisles merge (illustrative)

This example is illustrative. A site has 18 months of mobile-robot logs and overhead video. Over one weekend, it moves a packing cell 30 meters and merges two aisles into one wider aisle, and the fleet's map is rebuilt from version 14 to 15. Camera C4 is remounted during the work.

The register gets one entry. Joining the dataset to it marks 22 percent of trajectory examples as touching the changed zones. Of those, poses recorded under map 14 are relabeled through the published transform between maps; routes that used the old aisle 7 are retired because the path no longer exists; cycle-time examples from the old packing cell are kept as labeled history, tagged with the old layout, and excluded from current-site evaluation.

The grasp and tape examples recorded inside the packing cell are kept as current: the cell moved as one unit, so its fixtures and reach did not change. The exception is camera C4: its footage after the remount fails a drift check on image statistics, and the cause turns out to be a lens setting, not the layout. Those clips are held until recalibration is confirmed, under a separate register entry.

When the register cannot decide

Bring in a world-model researcher and someone from site operations when changes are gradual, informal or causally tangled. A register works well for discrete changes with dates; it works poorly for slow shifts such as seasonal product mix, a new crew's habits or clutter that builds up over weeks.

In those cases, narrow the claim instead of forcing a verdict. Mark the period as uncertain, keep it out of evaluation, and say so in the dataset's documentation. Datasheets for Datasets, a widely used documentation format, includes a maintenance section, and that is the place to record the register, the verdict rules and the periods nobody could explain.

Also stop and ask when the change touches safety systems or people. Examples that show workers in a reconfigured area may carry new privacy questions, and a register is not the tool for those.

Requesting operational data with its change history

When you request site data for a world model, ask for the change history with it: work orders, layout revisions, map versions and calibration logs for the same period. Without those, freshness has to be guessed from the footage.

Site datasets are not sitting in a SourceX catalog: each one is sourced from U.S. businesses after a buyer submits a request covering scope, volume, format, timing and licensing needs. Buyers generally look for years of consistent history in one system, and a change register is what makes a long history usable after the site has changed.

Frequently asked questions

Does a layout change make all older training data stale?

No. Only examples whose state, action or consequence depended on something that changed are stale. Local physics such as grasping or taping usually survives if only the location of the work moved.

Should stale examples be deleted?

Usually not. Relabel what can be corrected exactly, keep old-layout examples as labeled history where they help, and retire only what cannot be corrected. Keep retired examples in an archive with the register entry that retired them.

How do I know which map an example was recorded under?

Store the map version with every pose at recording time. If it was not stored, reconstruct it from the fleet manager's map history and the example's timestamp, and confirm with a sample check.

Can drift detection replace a change register?

No. Drift detection tells you that statistics moved; the register tells you why and which examples are affected. Use drift alarms to find changes the register missed.

What if the site never recorded the change?

Reconstruct it from work orders, layout drawings, map versions and the footage itself, and have someone on site confirm the dates and area. Mark any period that cannot be confirmed as uncertain.

Sources

  • The WILDS benchmark finds that standard training gives substantially lower out-of-distribution than in-distribution performance on real-world distribution shifts. Source
  • V-JEPA 2-AC is an action-conditioned world model post-trained on less than 62 hours of unlabeled robot video from the Droid dataset. Source
  • ROS REP 105 says map-frame poses can jump discretely, odometry drifts, retention should discard old data before error accumulates, and many problems come from assuming a static environment. Source
  • TensorFlow Data Validation detects drift between days of training data and skew between training and serving data by comparing dataset statistics. Source
  • OSHA's process safety management standard requires written procedures to manage changes to process chemicals, technology, equipment, procedures and facilities affecting a covered process. Source
  • Datasheets for Datasets proposes dataset documentation questions organized into sections that include collection process and maintenance. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies