Systems and records
Industrial world model temporal consistency: tests for machine states
By SourceX Editorial · Updated
9 sections
Short answer
To test temporal consistency in machine-state training records, rebuild each machine's timeline on the source timestamp and run rule checks: one state per machine at a time, only allowed transitions, unbroken sequence numbers, counters that only rise or reset on restart, plausible dwell times and effects that follow their causes. Then classify every failure as a source error, a missing event or real causal uncertainty before repairing, flagging or dropping it.
Video and sensors
Industrial world model
Key takeaways
- A world model learns transitions, so one impossible sequence in the training set teaches the model a machine behavior that cannot happen.
- Order every history on the time the source observed the value, keep the receipt time beside it, and measure the gap between the two.
- Write the allowed-transition table from the controller's documented state model before running any test, or the suite has nothing to check against.
- Most failures are not impossible physics: they are clock errors, lost messages or actions taken outside the logged system, and each needs a different fix.
- Never fill a gap in machine state with an interpolated value and then use it as a training target; flag the window or cut the episode there.
- A clean pass shows the records are internally consistent, not that they are true, so a line engineer still reviews the transition table and a sample of flagged episodes.
How do you test temporal consistency in machine-state training records?
Rebuild each machine's state history as a sequence of intervals ordered by the time the source observed each value, then test that sequence against explicit rules: the machine holds exactly one state at any instant, every change is an allowed transition, counters and message sequence numbers never move backward without a logged restart, and every consequence appears after the action that caused it. Each failure is then classified before anything is repaired.
This matters more for world models than for most training sets. A world model is trained to predict the next state given the current state and an action, so it absorbs whatever transitions the records contain. A press shown going from faulted straight to producing parts, with no reset in between, is not noise the model averages away. It is a transition the model will learn to predict.
Episode boundaries need the same care. Reinforcement learning tooling separates an episode that ended because the task reached a terminal state from one cut short by an outside condition such as a time limit; Gymnasium's step API reports these as two separate flags, terminated and truncated. Machine histories need the same distinction: a shift ending or a data export stopping is truncation, not a machine reaching an end state, and a model trained as if it were will learn that machines stop at midnight.
Which timestamp should order a machine's history?
Order the history on the time the value was observed at the source, keep the time it was received alongside it, and treat a large or changing gap between the two as evidence of a problem. Industrial protocols already separate these clocks. OPC UA defines a source timestamp applied by the data source and a server timestamp applied by the server that received the value, and the source timestamp is not supposed to change as the value passes between servers.
Stream processing frameworks draw the same line. Apache Flink's documentation distinguishes event time, when the event occurred on its producing device, from processing time, the clock of the machine handling it, and notes that real setups can deliver elements arbitrarily late, after the system has already moved past their timestamp. A historian or data lake export is a processing-time artifact until proven otherwise.
Three practical checks follow. Confirm the time zone and daylight-saving handling of every source, because a one-hour shift twice a year produces overlaps and gaps that look like machine behavior. Record each device's clock source and any known drift. And keep late-arriving values rather than discarding them, since a late value with a valid source timestamp is still a correct observation.
Build the allowed-transition table before testing anything
The suite tests records against a model of how the machine is allowed to behave, so that model must be written down first. Start from the controller's own state model: the states the PLC or line controller can report, the commands and events that move it between them, and the minimum and maximum time each state can last. Packaging and other discrete lines often use PackML, developed by OMAC and published by ISA as ISA-TR88.00.02, which defines machine modes, states and tag naming conventions without fixing a communications protocol.
At a glance
Build the allowed-transition table before testing anything
Where no published model applies, derive the table from the controller documentation and confirm it with the person who maintains the line. Each row should name a from-state, a to-state, the trigger that causes the move and the evidence a record must carry to show the trigger happened.
- States: every value the state field can take, including fault, maintenance and off, with the source system that reports each.
- Transitions: each allowed from-state and to-state pair, with its trigger (command, alarm, timer, operator action).
- Dwell limits: the shortest and longest time each state can physically last, from rated speeds and equipment manuals.
- Counters: which fields are cumulative, what resets them, and which event marks a legitimate reset.
- Capacity: buffer, conveyor and accumulation table limits that cap how many parts can sit between machines.
- Interventions: actions that happen outside the controller, such as lockout for maintenance, manual jogging or bypassed sensors, and where each is recorded (CMMS work orders, shift logs).
The state transition consistency test suite
Run these ten tests per machine and per linked pair of machines. Each returns the failing intervals, not just a count, so that triage can look at the actual records. The default action column is a starting point; the triage in the next section decides the final treatment.
Order matters. Run T6 first, because a clock offset makes T5, T7 and T9 fire on records that are otherwise fine. Run T3 and T8 next, so that windows with lost messages or dead sessions are marked before the transition tests read them as real behavior. T1, T2 and T4 then test each machine on its own, and T7, T9 and T10 test machines against each other and against the maintenance record. Report results per machine and per test as counts of failing intervals and as a share of total observed time, so a reviewer can see whether a problem is a few long gaps or many short ones.
| Test | What it checks | Failure signal | Default action |
|---|---|---|---|
| T1 Single state | State intervals for one machine never overlap | Two states share any instant | Hold episode for triage |
| T2 Allowed transition | Each consecutive state pair appears in the transition table | A pair missing from the table, such as fault to running with no reset | Triage: data error or table gap |
| T3 Sequence continuity | Message sequence numbers increase by one, with the protocol's wraparound | A skipped or repeated number | Mark the window as unobserved |
| T4 Counter monotonicity | Cumulative counters only rise, or fall to zero at a logged restart | A drop with no restart or birth event | Split the series at the drop and flag it |
| T5 Dwell bounds | Time in each state sits within physical minimum and maximum | A cycle shorter than the rated minimum, or a state far beyond the maximum | Check clocks first, then flag |
| T6 Clock sanity | Source time precedes receipt time within a set tolerance; no future times | Negative or drifting gap, or timestamps ahead of the export date | Correct only with a documented offset |
| T7 Causal order | Each consequence follows its action (reject after inspection, restart after fault cleared) | Effect timestamp earlier than its cause | Triage: clock skew or wrong link |
| T8 Session validity | Values after a disconnect or stale marker are not read as observed | State carried forward through a dead session | Truncate the episode at the disconnect |
| T9 Conservation | Parts out never exceed parts in plus work in progress; buffers never exceed capacity | More units leave a station than entered it | Flag both linked machines |
| T10 Intervention coverage | Gaps and odd transitions line up with logged maintenance or operator actions | Unexplained gap with no work order or log entry | Mark the action as unobserved |
Is the failure a clock error, a lost message, a real contradiction or an unlogged action?
It can be any of the four, and the job of triage is to decide which. Most failures fall into four classes, and they call for different treatment, so classify every failing interval before touching it.
At a glance
Is the failure a clock error, a lost message, a real contradiction or an unlogged action
- Source error: the machine behaved correctly but the record is wrong. Typical evidence is a clock offset that is constant across a device, a unit change after a firmware update, or a duplicated message. These can often be corrected with a documented rule.
- Missing observation: the machine did something that was never recorded. Sparkplug, an MQTT specification used on plant floors, requires edge nodes to number data messages so each is one greater than the last, wrapping from 255 to 0, so a skip shows lost messages. The same specification requires host applications to mark an edge node's metrics as stale when the connection is lost. A gap is a gap: it cannot be filled with the last known value.
- Impossible state: the record shows a combination the machine cannot reach, such as running while locked out. If the clocks are sound and no messages are missing, either the transition table is incomplete or a sensor is lying. Both need the line engineer.
- Causal uncertainty: the sequence is possible but the cause is not in the data, for example an operator who cleared a jam by hand without logging it. The record is valid as an observation but not as an action-consequence pair.
Worked example: a filler and capper on one packaging line (illustrative)
The following is an illustrative example; every count and percentage is invented to show the method, not drawn from a real line. A team assembles 14 days of state, counter and alarm history for a filler and the capper downstream of it, plus the line's maintenance work orders.
Clock checks come first. T6 shows the capper's source timestamps running a steady 4 seconds behind the filler's, with no drift, so the team records a fixed offset and applies it before any other test. Without that step, T7 would have reported capped bottles arriving before they were filled.
After the offset, T3 finds 37 skipped sequence numbers on the capper's edge node, clustered in two windows of about 20 minutes each. Both windows match a network switch replacement in the work orders. They are marked unobserved and the episodes are truncated there.
T2 flags 2.1 percent of the filler's transitions as fault to running with no reset command. This looks like corrupted data, and it is the exception worth remembering: the line engineer explains that the filler auto-resets a low-level alarm when the bottle infeed recovers. The transition table was wrong, not the records. The team adds the transition with its trigger, reruns the suite and the flag rate drops to under 0.1 percent, all of it on one shift.
That remaining shift is the real finding. T9 shows the capper reporting more caps applied than bottles the filler released for about 40 minutes, and T10 finds no work order. The supervisor's shift log records a manual reject-and-rerun of a batch. The episode is kept as a valid observation but its consequence is marked unobserved, because the action that caused it never reached the controller.
Repair, flag or drop: what each failure class permits
A source error with a constant, documented cause can be repaired. A missing observation can only be flagged or cut. An impossible state is held until an engineer explains it. Causal uncertainty keeps the observation and drops the causal label. The rule underneath all four is that a training target must be something that was observed.
Counters show why this matters. Prometheus defines a counter as a cumulative value that can only increase or be reset to zero on restart, so a monitoring system treats any fall as a reset. A machine part counter that falls with no restart record is not a reset to smooth over; it may be a replaced controller, a counter rollover or a second machine writing to the same tag. Split the series at the drop and keep the reason in a flag column.
Every repair belongs in a log that ships with the data: which test fired, which rule changed the record, the before and after values and who approved the rule. A buyer or a later team should be able to undo any repair and see the raw record.
| Failure class | Allowed treatment | Not allowed | Record in the repair log |
|---|---|---|---|
| Source error | Correct with a constant, documented rule | Per-record hand edits with no rule | Rule, scope, before and after values |
| Missing observation | Flag the window or truncate the episode | Carrying the last value forward as observed | Window start, end and evidence of the gap |
| Impossible state | Hold until explained, then fix the table or drop | Silently dropping the interval | Engineer's explanation and decision |
| Causal uncertainty | Keep the observation, mark the action unobserved | Inventing the missing action | What was searched for and not found |
What a clean pass does not prove, and when to call in the line engineer
Stop and bring in the line engineer, and a world model researcher if the training design depends on it, in four situations: the transition table cannot be confirmed by anyone who runs the line; more than a small share of transitions fail T2 after the table is corrected; failures cluster around one device in a way that suggests a faulty sensor; or the episodes most useful for training are the ones T10 cannot explain.
The suite has limits worth stating in the dataset documentation. It proves internal consistency, not truth: a sensor that reports a plausible but wrong value passes every test. It cannot recover actions taken off-system, so it cannot turn observational records into evidence of what would have happened under a different action. And its rules go stale. Rerun it, and revise the transition table, whenever sensors, layout, controller firmware, calibration or the ownership of a data source changes.
Who may use the records is a separate question from whether they are consistent. Access to a historian, a vendor's right to telemetry under a service agreement and a company's right to license the data are different things, and the contract, not the test suite, settles them.
Asking for machine-state histories that can pass these tests
A request that makes these tests possible asks for more than state values. It asks for source and receipt timestamps with their time zones, protocol sequence numbers or message identifiers, counter definitions and reset events, the controller's state model, the maintenance and shift records that explain interventions, and a statement of which systems each field came from.
Teams building industrial world models can ask SourceX to source this kind of history from U.S. businesses through the request-data form, which is where scope, volume, format, timing and licensing needs are described; datasets are sourced to order rather than held in stock. Machine histories spanning at least two years and tens of thousands of records or more from one system are the usual buyer expectation, though a rare operation can justify a smaller set and every request is judged individually.
Frequently asked questions
What is temporal consistency in industrial training data?
It means a machine's recorded history could actually have happened in the order shown: one state at a time, only allowed transitions, counters that do not run backward and effects that follow causes. It is checked by ordering records on source time and testing them against the machine's documented state model.
Should a world model be trained on records with gaps in them?
Yes, as long as the gaps are marked. Truncate episodes at a gap or flag the unobserved window, and never fill it with the last known value as if it had been observed. A model can learn from partial episodes; it should not learn from invented ones.
Which timestamp should order machine records: source or server?
Use the source timestamp, the time the device observed the value, as the ordering key. Keep the server or receipt timestamp as a second field, because the gap between them reveals clock drift, buffering and late delivery.
Can an impossible state transition ever be legitimate?
It can look impossible only because the transition table is incomplete. Auto-reset alarms, bypass modes and manual jogging often create real transitions that were left out of the documented model, so an engineer reviews a sample of flagged transitions before any are dropped.
How often should the consistency tests be rerun?
Rerun them on every new data delivery and whenever sensors, controller firmware, line layout, calibration or data ownership change. Each of those can invalidate the transition table, dwell limits or clock offsets that the tests rely on.
Sources
- Gymnasium's step API reports terminated (the agent reached a terminal state of the task) separately from truncated (a condition outside the task, typically a time limit). Source
- OPC UA's DataValue carries a source timestamp applied by the data source and a server timestamp applied by the receiving server; the source timestamp is passed on unchanged. Source
- Apache Flink distinguishes event time from processing time and notes that real-world elements can arrive arbitrarily late, after the watermark for their timestamp. Source
- PackML, published as ISA-TR88.00.02, defines machine modes, states and tag naming conventions and does not specify a communications protocol. Source
- Sparkplug requires each data message from an edge node to carry a sequence number one greater than the previous message, wrapping from 255 to 0, plus a timestamp of when it was sent. Source
- Sparkplug requires a host application to mark an edge node's metric data as stale when it loses its connection to the MQTT server. Source
- Prometheus defines a counter as a cumulative metric that can only increase or be reset to zero on restart. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.