AI uses for records
World model intervention data: linking human overrides to effects
By SourceX Editorial · Updated
9 sections
Short answer
Records of human interventions give a world model the off-default transitions routine logs never show: a person saw a state, replaced the planned action and the record shows what followed. They help only when each override is linked to its pre-state, the default action it displaced and its downstream effects inside a declared window, with truncation, competing interventions and selection bias recorded rather than ignored.
Video and sensors
World model intervention
Key takeaways
- An intervention record holds two actions, the one taken and the default it displaced; without the default, half of its value is gone.
- Capture the default from the engine log or plan version at decision time, never rebuild it from a later plan.
- Link effects as separate rows with a link type (direct, propagated or inferred) and the evidence for the link.
- Declare effect windows per intervention type and record whether each one closed on an outcome, on its time limit or because another intervention arrived.
- People intervene when things look wrong, so keep near misses and trigger rules as a comparison set before reading any effect as causal.
What can records of human interventions contribute to a world model?
They contribute the transitions a system's default behavior never produces on its own: a person saw a state, overrode what the plan or the automation was about to do, and the record shows what followed. For a world model that has to predict how states change under actions, those off-default branches are some of the most informative examples available, because they show the process responding to an action it would not otherwise have taken.
Ha and Schmidhuber describe a world model as a learned, compressed spatial and temporal representation of an environment, one an agent can use to imagine outcomes before acting. A model like that only learns consequences it has seen. Logs dominated by routine, automated decisions teach it the default path well and everything else poorly.
The value is conditional. An intervention record helps only when it is linked to the state before it, to the action that would have happened without it, and to the downstream effects within a defined window. An override with no linked consequence is an anecdote, and a consequence with no linked intervention is just another outcome. The rest of this page is about building and validating that link.
How is an intervention different from any other logged action?
An intervention is an action that replaces a default: a recommendation, a schedule, an automated rule or an autonomous policy that would otherwise have run. Every intervention record therefore has two actions in it, the one taken and the one displaced, and a world model dataset needs both.
Robotics research made this structure explicit. In HG-DAgger, the human expert watches the learning policy and takes control when the system heads toward unsafe states, and the takeover segments become training data. The Sirius work on robot learning during deployment treats human takeovers as corrective labels and as a signal of where the policy needs help, then uses the deployment logs to train the next policy. The robot failure and intervention guide on this site covers that robotic case in depth.
Business systems record the same pattern under other names. A dispatcher reassigns a job the scheduling engine routed to a different technician. A planner expedites a purchase order the MRP run left on its standard date. A supervisor releases an order held by an automated fraud rule. A warehouse lead pulls a wave early. A support agent takes over a conversation from a bot. Each one is an intervention only if the system also kept what the default would have been; otherwise it is an ordinary action and the counterfactual half is lost.
Which fields link an intervention to its downstream effects?
The schema below is the asset this page exists for. It has one intervention record per override, plus one effect link per downstream observation. Field names are suggestions; the rules in the last column are what a validator should enforce. Version 1.0, dated 2026-10-18.
| Field | What it holds | Why the world model needs it | Validator rule |
|---|---|---|---|
| intervention_id and source_system | Stable id plus the system of record (for example a ServiceTitan job, a NetSuite purchase order or a Zendesk ticket) | Lets every effect link and every reviewer trace the record back | Unique; source_system from a declared list |
| object_refs with qualifiers | Every business object the intervention touched, each with its role (job, assigned technician, displaced technician, customer site) | States and effects are read per object, so the roles must be explicit | At least one object per declared role; no unqualified references |
| pre_state_snapshot | The state as it was known at decision time: queue, capacity, open commitments, alerts | This is the state the action was conditioned on | Every value has an as-of timestamp at or before decision_time |
| default_action and default_source | What the plan, rule or policy would have done, and where that came from (engine log, plan version, rule id) | Without it the record shows an action but not the alternative it displaced | Required; default_source must be captured before or at decision_time, never rebuilt later |
| intervention_action | What the person actually did, in the same vocabulary as default_action | The action label for the transition | Must differ from default_action; same action vocabulary |
| actor_role and trigger_reason | Role of the person (dispatcher, planner) and the reason code or note, never the person's identity | Separates interventions by expertise and motive, which shapes what they reveal | Role from a declared list; free-text reasons reviewed for personal data |
| decision_time, record_time, clock_source | When the decision was made, when the system stored it, and how the clock is known | Ordering between state, action and effect depends on it | Timestamps carry an explicit offset; record_time minus decision_time within a declared tolerance |
| effect_window | Declared start, end and the event that closes it early | Defines which consequences belong to this intervention | Declared per intervention type before any effects are linked |
| window_status | Closed by outcome, cut off by the window end, or interrupted by another intervention | Separates a finished consequence from a missing one | Exactly one status; interrupted windows name the later intervention_id |
| rights_and_privacy_flags | Whether the records may be licensed, and which fields were de-identified | A model trained on records the supplier cannot license is a liability | Licensing basis recorded; identity fields removed or pseudonymized |
What belongs in each downstream effect link?
Each effect link connects one intervention to one observed consequence, and it should say how confident that connection is. Keep effects as separate rows rather than columns on the intervention, because one override can ripple into several objects at different times.
At a glance
What belongs in each downstream effect link
- effect_id and intervention_id: the pair is unique; an effect may link to more than one intervention only when the link type says so.
- object_ref and qualifier: which object changed (the reassigned job, the displaced technician's next job, the customer's follow-up ticket).
- observed_at and observed_value: the measured consequence, with units and an explicit time offset.
- expected_under_default: what the system's own forecast or plan said for the same field before the intervention, if it kept one. Leave it empty rather than estimating it afterwards.
- link_type: direct (the same object, the field the intervention changed), propagated (a related object through a declared relationship) or inferred (matched by time and place only).
- link_evidence: the key, relationship or rule that produced the link, so a reviewer can rerun it.
- confounders_noted: other events inside the window that could explain the change, such as a second intervention, an outage or a rule change.
Where should the effect window end?
End the window at the first of three events: the outcome the intervention was aimed at is recorded, a declared time limit passes, or another intervention touches the same objects. Then record which of the three happened, because a world model treats them differently.
Reinforcement learning formats already distinguish these cases. Gymnasium's step API returns terminated when the agent reaches a terminal state of the task and truncated when a condition outside the task, typically a time limit, ends the episode. The RLDS format marks steps as first, last and terminal, and says an episode that ends on a step that is not terminal has been truncated. A window closed by its time limit is truncation: the consequence may still be unfolding. Training code that reads it as a final outcome will learn that interventions stop mattering at exactly the length you happened to pick.
Choose the time limit from the process, not from convenience. A dispatch override might settle within the same shift, while a purchase-order expedite only shows its effect when the goods arrive and the downstream work order runs. Declare the window per intervention type before linking effects, so nobody can lengthen it later to capture a favorable outcome.
What does a valid dispatch override record look like? (illustrative)
The record below is illustrative. It is not drawn from any SourceX supplier or customer, and every count and duration in it is invented to show the structure.
At a glance
What does a valid dispatch override record look like? (illustrative)
A field service company's scheduling engine assigns an urgent no-heat call to the nearest available technician. The dispatcher overrides it and sends a more senior technician from farther away, citing the equipment type.
- intervention_id INT-0001; source_system: field service job system; object_refs: job J-17 (role: reassigned job), technician T-A (role: displaced assignee), technician T-B (role: new assignee), job J-22 (role: T-B's next scheduled job).
- pre_state_snapshot as of 09:12 local time with explicit offset: T-A idle and 12 minutes away; T-B on site at another job, 35 minutes away after finishing; equipment flagged as a model with a history of repeat visits.
- default_action: assign J-17 to T-A; default_source: engine recommendation log entry written at 09:11.
- intervention_action: assign J-17 to T-B; actor_role: dispatcher; trigger_reason: equipment expertise.
- decision_time 09:13, record_time 09:13, clock_source: server time with offset.
- effect_window: from decision until J-17 is closed or 72 hours pass, whichever comes first; window_status: closed by outcome.
- Effect E-1 (direct): J-17 arrival 47 minutes after decision; expected_under_default: engine forecast of 15 minutes for T-A.
- Effect E-2 (direct): J-17 closed on the first visit, no return trip inside the window.
- Effect E-3 (propagated through T-B's schedule): J-22 started 40 minutes late; confounders_noted: none recorded.
What should the validator report on a failing record?
Here is the same intervention as it often arrives from a real export, with the findings a validator should raise. Again, the record is illustrative.
| Defect in the record | Rule it breaks | Validator finding | Fix |
|---|---|---|---|
| default_action filled from the current plan, which was regenerated after the override | Default must be captured at or before decision_time | default_source timestamp is later than decision_time | Recover the engine log entry or mark the record as having no default; do not guess |
| Arrival time stored as local time with no offset, decision time stored in UTC | Timestamps carry an explicit offset | Effect appears to precede the intervention by several hours | Normalize both with their offsets; RFC 3339 reserves -00:00 for a UTC time whose local offset is unknown |
| A second dispatcher moved J-22 forty minutes into the window | Interrupted windows name the later intervention | window_status says closed by outcome but a second intervention touches T-B | Set window_status to interrupted and link E-3 to both interventions |
| T-B shown as on site at two jobs at once | State must be physically possible | Overlapping on-site intervals for one technician | Check the job status history; the earlier job was closed late on paper |
| Closure recorded in a separate billing system that was not exported | Every declared outcome has a source | Window closed by time limit with no outcome event | Mark as truncated and request the missing system, rather than reading it as no effect |
What can't intervention records prove about cause and effect?
Intervention records are observational. People intervene when something looks wrong, so the cases with overrides differ from the cases without them before anyone acts. A world model trained naively on them can learn that interventions cause bad outcomes, when the bad situations caused the interventions.
Treat effect estimates as claims that need assumptions. The DoWhy library frames effect estimation as four steps (model, identify, estimate and refute) and provides refutation tests that check the assumptions behind an estimate. That discipline fits here: write down which pre-state fields drive both the decision to intervene and the outcome, and test whether the estimated effect survives when those assumptions are probed.
Three habits reduce the damage. Keep the non-intervened near misses, the cases where the same alert or condition fired and nobody overrode the default, as a comparison set. Keep the trigger thresholds and rule versions in force at the time, so a reviewer can see why some cases surfaced for human attention. And keep the inferred links separate from the direct ones, so a model can be trained or evaluated with and without them.
Object-centric event formats help with the linking half of the problem. The OCEL 2.0 specification lets one event relate to several objects, each with a qualifier describing its role, and lets objects relate to each other. That is the structure an effect link needs when one override touches a job, two technicians and a customer site.
How can a buyer request intervention-linked records from operating businesses?
Most companies that make interventions every day do not keep them in this shape. The pieces usually exist in an engine log, a job or order history, and a status table, but nobody has joined them. A request that names the linkage schema, the intervention types and the window rules gives a supplier a concrete export to scope instead of an open question.
SourceX covers qualification, rights review, contracting, preparation and delivery when businesses license selected records for AI. Buyer datasets are found at U.S. businesses for each request rather than held in stock, through a request form that asks for scope, volume, language, format, timing and licensing needs. Intervention histories usually interest buyers when they span a couple of years or more and tens of thousands of records in one system, although rare workflows can qualify on less.
Keep the permission questions separate. Being able to export an engine log is an access question. Whether operator notes and customer sites can be shared is a privacy question. Whether the company can license the records for training is a rights question. De-identifying the actor and customer fields does not, on its own, create the right to license them.
Frequently asked questions
Is intervention data the same as demonstration data?
No. A demonstration shows an expert doing the whole task, while an intervention shows an expert replacing a specific default at a specific state. Intervention records are valuable because they pair the displaced action with the chosen one, which a demonstration does not contain.
Can a world model learn from interventions alone?
Not well. Interventions concentrate on unusual or risky states, so a model trained only on them sees a skewed slice of the process. Pair them with routine, non-intervened records and with the near misses where an alert fired and nobody acted.
Do I need to know who made each intervention?
Usually not. The role, experience band and reason code carry most of the signal a world model can use, and they avoid sharing an individual's identity. Keep any pseudonymous actor key only if the analysis needs to separate individuals, and decide that before export.
What if the system never logged its default recommendation?
Then the record shows an action but not the alternative it displaced. Mark those records as having no default rather than reconstructing one from a later plan, and use them only where the default is not needed, such as consequence prediction after the action.
How long should the effect window be?
Long enough for the outcome the intervention targeted to be recorded in a typical case, and declared per intervention type before linking. Record whether each window closed on an outcome, on its time limit or because another intervention arrived, so truncated cases are not read as finished ones.
Sources
- Ha and Schmidhuber describe a world model trained to learn a compressed spatial and temporal representation of the environment. Source
- HG-DAgger lets a human expert take control when the learning policy heads toward unsafe states, and trains on those takeovers. Source
- Sirius treats human takeovers during deployment as corrective labels and a signal of trust, and uses deployment logs to train later policies. Source
- Gymnasium's step API separates terminated (a terminal state of the task) from truncated (a condition outside the task, typically a time limit). Source
- RLDS marks first, last and terminal steps, and an episode ending on a non-terminal step has been truncated. Source
- DoWhy structures effect estimation as model, identify, estimate and refute, with a refutation API that tests causal assumptions. Source
- OCEL 2.0 supports qualified event-to-object and object-to-object relationships for object-centric event data. Source
- RFC 3339 defines -00:00 as a UTC time whose offset to local time is unknown. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.