Systems and records
Missing transition audit: find business state changes your event log skipped
By SourceX Editorial · Updated
10 sections
Short answer
To audit state changes that never reached an event log, test each case's logged history against an allowed-step state model and against the live record's current state. Illegal jumps and mismatched end states mark missing transitions. Rule out clock and identifier problems first, find the write path that skipped logging, label each step observed or inferred, and reconcile case counts back to source totals.
Checklist
Missing transition audit
Key takeaways
- A missing transition shows up as an illegal jump between logged states or a last logged state that differs from the live record.
- Rule out clock offsets and identifier merges before blaming the log, because both make correct histories look broken.
- Group gaps by write path and period; clusters usually point to an integration or tracking change, not business behavior.
- Infer a step only when exactly one allowed path fits, never invent its timestamp, and label every step observed or inferred.
- Reconcile case counts by last logged state to the live table so the audit has an independent check.
How do you audit state changes that never appeared in an event log?
You audit unrecorded state changes by testing each case's logged history against two things the log did not write itself: a model of which state-to-state steps are allowed, and an independent record of where the case actually ended up. A case whose consecutive logged states are not an allowed step, or whose last logged state differs from the current record, has at least one transition the log never captured.
The audit then decides what to do with each gap: infer the missing step when only one path fits, mark it ambiguous when several do, or exclude the case. The output is a per-case completeness label and a reconciliation back to source totals, so anyone using the reconstructed episodes knows which steps were observed and which were inferred.
Process mining has formal versions of the first test. Token-based replay and alignments both compare a trace with a process model and report where the model needed a step the trace does not contain. The audit below borrows that idea and adds the checks that matter for business systems: tracking scope, retention, clocks and identity.
Why do audit logs skip business state changes?
Most gaps come from how history was configured, not from bugs. A log records what someone chose to track, from the day they chose to track it, through the paths that write to it. Each cause below leaves a different signature, which is why the audit checks for them separately.
At a glance
Why do audit logs skip business state changes
- Tracking scope: only selected fields are tracked. Salesforce field history covers up to 20 fields per object, cannot track formula, roll-up summary or auto-number fields, and for long text records only that an edit happened, without the values.
- Tracking start date: changes made before history was switched on for a field were never recorded, so early cases begin mid-process.
- Retention: history ages out. Salesforce keeps standard field history for up to 18 months in the app and 24 months through the API without Field Audit Trail.
- Bypass paths: bulk imports, integrations or direct database fixes can change a status without going through the code path that writes the application audit trail.
- Uncommitted and unsaved states: database change capture such as Debezium emits only committed row-level changes, so a change that was rolled back, or a status an application set and reverted in memory before saving, never reaches the stream.
- Derived states: a status computed from other fields, such as overdue, may change without any write at all.
Which inputs does a transition completeness audit need?
Collect the inputs before running any test, and write down where each came from. An audit run on an undocumented state model or an unknown clock produces confident labels that nobody can check later.
| Input | What to collect | Why it matters |
|---|---|---|
| Case identity | The case ID, plus merge, split and re-key history for that ID | A merged or re-keyed case looks like a jump between states |
| State model | Every allowed state and every allowed step, including legitimate shortcuts | Without it, a skip cannot be told from a valid express path |
| Tracking configuration | Which fields were tracked, from which date, with which retention | Separates never-recorded from lost-to-retention |
| Clock mapping | Time zone or UTC offset of each source, and whether times are event time or write time | Out-of-order events look like illegal steps |
| Source totals | Counts of cases by current state from the live table on the audit date | The independent check the log cannot fake |
| Write paths | Which users, integrations and jobs can change the state, and which of them log | Points to where the missing transitions came from |
| Role meanings | What each actor type means: person, automation, integration, system | Distinguishes a skipped human step from an automatic one |
| Concurrency and retries | Whether two updates can land at once and whether retries replay events | Duplicates and races produce false gaps |
Which checks make up the transition completeness audit, and in what order?
Run the checks in order, because later checks assume earlier ones passed. Each produces counts by period and by source, never one global number, since a gap rate that jumps in one month usually points to a configuration or integration change rather than to business behavior.
The table is the audit itself. A data team can run it as written and attach the output to any dataset of reconstructed cases.
| Check | Test | Failure signal | Action |
|---|---|---|---|
| 1. Totals reconcile | Cases by last logged state versus cases by current state in the live table | Counts differ for a state | List the differing cases; they carry an unlogged final transition |
| 2. Identity is stable | Every event's case ID exists, and merges or re-keys are resolved | Orphan events or two IDs for one case | Map IDs before any sequence test |
| 3. Clocks are comparable | All timestamps converted to UTC with their offsets; event time separated from write time | Steps that run backward after conversion | Fix the mapping or mark the case's order unreliable |
| 4. Steps are legal | Each consecutive pair of states checked against the allowed-step model | A pair the model does not allow | Classify as a missing transition, a model gap or a data error |
| 5. Start is observed | First logged state is a valid entry state and falls after tracking began | Case starts mid-process | Mark the case left-censored |
| 6. End matches record | Last logged state equals the current field value | Mismatch | Infer an unlogged final step and record its source |
| 7. Write paths covered | Gaps grouped by the actor or integration that last touched the case | Gaps cluster on one path | Treat that path as unlogged and fix capture going forward |
| 8. Duplicates removed | Same case, state and timestamp within a retry window | Repeated events | Keep one and record the rule |
| 9. Confidence assigned | Each step labeled observed, inferred-unique, inferred-ambiguous or unknown | Any step without a label | Do not release the episode until labeled |
How does replay against a state model expose a missing step?
Replay walks each case through the allowed-step model and notes every place the model had to do something the log did not show. pm4py documents two standard forms. Token-based replay reports where tokens were missing or left over; a trace fits only if every transition could fire without inserting missing tokens. Alignments pair trace events with model transitions and label each move: a move on log is an event the model does not allow, and a move on model is a step the model needed that the trace lacks.
For a transition audit, a move on model involving a visible step is exactly the candidate missing transition. A move on log is the opposite problem: the log shows something the model forbids, which usually means the state model is incomplete or a record was edited outside the normal process.
You do not need a full Petri net to get value from the idea. A simple matrix of allowed from-to pairs, with legitimate shortcuts listed explicitly, catches most gaps in business workflows. Graduate to formal conformance checking when cases branch, run steps in parallel or involve several linked objects.
Could clock drift, identifier mismatch or a missing step change the reconstructed case?
Yes, and the audit must rule out the first two before blaming the third. When two systems stamp times differently, a correct sequence can look out of order, and a reordered pair can look like an illegal step. RFC 3339 timestamps carry an explicit offset from UTC for this reason; normalize every source to UTC with its offset before comparing, and keep the original value.
Ordering also depends on how each history is returned. The Jira changelog endpoint, for example, returns an issue's changes sorted oldest first, with each item naming the field and its from and to values. Two items with the same timestamp still need a documented tie-break rule, such as the history entry ID.
Identifier problems are subtler. A support ticket merged into another, an order split into two shipments, or a customer record re-keyed during a migration all produce histories that seem to jump. Object-centric formats such as OCEL 2.0 help here, since they relate events to several objects and let object attribute values change over time, but they only help if the merge and split history was captured.
How can orders ship without being packed? An illustrative case
All counts here are illustrative. A distributor reconstructs 12,000 order cases from an ERP status history with states received, picked, packed, shipped and invoiced. Check 4 finds 9 percent of cases jump from picked straight to shipped.
Grouping by write path in check 7 shows almost all of those cases were last touched by the warehouse system integration, and the jumps begin in the month that integration was replaced. The new connector writes status through a bulk endpoint that does not create history rows. The packed step happened; it was simply never logged. Because only one path fits between picked and shipped, those steps are labeled inferred-unique, with no timestamp invented.
The exception sits in the same data. A small share of the jumps are drop-shipped orders, which skip packing by design. They were flagged only because the state model omitted the drop-ship path. Adding that path to the model removes them from the gap count, and they keep observed labels.
Check 1 then closes the loop: cases by last logged state match current statuses except for a set still showing shipped in the log but invoiced in the live table. Each gets an inferred final step, sourced to the live table.
How much inference is too much?
Inference is acceptable when exactly one allowed path fills the gap and the step's timing is not used downstream. It is not acceptable when several paths fit, when the missing step carries a decision, such as an approval, or when a model will learn from the time between steps.
Never fabricate timestamps for inferred steps. Record an interval between the surrounding observed events instead, and carry the confidence label into every exported episode. A dataset where inferred steps look identical to observed ones teaches a model a process that may never have run that way.
The process mining manifesto sets the bar this audit aims at: events should be trustworthy, so it is safe to assume they actually happened, and logs should be complete, so no events are missing within a given scope. When a source cannot meet that bar for a period or a path, the honest choice is to narrow the scope rather than fill the gap.
When should the audit stop and call in the workflow owner?
Stop when the state model itself is in doubt. If business users disagree about which steps are allowed or skippable, the legality test is measuring opinion, and a workflow owner needs to sign the model first.
Narrow the scope when gaps cluster on a period or write path you cannot repair, such as the months before tracking began or a retired integration. Escalate to a process mining or data engineer when cases run in parallel, link several objects or need formal conformance checking. Escalate to a privacy reviewer before export, since status histories carry user names, customer identifiers and sometimes free-text reasons.
Audited event histories in a licensing review
A history with a completeness label per step and a reconciliation to live totals answers the first questions an AI team asks about workflow data. SourceX is set up for exactly that hand-off: it qualifies the company, reviews rights, handles contracting, prepares the event data, delivers it and manages payment.
Event histories are released only after the company signs off on scope and terms, and a short assessment with no upload shows first whether the company qualifies. Event logs too large to transfer comfortably need not leave your storage: SourceX has them shipped on encrypted drives or left in place instead of hosting them.
Frequently asked questions
What is a missing transition in an event log?
A missing transition is a state change that happened in the business but left no event in the log. It shows up as two consecutive logged states that the process does not allow, or as a last logged state that differs from the current record. Untracked fields, bypassed write paths and retention limits are the usual causes.
Is a skipped state always a logging gap?
No. Some workflows have legitimate shortcuts, such as drop-shipped orders that are never packed or tickets solved on first contact. List those paths in the state model before auditing, or the audit will report valid cases as gaps.
Can change data capture replace an application audit log?
It can fill many gaps, because tools such as Debezium capture committed row-level changes from the database regardless of which application made them. It sees only committed changes, so a rolled-back change or a status an application never saved does not appear, and it cannot recover changes made before capture started.
Should I add timestamps to inferred steps?
No. Record the interval between the surrounding observed events and a confidence label instead. Invented timestamps distort duration features and make inferred steps indistinguishable from observed ones.
What tools can run conformance checks on business logs?
Open-source libraries such as pm4py offer token-based replay and alignment-based conformance checking against a process model. For simple linear workflows, a table of allowed from-to state pairs checked in SQL is often enough to find most gaps.
Sources
- pm4py's token-based replay reports places with missing or remaining tokens, and its alignments label moves on log and moves on model as deviations from the process model. Source
- Debezium captures row-level changes from databases for change data capture, and only committed changes are visible to consumers. Source
- Salesforce field history tracking covers up to 20 selected fields per object, excludes formula, roll-up summary and auto-number fields, records long text edits without values, and keeps history up to 18 months in the app and 24 months through the API. Source
- The Jira Cloud changelog endpoint returns an issue's history sorted oldest first, with each item giving the field and its from and to values. Source
- RFC 3339 timestamps express each time with a stated offset from UTC. Source
- OCEL 2.0 relates events to objects and allows object attribute values to change over time. Source
- The process mining manifesto says events should be trustworthy and event logs complete, meaning no events are missing within a given scope. Source
See if your company qualifies
A short company assessment. No data uploads are needed.