Systems and records
Event log ground truth limits: when an audit log is not the business truth
By SourceX Editorial · Updated
10 sections
Short answer
An audit log entry proves a system wrote a value at a time on its own clock, not that the business event happened. Treat it as verified only after the case is linked across systems, the timestamp's meaning and time zone are confirmed, the workflow owner defines the state, and reconstructed cases reconcile to independent totals such as invoices or shipments. Grade every case and publish the grades with the data.
Record timeline
Event log ground truth
Key takeaways
- A logged status is a system claim; business truth needs identity, time, meaning and an independent outcome record.
- Transaction timestamps, out-of-order delivery, truncated values, unaudited actions and retention limits are documented ways logs diverge from reality.
- Reconciling reconstructed cases to independent totals is the decisive completeness test.
- Grade each case as verified, system-asserted or uncertain and ship the grades with the dataset.
- Escalate to the workflow owner when a state's meaning changed or automations write the same fields as people.
How do you tell an audit log event from what actually happened in the business?
Treat every audit log entry as a claim that a system wrote something at a time on its own clock, then test that claim before using it as ground truth. An event becomes verified business reality only when its case identity is linked, its time is mapped to the business clock, its meaning is defined by the workflow owner and the reconstructed cases reconcile back to independent source totals.
The distinction matters because event logs are now routinely used to build training and evaluation data for agents: a ticket's status history, an order's change log, a field history table. A status of Closed records that someone or something set a field. It does not prove the work was done, that the customer agreed, or that the close happened when the timestamp says.
This guide sets out where logged events and business reality diverge, the inputs you need to test them, a source limitation and validation worksheet, a reconciliation rule and a worked scenario. The worksheet is designed so a data engineer and a workflow owner can fill it in together and attach it to any dataset built from event histories.
Where a logged event and the business fact it implies can diverge
Most gaps come from how systems write logs, not from bad data entry. Five mechanisms account for the common cases, and each is documented by the vendors and standards bodies that define the systems.
- Transaction time is not business time. PostgreSQL's current_timestamp and now() return the start time of the current transaction, so every change in one transaction carries the same timestamp, whatever order the business steps happened in.
- Delivery order is not event order. Stripe's webhook documentation warns that events may not arrive in the order they were generated, so handlers must not depend on sequence, and endpoints can receive the same event more than once; any log built from received integration events inherits both effects.
- Recorded values can be partial. Salesforce field history records only that a long text field over 255 characters changed, not its before and after values, and Dataverse caps large audited values at about 5,000 characters.
- Some actions are never audited. Microsoft's Dataverse documentation says auditing does not cover table or column definition changes, authentication, or retrieve and export operations.
- History expires. Without Field Audit Trail, Salesforce retains field history for up to 18 months in the interface and up to 24 months through the API, and Dataverse deletes audit records once they pass the retention period stamped on them.
What process mining practice treats as a trustworthy log
The Process Mining Manifesto, published by the IEEE Task Force on Process Mining, sets out the quality bar most event-log work still uses. It asks that recorded events can safely be assumed to have actually happened with correct attributes, that the log is complete for its stated scope with no events missing, and that events have well-defined semantics. Logs that fall short of those properties are common, and the manifesto treats finding, merging and cleaning event data as a core challenge rather than an edge case.
Clock problems sit underneath all of this. NIST SP 800-92 notes that log generation and storage are complicated by inconsistent content, formats and timestamps across sources, and that each host typically stamps entries using its own internal clock. When a case spans a CRM, a ticketing tool and an ERP, three clocks and three meanings of time contribute to one reconstructed history.
None of these properties can be read off the log itself. Trustworthiness, completeness and semantics have to be shown against something outside the log, which is the job of the worksheet below.
Inputs the validation needs before any event is trusted
Collect these inputs first. If one is missing, the worksheet records the gap and lowers the confidence grade instead of assuming a value.
| Input | Definition | Typical source | If missing |
|---|---|---|---|
| Case identifier map | How each event links to one business case across systems | Primary keys, external IDs, integration mapping tables | Cases cannot be joined; validate one system only |
| Clock mapping | Time zone, clock source and timestamp meaning per system | System settings, database docs, integration logs | Order within a few minutes cannot be trusted |
| State definitions | What each status or field value means in the workflow | Workflow owner, process documentation, picklist history | Event labels stay uninterpreted and cannot be graded |
| Source totals | Independent counts of cases and outcomes for the period | Finance ledger, billing, shipping manifests, monthly reports | Completeness cannot be measured |
| Role meanings | Which users, integrations and automations can write each event | Permission sets, integration user list, automation inventory | System actions are mistaken for human decisions |
| Retention and audit scope | What is audited, for how long and with what value limits | Audit settings, retention policy, vendor documentation | Silent gaps look like quiet periods |
The source limitation and validation worksheet
Fill one row per check for each event source. The pass rule is deliberately concrete so two reviewers reach the same answer; the last column says what to do when the check fails, which is usually to narrow the claim rather than drop the data.
At a glance
The source limitation and validation worksheet
| Check | Question | Evidence that passes | If it fails |
|---|---|---|---|
| Identity | Does every event resolve to exactly one case? | Unmatched and multiply matched events listed and counted | Exclude unmatched events and report their share |
| Clock meaning | Is the timestamp the business action time, the commit time or the receipt time? | Documented meaning per source, tested on sample cases | Use date-level ordering only |
| Clock alignment | Are sources on one time zone and a synchronized clock? | Offsets measured on cases that appear in two systems | Widen the ordering tolerance and flag close calls |
| Ordering and duplicates | Can arrival order or retries change the sequence? | Events deduplicated on event ID; sequence uses source time | Mark sequence as uncertain for affected cases |
| Semantics | Does each state mean the same thing across teams and years? | Workflow owner sign-off and a dated change log of definitions | Split the dataset at the definition change |
| Actor | Was the change made by a person, a rule or an integration? | Actor type recorded from user and integration lists | Do not treat the event as a human decision |
| Value completeness | Are before and after values stored in full? | No truncated or edited-only entries in fields that matter | Drop the field from labels; keep as a change flag |
| Coverage | Are any actions or periods outside audit scope or retention? | Audit scope and retention dates documented per source | State the covered period and actions explicitly |
| Reconciliation | Do reconstructed cases match independent totals? | Match rate within the agreed tolerance per period | Investigate before release; do not publish rates |
| Outcome corroboration | Is the business outcome confirmed outside the log? | Invoice, shipment, payment or customer reply linked | Label the outcome as system-asserted only |
Reconcile to source totals, then assign a confidence grade
Reconciliation is the decisive test because it uses evidence the log did not produce. For each period, count the cases the reconstruction says reached a business outcome, count the same outcome in an independent source such as the billing ledger or shipping manifests, and compute the match rate as matched cases divided by the source total. Then look at the unmatched cases in both directions: cases in the log with no outcome record, and outcomes with no case in the log.
Grade each case, not just the dataset, because a dataset-level match rate hides which cases are safe to use as labels. A practical scheme uses three grades: verified, where identity, time, semantics and an outside outcome record all pass; system-asserted, where the log is internally consistent but no outside record confirms the outcome; and uncertain, where identity, ordering or meaning failed a check. Publish the share of cases in each grade with the dataset, and let users decide which grades their task can tolerate.
Agree the match tolerance with the workflow owner before you run the numbers, and write it down. Setting it afterward invites the tolerance to drift toward whatever the data happens to show.
Worked scenario (illustrative): closed tickets that were not resolved
All figures here are invented for illustration. A field service company wants to build an agent evaluation set from 12,000 helpdesk tickets marked Closed in one quarter, using Closed as proof the job was completed. The billing system shows 10,900 completed service visits invoiced for the same quarter.
Identity checks match 11,400 tickets to a work order; 600 have no work order because an automation closes tickets after 14 days without a customer reply. Of the 11,400, 10,700 link to an invoiced visit. The other 700 were closed by technicians who marked a follow-up visit, so the job continued under a new ticket. Clock checks show the helpdesk records UTC while billing records local time, so 3 percent of same-day cases flip order until the offset is applied. The 200 invoiced visits with no matching ticket go on the worksheet as outcomes the log missed.
The result: about 89 percent of Closed tickets are verified completions, 6 percent are follow-up handoffs that need a different label, and 5 percent are automated closures that say nothing about the work. Using Closed as ground truth would have mislabeled roughly one case in nine. The worksheet records each finding, and the dataset ships with per-case grades instead of one status field.
Which assumption could change the reconstructed case?
Test the assumptions most likely to flip a case before you publish anything. Clock drift matters when two events in different systems fall within the alignment tolerance; rerun ordering with the tolerance doubled and count how many cases change sequence. Identifier mismatch matters when merges, re-keyed records or shared accounts exist; sample matched cases by hand and record the error rate you find.
An unrecorded transition matters when the workflow allows changes outside the audited path, such as bulk updates, imports or direct database writes. Compare end states in the current record against the last logged value; every mismatch is evidence of a write the log did not see.
Show uncertainty on the case, not in a footnote. A sequence flag, a confidence grade and the name of the failed check let downstream users filter, and they stop a training pipeline from treating an inference as an observation.
Signals that a workflow owner has to settle the question
Some questions cannot be answered by data engineering. Bring in the workflow owner when a state's meaning changed during the period, when automations or integrations write the same fields people do, when the reconciliation gap cannot be explained by the documented mechanisms, or when the outcome that matters is recorded only in email or on paper.
Escalate with the worksheet row, the affected case count and two or three example cases. Owners answer faster when they can see the concrete record, and their answer becomes part of the dated semantics that the next reviewer relies on.
Validated event histories and licensing review
Event logs often carry user names, email addresses and free-text notes, and integration events can include data the company received under a customer or vendor agreement. Validating the log does not settle who may use it: system access, privacy permissions and contractual rights to license are separate questions from data quality.
Validated operational histories are among the records SourceX helps companies license to AI developers, with rights review, contracting, preparation and delivery handled as part of the deal. Very large histories can remain in the company's storage or ship on encrypted drives, and whether a company qualifies is first checked through a brief assessment that needs no upload.
Frequently asked questions
What should a source limitation and validation worksheet include?
One row per check for each event source: identity, clock meaning, clock alignment, ordering and duplicates, semantics, actor, value completeness, coverage, reconciliation and outcome corroboration. Each row states the question, the evidence that passes and what to do when it fails.
Could clock drift, identifier mismatch or an unrecorded transition change a reconstructed case?
Yes, each can. Clock drift reorders events that fall close together across systems, identifier mismatch joins events to the wrong case, and unrecorded transitions leave the log's last value different from the record's current state. Test each and flag affected cases.
Is an audit trail the same as ground truth?
No. An audit trail shows that a system recorded a change at a time on its clock. It becomes ground truth for a business outcome only after the case is linked, the time and meaning are confirmed and an independent record such as an invoice or shipment corroborates it.
How should uncertainty in an event log be shown?
Attach it to each case: a confidence grade such as verified, system-asserted or uncertain, a flag on uncertain sequences and the name of the check that failed. Publish the share of cases in each grade with the dataset.
What are source totals in event log validation?
Independent counts of cases or outcomes for the same period from a system that did not produce the log, such as a billing ledger, shipping manifests or monthly operational reports. Matching reconstructed cases to them tests completeness.
Sources
- PostgreSQL's current_timestamp and related functions return the start time of the current transaction, so all changes in one transaction share a timestamp. Source
- Stripe says webhook events may not be delivered in the order they were generated, and endpoints may occasionally receive the same event more than once. Source
- Salesforce field history records only that a long text field over 255 characters changed, and retains history for up to 18 months, or 24 months through the API, without Field Audit Trail. Source
- Dataverse auditing does not cover table or column definition changes, authentication, or retrieve and export operations, caps large audited values at about 5,000 characters, and deletes records past their retention period. Source
- The Process Mining Manifesto asks that logged events be trustworthy, complete for their scope and semantically well defined, and lists finding, merging and cleaning event data as a challenge. Source
- NIST SP 800-92 lists inconsistent log content, formats and timestamps among sources as a log management challenge and notes hosts stamp entries using their own internal clocks. Source
See if your company qualifies
A short company assessment. No data uploads are needed.