Skip to content

Systems and records

Cross-system event join confidence: do these events belong to one case?

By SourceX Editorial · Updated

10 sections

Short answer

Measure confidence link by link. Tag each event-to-case link with how it was made: a system key, a typed reference or a probabilistic match. Score probabilistic links from grouped, frequency-aware signals, split them into accept, review and reject bands with two thresholds, and review the middle band plus random samples from the others. Report precision per band with an interval and grade each case by its weakest link.

Call to transcript

Cross system event join

This page covers cross-system event join confidence: do these events belong to one case.Illustrative

Key takeaways

  • Record how every link was made; a stored key and a time-window guess are not the same evidence.
  • Correlated signals such as email, name and phone from one contact record count once, not three times.
  • Common values like switchboard numbers need frequency adjustment or review.
  • Sample the accept and reject bands too, or false links and missed links stay invisible.
  • Grade each case by its weakest link and keep unresolved cases, labelled, so totals reconcile.

How do you measure confidence that events belong to the same business case?

Measure it at the level of each link, then roll it up. Every event-to-case assignment gets an evidence tier: a foreign key both systems stored, a reference one side typed in, or a probabilistic match on shared attributes and timing. Probabilistic links get a score built from independent signals, two thresholds split them into accept, review and reject bands, and people review the middle band plus random samples from the other two. The reviewed samples give an estimated precision per band with an interval, and each case inherits a label from its weakest link.

That gives three numbers a reader can check: how each link was made, how often links made that way are right, and how many cases rest on links nobody verified. A single join rate cannot answer any of them. Two systems can join almost every event and still put a meaningful share in the wrong case if the join used a phone number shared by a reception desk.

This page sets out a join confidence and manual review protocol for teams turning raw events from CRM, help desk, ERP and telephony systems into business cases or task episodes. It is a method page: the example is illustrative, and no measured results are reported.

Before scoring anything, classify every link by the kind of evidence that created it. The tier sets the default treatment and is stored on the link row, so a downstream user can filter on it without rerunning the pipeline.

From stored foreign key to time-window guess: ranking how each link was made
TierEvidence behind the linkBusiness system exampleDefault treatment
K1 system keyA foreign key written by the system on both sidesInvoice line carrying the sales order ID created by the ERPAccept; verify referential integrity and counts only
K2 entered referenceA reference a person typed into one recordHelp desk ticket field holding an order number keyed by an agentAccept after a format and existence check; sample for typos
P1 shared identifier plus timeSame email, phone or account within a time windowPhone call matched to a ticket by caller number within two hoursScore; thresholds decide accept, review or reject
P2 content matchAmount, date, subject or product agree with no identifierPayment matched to an invoice by amount and dateScore with a stricter threshold; review more
S1 sequence onlyOrdering inferred from timestamps aloneA status change assumed to follow the last email in the threadDo not join; keep as unlinked context or label as inferred

Turning several weak signals into one score without double counting

Use the Fellegi-Sunter model, the standard framework for probabilistic record linkage, or a tool that implements it. Splink's documentation describes its three parameters: a prior probability that any two records match, an m probability that a given comparison outcome occurs when the records truly match, and a u probability that it occurs when they do not. The match weight for each comparison is the log of m over u, the weights add up with the prior's weight, and the total converts to a match probability used to set thresholds.

The additivity rests on an assumption the same documentation states plainly: comparisons are treated as independent. Business event data breaks it easily. A contact's email address, name and phone number usually come from one CRM record, so agreement on all three is closer to one piece of evidence than three. Group correlated fields into a single comparison with levels, such as exact email or exact phone and name, rather than adding separate weights.

Common values need their own handling. Splink notes that the basic model does not account for skew in how values are distributed, so the u probability is too low for common values. In operational data, the common values are switchboard numbers, shared inboxes such as accounts payable addresses, and placeholder dates. Adjust for term frequency or exclude those values from scoring and send their candidate links to review.

Which timing and identifier faults make a time-window join unsafe?

Six faults recur: clock offset, missing time zones, mistyped identifiers, duplicate events, unrecorded transitions and events that serve several cases. Each changes the answer in a different way, so test for each before trusting time-window links. The protocol treats them as named risks with a check and a consequence.

At a glance

Which timing and identifier faults make a time-window join unsafe

  • Clock offset between systems: NTP's specification separates the time offset between clocks from the frequency offset that makes it grow, so a telephony server and a help desk can disagree by a stable amount that slowly changes. Estimate the offset from K1 or K2 links whose true order is known, and widen or shift windows per system pair.
  • Missing or wrong time zone: RFC 9557 updates RFC 3339 so that Z marks a time whose UTC value is known but whose local offset is not. Exports that write local time with no offset at all are worse; convert them only when the source system's zone setting is documented, and flag them otherwise.
  • Identifier mismatch: typed references carry transposed digits, prefixes and old numbering schemes. Check every K2 reference exists in the target system, and send near-misses to review rather than fuzzy-accepting them.
  • Retries and duplicate events: integrations resend messages, and webhooks fire twice. Deduplicate on source event ID before linking, or two copies of one event will add false weight to a case.
  • Unrecorded transitions: work done by phone or in a spreadsheet leaves no event. Label the gap instead of interpolating a status change, and count cases with known gaps separately.
  • One event, several cases: the OCEL 2.0 specification lets an event relate to several objects with qualifiers naming each object's role. A shipment covering two orders belongs to both; forcing it into one case is an error, not a confidence problem.

The join confidence and manual review protocol

Run the protocol per link type, such as call to ticket or payment to invoice, because error rates differ by link type far more than by case. Version every step so the confidence figures can be reproduced after the data or the rules change.

  • Step 1, define the case object and the link types that attach events to it; write down which systems each link type joins.
  • Step 2, assign every link an evidence tier from the ranking above, and record the blocking rule used to generate candidate pairs, because blocking decides which true links were never compared.
  • Step 3, score P1 and P2 candidates with grouped, frequency-aware comparisons; store the score, the comparison levels hit and the rule version on the link row.
  • Step 4, set an upper and a lower threshold from the errors you can tolerate, following the three-way rule described in Census Bureau record linkage research: above the upper threshold is a link, below the lower is a non-link, between them is held for clerical review.
  • Step 5, review every candidate in the middle band where capacity allows; if it does not, review a random sample and leave the rest labelled unreviewed.
  • Step 6, draw random samples from the accept band and the reject band as well, so that false links and missed links are both estimated, not only the doubtful ones.
  • Step 7, have a second reviewer label a subset blind, record disagreements and refine the review guide before trusting the labels.
  • Step 8, compute precision for each band from the reviewed sample with a binomial interval, such as the Wilson interval NIST's handbook describes, and report the sample size beside it.
  • Step 9, roll link labels up to a case label, and reconcile case and event counts back to source system totals.
  • Step 10, publish the thresholds, sample sizes, intervals and disagreement notes with the dataset version.

Review labels and the evidence a reviewer must cite

Reviewers label links, not cases, and each label requires a cited piece of evidence from the source records. A label without evidence is treated as unreviewed.

Review labels and the evidence a reviewer must cite
Review labelMeaningEvidence the reviewer citesEffect on the case
Same caseThe event belongs to this caseA reference, content or participant that ties both records to one requestLink counts as verified
Different case, same customerRight customer, wrong requestA distinct request, order or ticket the event refers toLink removed; event reassigned or left unlinked
Different customerThe shared attribute was coincidentalConflicting names, accounts or contentLink removed; value considered for frequency adjustment
Several casesThe event legitimately serves more than one caseContent naming each caseEvent related to each case with a role qualifier
Cannot tellEvidence insufficient either wayWhat was checked and what was missingLink kept as unresolved; case cannot rise above the lowest label

A case is only as reliable as its weakest link that matters for the use. Grade A cases contain only K1 and K2 links that passed existence checks. Grade B cases include scored links from the accept band of a link type whose sampled precision met the threshold set in step 4. Grade C cases include at least one link verified by review and none unresolved. Grade U cases contain an unresolved or unreviewed middle-band link, or a known gap.

Grades let a user choose. Training data for an agent that must act on the right record can use A and B; analysis of workflow timing may accept C; U cases stay in the release with their grade so totals still reconcile. Recall needs its own figure: Splink's evaluation guide points out that precision, recall and related metrics all depend on ground truth from clerical labels, and the reject-band sample from step 6 is the only source of that truth for missed links.

Following one order through a ticket and two phone calls (illustrative)

This example is invented to show the protocol; no records or measurements from a real company are used. A distributor wants cases that start with a customer order in the ERP and include every support ticket and call about it.

The ticket links to the order through an order number the agent typed into a custom field: tier K2. The existence check fails because two digits are transposed, but the corrected number exists, belongs to the same customer account and was shipped two days before the ticket. The link goes to review, and the reviewer labels it same case, citing the ticket text that names the product on the order.

The first call matches the ticket's contact phone number twelve minutes before the ticket was opened: tier P1, high score, accept band. Comparing K1-linked calls and tickets earlier in the year showed the telephony server's clock running a few minutes ahead of the help desk, so the window had already been shifted to compensate. The second call matches the same number three days later, but that number is the customer's main switchboard, which appears on hundreds of accounts' calls. With a frequency adjustment the score drops into the middle band, and the reviewer labels it different case, same customer: the caller asked about another order.

Exception: in the random sample from the accept band, a reviewer finds a call linked to the wrong customer entirely because two small businesses share an office phone line. That single error does not decide anything, but it is recorded, the shared number is added to the frequency table, and the accept-band precision is recomputed on the next release. The case ends as grade C: verified, but dependent on review rather than keys.

When reviewers disagree or the keys run out

Stop and narrow the scope when blind reviewers disagree on a large share of middle-band links of one type, when no K1 or K2 links exist to estimate clock offset, or when the reject-band sample shows that blocking is excluding true links. Those are signs the evidence cannot support the link type, and a smaller dataset built only on keyed links is the honest result.

Linking systems also concentrates personal information. Joining calls, tickets and orders on phone numbers and email addresses builds a richer profile of each person than any one system holds, so plan de-identification of the linked case, not just each source table, and bring in privacy review before linking data about individuals at scale.

Graded case data in a licensing review

Linked cases with published confidence grades are easier to evaluate than raw exports, because a reviewer can see how each case was assembled and which part rests on judgment. A company that wants to license graded cases can rely on SourceX for qualification, rights review, contracting, preparation and delivery to AI developers. The usual bar is several years of steady history, two at the least, and record counts in the tens of thousands within one system, with exceptions for rare work. Whether a company clears it is settled in a short assessment, before any linked case or raw export is shared.

Frequently asked questions

What is a good match probability threshold for joining events?

There is no general number. Set the upper and lower thresholds from the error rates the use can tolerate, then check them with reviewed samples from each band. Different link types, such as call to ticket and payment to invoice, usually need different thresholds.

How many links should be manually reviewed?

Review every middle-band link if capacity allows, plus random samples from the accept and reject bands. Size the samples so the interval around each band's estimated precision is narrow enough for the decision, and report the sample size with the estimate.

Is a high join rate enough to show the links are right?

No. A join rate shows how many events found a case, not whether they found the right one. Wrong links on common values such as switchboard numbers can produce a high join rate with poor precision, which only a reviewed sample reveals.

How do you correct for clock differences between two systems?

Use links whose order is known from keys, such as an order and its invoice, to estimate the offset between each pair of systems. Shift or widen time windows per system pair, and recheck the offset over time because it can drift.

Should unresolved links be dropped from the dataset?

Keep them, labelled. Dropping them makes case counts stop reconciling to source totals and hides how much of the dataset rests on judgment. Users can filter by case grade instead.

Sources

  • The Fellegi-Sunter model uses a prior, m and u probabilities and additive match weights, and assumes comparisons are independent. Source
  • The basic Fellegi-Sunter model does not account for skewed value distributions, so u probabilities are too low for common values. Source
  • Linkage precision and recall depend on ground truth, generally created by clerical labelling. Source
  • Census Bureau record linkage research describes a three-way rule: links above an upper threshold, non-links below a lower one, and clerical review between them. Source
  • NIST's handbook describes the Wilson method for confidence intervals for proportions. Source
  • OCEL 2.0 lets an event relate to several objects, with qualifiers describing each object's role. Source
  • NTP version 4 models a clock's time offset separately from the frequency offset that changes it over time. Source
  • RFC 9557 updates RFC 3339 so that Z marks a time whose UTC value is known but local offset is not. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies