AI uses for records
Long-running case context completeness: a four-checkpoint test
By SourceX Editorial · Updated
10 sections
Short answer
Long-running case context completeness is the share of a case's life you can still see and attribute: how it began and under whose authority, every intermediate decision, every change of owner, and an end state confirmed outside the case system. Test those four checkpoints, then measure the unexplained silences between them, because long cases are the ones platforms split, archive and hide.
Agent run
Long running case context
Key takeaways
- Long cases usually fail completeness in the middle: a call, a reassignment or an archived ticket that left no trace.
- Platform limits split long cases: Gmail thread breaks, Zendesk follow-ups and archiving, and the free Slack history horizon each cut the timeline.
- Grade four checkpoints, initiation, intermediate decisions, handoffs and final observable state, then measure unexplained silences.
- A partial case still supports some uses; the first check it fails decides which.
- Join on work links, and sample by start date with open cases flagged, or long cases quietly drop out.
What does context completeness mean for a case that runs for months?
For a long-running case, context completeness means you can still see and attribute four things: how the case began and under whose authority, each intermediate decision, every change of owner, and an end state confirmed outside the case record. A case that runs for months rarely fails all four. It fails one quietly, usually in the middle, where a call, a reassignment or an archived ticket left no trace.
Length is the risk factor, not a detail. The longer a case runs, the more platform limits it crosses. Gmail breaks a conversation off into a new one when it passes 100 messages or the subject line changes. Zendesk turns a reply to a closed ticket into a separate follow-up ticket, and in most cases archives tickets 120 days after they close, removing them from views and from the customer's interaction history. Free Slack workspaces show only the last 90 days of messages and files, and Slack said that starting 26 August 2024 it would begin deleting free-workspace messages and files more than a year old.
Each limit is sensible on its own. Together they cut one long case into pieces that look like several shorter ones, so completeness has to be judged on the reassembled timeline, never on a single record.
Where platforms split a long case, and the key that rejoins it
Every split leaves a key that can rejoin the pieces, provided the export keeps it. Collect the keys before grading anything: a case judged from one fragment will fail checkpoints it actually passed, and the error runs one way, toward calling long cases incomplete.
The splits worth mapping first:
| Platform behavior | Effect on a long case | Key that rejoins it | Check before export |
|---|---|---|---|
| Gmail starts a new conversation past 100 messages or on a subject change | One negotiation shows up as two or more threads | Case numbers quoted in subjects or bodies; the same participants | Count threads per case; more than one means a join is needed |
| Zendesk opens a follow-up ticket when someone replies to a closed ticket | A dispute that resurfaces gets a new ticket and a new ID | The follow-up's reference to the closed ticket | Export the reference, not just ticket IDs |
| Zendesk archives tickets about 120 days after they close | Earlier episodes drop out of views and the customer's interaction history | Ticket IDs, which search and the API still reach | Pull archived tickets explicitly; a view-based export misses them |
| Free Slack workspaces show 90 days and delete content after a year | Early decisions made in chat may no longer exist | Dates and links quoted in later records | Record the deletion horizon as a known gap |
| Reassignment to a new owner or team | Context moves in a note, a call or not at all | Assignee-change events and transfer notes | Every owner change should carry an event and a note |
| A system migration partway through the case | History before cutover sits in a retired system or a summary field | Legacy IDs carried into the new system | Confirm legacy IDs survive in the export |
The four-checkpoint test for a long-running case
Grade each checkpoint pass, partial or fail against the evidence it requires. The checkpoints follow the case's life, so a reviewer can run them in order and stop at the first failure that rules the case out.
Initiation and final state anchor the case; decisions and handoffs fill it in. A case that fails initiation cannot be graded at all, because nobody can say what was asked or under which terms. A case that fails final state can still teach process, but it cannot be scored on outcome.
What each checkpoint asks, and what passes it:
| Checkpoint | Question it answers | Evidence that passes | Typical failure in long cases |
|---|---|---|---|
| Initiation | What was requested, by whom, under which terms in force that day? | The original request, the requester's role and the contract or policy version then in force | The first message sits in a thread that split off months earlier |
| Intermediate decisions | What was decided at each turn, by whom and why? | A dated record for every decision that later records rely on | A later note cites a call or meeting that left no record |
| Handoffs | Who owned the case at each point, and what moved with it? | Assignment events plus a transfer note for each owner change | Ownership changes with no note, so the next owner starts over |
| Final observable state | What actually happened, confirmed outside the case system? | A ledger entry, shipment, return receipt or signed document matching the stated outcome | The status says resolved and nothing downstream shows it |
Measuring the silences between checkpoints
A long case also needs a silence check: the longest stretch with no event that the record does not explain. Some quiet is normal. A case waiting on a customer, a part or a regulator should show that wait as a status, a scheduled follow-up or a dated record in another system.
Count an interval as explained when a waiting status with a reason, a scheduled date or a linked external record covers it; everything else is unexplained. Report the longest unexplained silence per case and the number of silences longer than the workflow's normal wait, with that normal wait set from closed cases of the same type before grading starts, so the threshold cannot drift toward whatever the sample happens to show.
Unexplained silences are where missing decisions tend to hide. They are the first place to look for a phone call, a meeting or a chat that never reached the case record, and the first place a model trained on the case would learn the wrong lesson: that nothing happened.
A seven-month warranty claim as an annotated task packet (illustrative)
The packet below rebuilds an invented claim in which a distributor's customer reports a failed compressor on a unit still under warranty. It touches email, a help desk, field service, engineering and the ERP, and it runs about seven months from first report to credit.
Keep means the record enters the packet in full; drop means it stays out because it fills no checkpoint and explains no silence. Note what the packet does not need: the customer's full order history, every email in the thread and the marketing contact log.
The timeline, with each record's checkpoint and verdict:
| Day | Event and source | Checkpoint | Packet verdict |
|---|---|---|---|
| 0 | Customer email reports the failure with serial number and purchase date | Initiation | Keep, with the warranty terms version in force that day |
| 2 | Help desk ticket opened from the email | Initiation | Keep the ticket ID and opening fields; the email already holds the request |
| 9 | Engineering asks for a failure analysis before any decision | Decision | Keep |
| 31 | Ticket reassigned to field service with a transfer note | Handoff | Keep the assignment event and the note |
| 64 | Internal note: parts covered, labor not, as agreed on a call | Decision | Keep and flag: the agreement itself has no record |
| 118 to 150 | Ticket solved, then closed; the customer replies and a follow-up ticket opens | Handoff | Keep both IDs and the follow-up reference |
| 203 | Credit memo for the part posted in the ERP; return receipt logged | Final state | Keep: it confirms the outcome outside the help desk |
| Various | Marketing emails and a satisfaction survey sent to the same customer | None | Drop: no checkpoint filled, no silence explained |
Completeness map for the claim, and the verdict it supports
Graded checkpoint by checkpoint, the claim passes three, is partial on one, and the silence check finds one gap. That profile decides its use: the case can grade the final coverage decision, but it cannot teach how the labor exclusion was settled, because the conversation that settled it is missing. Under the decision tree in the next section the case is split at the gap: days 0 to 31 form one complete episode, and the stretch from day 64 to the credit is graded on its outcome, with the labor decision supplied as a given input rather than used as a label.
The map, with the evidence behind each grade:
| Check | Grade | Evidence or gap | Consequence |
|---|---|---|---|
| Initiation | Pass | Original email, serial number, warranty terms version | Request and authority are clear |
| Intermediate decisions | Partial | Failure analysis request recorded; labor exclusion cites an unrecorded call | The labor decision cannot serve as a training label |
| Handoffs | Pass | Assignment with note on day 31; follow-up linked on day 150 | Ownership is traceable end to end |
| Final observable state | Pass | Credit memo and return receipt in the ERP | The outcome can be graded |
| Longest unexplained silence | Gap | Days 31 to 64: no waiting status, no linked record | Likely site of the missing call; raised with the supplier before release |
Which uses survive a partial long case?
A partial case is rarely worthless; it supports fewer uses. Run the checks in order and apply the first branch that fits.
At a glance
Which uses survive a partial long case
- Initiation fails: exclude the case. Without the request and the terms in force, no decision in it can be judged.
- Final state cannot be observed: keep the case for process tasks such as drafting the next step, never for outcome grading.
- A decision is missing: split the case at the gap and keep the complete segments as shorter episodes, each with its own start and end.
- A handoff lacks a note, or a silence stays unexplained: keep the case, but leave it out of any test of handoff quality.
- All four checkpoints pass and every long silence is explained: the case is eligible for training and for outcome evaluation.
Does joining more records make a long case more complete?
No. Completeness comes from filling checkpoints and explaining silences, and joins beyond that add noise and linkage risk. Joining every record that shares a customer pulls in unrelated orders and older disputes, and a long case attracts more of them simply because it stays open longer.
Process-mining research names the mechanics. A 2023 systematic review of object-centric process mining on arXiv notes that flattening event data that involves many objects into a single case notion leads to three problems: deficiency, convergence and divergence. Put plainly, some events fall out of every case, one event is copied into several cases, and unrelated repeats of an activity pile into one case as though they were a sequence. A customer-keyed join on a long case can produce all three.
The rule that follows: join on work links, such as a follow-up reference, a case number quoted in an email or an order number, and admit a record only when it fills a checkpoint or explains a silence. Personal data grows with every join, so the admission bar doubles as a privacy control.
Sampling long cases without favoring the short ones
Sample by start date and keep open cases, flagged, or the dataset will under-represent exactly the cases this test is about. Take only cases closed inside an export window and short cases fill the sample, because a case that runs longer is less likely to finish inside the window.
Reliability engineering has a name for the problem. The NIST/SEMATECH e-Handbook calls data from units still running when a test ends right censored, because their failure times beyond the test length are missing. A case still open at export is censored the same way: it can support tasks about decisions already made, but its outcome label does not exist yet, so it stays out of outcome grading and is counted rather than dropped.
Report the sample's duration distribution beside the source population's, and the share of open cases, so a buyer can see whether the long cases survived selection.
Requesting long cases a buyer can actually grade
Write the reassembly into the requirement, along with the test that proves it. Ask for follow-up and legacy-ID references, archived tickets pulled explicitly, assignment events with transfer notes, a downstream record for each final state, and a gap list naming periods that were deleted or never recorded. The acceptance test: for a sample of cases, an independent reviewer passes all four checkpoints from the delivered packet alone.
SourceX builds datasets from U.S. businesses to fit a written buyer requirement, not from an existing catalog, so checkpoint rules placed in the request decide how cases get assembled. Whether a company's long cases survived its own platforms' limits is established for each dataset, never promised up front, and nothing is delivered until the company that holds the records has approved the scope and terms and signed.
Frequently asked questions
How long does a case need to run before this test matters?
Go by the platforms rather than the calendar. A case is long-running once it can cross a limit that splits or hides it, such as a closure and follow-up cycle, an archive window or a chat history horizon, or once it changes owners. In many help desks that happens within a few months.
Should open cases go into an AI training dataset?
They can, flagged as open. An open case still shows how decisions were made up to the export date, so it can support process tasks, but its outcome is unknown and it should stay out of outcome grading until it closes.
What if a key decision was made on a phone call?
Record it as a referenced but unrecorded decision. The case can still be graded on what followed, but that decision cannot become a training label unless a dated record turns up, such as call notes or a confirming email.
Is a case complete when its status says closed?
No. A status is an assertion inside one system. Completeness needs the end state observed somewhere else, such as a posted credit, a shipment or a signed document, and it has to match what the case says happened.
How do you rejoin a case that resurfaced or split?
Follow the links the systems leave: a follow-up ticket's reference to the closed one, case numbers quoted in later emails, and legacy IDs carried through a migration. Avoid joining on customer identity alone, which drags unrelated episodes into the case.
Sources
- Gmail breaks a conversation off into a new conversation when the subject line changes or it reaches more than 100 emails. Source
- Zendesk archives most tickets 120 days after they close; archived tickets are excluded from views and from the customer's interaction history but remain reachable through search and the API. Source
- A reply to a closed Zendesk ticket creates a follow-up ticket that references the closed ticket. Source
- Free Slack workspaces can view and search the last 90 days of messages and files, and from 26 August 2024 Slack began deleting free-workspace content more than a year old. Source
- Flattening object-centric event data into a single case notion leads to deficiency, convergence and divergence problems. Source
- In reliability testing, units still running when the test ends give right-censored data, because their failure times beyond the test length are missing. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.