AI uses for records
Agent trajectory task boundary detection: where tasks start and end
By SourceX Editorial · Updated
10 sections
Short answer
To find task boundaries in a noisy business event log, anchor each task on the obligation it serves: start at the event that creates a request someone must act on, end at the event that verifies it was met plus a quiet period with no reopen. Treat case IDs and time gaps as hints, apply written rules for waits, reopens, merges and interleaving, and measure annotator agreement.
Agent run
Agent trajectory task
Key takeaways
- A task runs from the event that creates an obligation to the event that verifies it was met, followed by a quiet period with no reopen.
- Case IDs and inactivity gaps are candidate boundaries only; records hold several requests and real work waits for days.
- Write the rules for waits, reopens, merges, interleaving, handoffs and abandoned work before anyone annotates.
- Measure boundary agreement with segmentation metrics such as Pk or WindowDiff and outcome agreement with Cohen's kappa.
- Confirm the cut by having a second team rebuild each episode's start state and outcome from the bounded records.
How do you find where a business task starts and ends in a noisy event log?
Anchor each task on the obligation it serves, not on the clock or the record. A task starts at the event that creates an obligation someone must act on, such as a customer request, an approval request or a work order, and ends at the event that verifies the obligation was met, followed by a quiet period with no reopening. Everything between those two events that serves the same obligation belongs to the task, even if it spans records, people and days.
In practice you get there with a written annotation guide that turns that principle into rules for the messy cases (waits, reopens, interleaved requests, merged records, handoffs and abandoned work) and with a measured check that two annotators applying the guide cut the same log in the same places.
Getting the boundary right decides what an agent learns. Cut a task too early and the trajectory ends before the outcome, so there is nothing to grade. Cut it too late and the episode swallows a second request, teaching the agent that two unrelated jobs are one.
Why isn't a case ID or a time gap a task boundary?
A case ID marks where a system stores work, and a time gap marks when people paused. Neither reliably marks where one piece of business work starts and stops, so treat both as candidate boundaries to confirm rather than as answers.
Process mining logs make the case ID central. The XES standard definition says the concept:name attribute of a trace usually stores the case ID, while on an event it names the executed activity; the pm4py library uses case:concept:name as its default case column and time:timestamp, org:resource and lifecycle:transition for time, actor and lifecycle phase. That works when one case equals one task. Business systems break the rule constantly: one support ticket collects a second question, one order is fulfilled through three shipments, a duplicate ticket is merged into an older one.
Some work has no single case at all. The OCEL 2.0 specification exists because events can relate to several objects at once, with qualifiers describing each object's role, so an event such as packing a shipment belongs to an order, several items and a package simultaneously. Picking one object as the case splits or duplicates the task.
Inactivity rules come from web analytics, where Google Analytics ends a session after 30 minutes without activity by default. Business tasks routinely wait days on a customer, a supplier or an approver, so a fixed gap rule will cut most real tasks in the middle of a legitimate wait.
What start and end evidence does each kind of system leave behind?
Each system leaves different evidence of an obligation opening and closing. List it per source before annotating, because the guide's rules have to name concrete events, not ideas.
- Zendesk's documentation describes audits as a read-only history of all updates to a ticket, each audit representing one update that can contain several events, such as a field change with its previous and new value. That granularity is what lets an annotator see where a second request entered a ticket.
| Source | Start evidence | End evidence | Noise to expect |
|---|---|---|---|
| Help desk audit trail (for example Zendesk ticket audits) | Ticket creation audit or a new requester comment introducing a new request | Status change to solved, confirmed by no reopen within the quiet period | Automated notifications, tag changes and macro updates in the same audits |
| Order and invoice records in an ERP | Order or invoice created against a customer or supplier | Payment applied or credit issued that closes the balance | Partial shipments, partial payments and batch postings |
| Field service work orders | Work order created or dispatched | Job completed with sign-off and any return visit closed | Reschedules, parts holds and technician swaps |
| Shared inbox or email threads | First message asking for something not already open | Reply confirming the request is done, or a linked record closing | Forwards, CCs and one thread carrying several requests |
| Desktop or task-mining captures | Opening the record that the request concerns | Saving or submitting the change that fulfills it | Window switching, lookups for other work and idle time |
Which rules should the task boundary annotation guide contain?
This guide is the asset. Give it to annotators with the source list above and a set of worked examples from your own systems, and version it: every rule change gets a date, and earlier annotations are rechecked against the new rule. The examples in the last column are illustrative.
| Rule | Definition | How to annotate | Illustrative example |
|---|---|---|---|
| Start | First event that creates an obligation someone must act on | Mark the event as task start and record the obligation in one sentence | Customer writes asking to change a delivery address |
| End | Event that verifies the obligation was met, plus a quiet period with no reopen | Mark the verifying event as task end and record the outcome | Address updated and confirmation sent; nothing further within the quiet period |
| Wait | Pause while waiting on the customer, a supplier or an approver | Keep inside the task and mark the wait span; never end a task on a wait | Ticket pending for four days awaiting a photo |
| Reopen | Same obligation reopened because the outcome failed | Continue the original task and mark a reopen event | Customer replies that the address change did not apply |
| New request in old record | Different obligation raised in an existing record | Start a new task at that event, linked to the same record | Same ticket later asks about a refund |
| Merge | Several records carry one obligation | One task spanning the records, with every record ID listed | Duplicate ticket merged into the original |
| Interleaving | One person works several tasks at once | Assign each event to its obligation; events can be skipped but not shared | Agent alternates between two customers' orders |
| Handoff | Work moves to another person or team | Keep one task and record the actor change | Escalation from tier one to billing |
| Abandoned | Obligation dropped without a verified outcome | End at the last action with outcome marked abandoned | Requester stops replying and the ticket auto-closes |
| System noise | Automated events not caused by the obligation | Exclude from the task and keep in a noise list | Nightly tag sweep or a satisfaction survey email |
Which fields must every bounded episode carry?
A boundary is only useful if the episode inside it can be replayed. Every bounded task should carry the fields below, mapped to whatever your source calls them; the process mining keys in brackets are the pm4py defaults where one exists.
At a glance
Which fields must every bounded episode carry
- Task ID, plus every source record ID the task spans (case:concept:name where a single case applies).
- Step IDs in order, each with the activity name (concept:name) and timestamp (time:timestamp).
- Actor for every step (org:resource) and the actor's role, with personal identifiers pseudonymized.
- Lifecycle phase where the source records it (lifecycle:transition), so waits and resumptions are visible.
- Observation: what the actor could see at that step, such as the record state or the message received.
- Action and the tool response it produced, including errors and retries.
- Links to intermediate artifacts such as attachments, drafts, quotes or generated documents.
- Verified outcome with the evidence event that verifies it, and the quiet-period rule applied.
- Boundary rationale: which guide rule set the start and the end, and the guide version.
- Replay instructions: snapshot version and the order in which a second team should load the records.
Do two annotators draw the same boundaries?
Measure agreement before scaling annotation. Have two annotators bound the same sample of logs independently, score how closely their boundaries match, adjudicate disagreements, and revise the guide where a rule caused them.
Boundary placement is a segmentation problem, and text segmentation already has metrics for it. NLTK implements Pk, from Beeferman, Berger and Lafferty, and WindowDiff, from Pevzner and Hearst, which slides a window across two segmentations and counts the windows where the number of boundaries differs. Both give partial credit for near misses, which suits event logs where a boundary one event off is a smaller error than a missing task.
Score outcome labels separately. Cohen's kappa, implemented in scikit-learn as a measure of inter-annotator agreement, corrects observed agreement for the agreement expected by chance, which matters when most tasks share one label such as resolved.
Do not set a universal pass mark. Decide in advance what agreement level is acceptable for your use, report it with the sample size, and treat any rule that produces most of the disagreements as the next guide revision.
The rebuild check: can another team replay the episode from your cut?
Hand a second team the bounded tasks, the field list and the replay instructions, without the annotators' notes, and ask them to reconstruct the state at the start of each task and the outcome at its end. If they reach the same state and outcome, the boundary and the links are complete.
Where they differ, the cause is usually one of three things. The start missed context the actor already had, such as an earlier message in the same record. The end relied on evidence outside the bounded records, such as a payment posted in another system. Or an artifact link was missing. Fix the guide or the field mapping, not just the one task.
How does the guide split an illustrative ticket that holds three tasks?
This example is illustrative; the ticket and its events are invented to show the guide in use. One help desk ticket holds 31 audit events over nine days. Under a one-ticket-one-task rule it would be a single trajectory.
Applying the guide gives three tasks. Task one starts with the customer's request to change a delivery address and ends when the address is updated and a confirmation is sent; a two-day wait for an order number stays inside it. Task two starts when the customer asks, in the same ticket, for a refund on a damaged item; it is handed from the front-line agent to billing and ends when the refund is issued. Task three starts with a question about a loyalty discount and ends abandoned when the customer stops replying and the ticket auto-closes.
Six events are system noise: survey emails, automatic tag updates and a reminder macro. Two annotators agreed on the start and end of tasks one and three and disagreed on task two, where one ended it at the refund approval and the other at the refund posting. The guide was revised to say that the end is the event that changes the customer's balance, not the approval.
What about processes with no natural end, and when should the scope narrow?
Some work has no verified end: account management, monitoring, long-running projects. For those, bound tasks around discrete obligations inside the stream, such as one renewal or one alert investigation, and label anything cut by the edge of the export as truncated rather than ended, so it is never graded as a failure.
Stop and narrow the scope when the outcome cannot be verified from the records you have, for example when completion happens in a system outside the export, or when agreement stays poor after two guide revisions. Both are signals that the log cannot support task-level trajectories for that workflow yet, and that a task expert who runs the process should redefine the obligation before more annotation is paid for.
How do you ask for boundary-ready records from licensed business systems?
Boundaries are easiest to draw when the export includes audit histories, linked records and outcome events rather than only final states, which is a scoping question for the dataset request itself. An AI team can list audit histories and outcome events as required scope on SourceX's request-data form, next to volume, language, format, timing and licensing needs, and the matching records are then found at U.S. businesses for that one request.
Actor fields will reach annotators in de-identified form, because SourceX de-identifies the licensed records before delivery through approved processing routes, so plan the pseudonymized org:resource values into the guide from the start.
Frequently asked questions
What is task boundary detection in an event log?
It is deciding which events belong to one unit of business work, from the event that creates an obligation to the event that verifies it was met. It matters for agent training because each bounded task becomes one trajectory with a gradable outcome.
Can I use the case ID as the task boundary?
Only after checking. A case ID marks where a system stores work, and records often hold several requests or split one request across several records. Use the case ID as a candidate, then apply rules for new requests, merges and reopens.
Should a long wait end a task?
No. Waiting on a customer, supplier or approver is part of the task, so mark the wait span and keep going. Fixed inactivity timeouts suit web sessions, not business work that pauses for days.
How do you measure annotator agreement on task boundaries?
Double-annotate a sample and compare boundaries with segmentation metrics such as Pk or WindowDiff, which give partial credit for near misses. Compare outcome labels with Cohen's kappa, which corrects for chance agreement.
What if the outcome happens in another system?
Then the end cannot be verified from the log alone. Either add the other system's outcome events to the export, linked by a shared ID, or label those tasks unverified and keep them out of graded training and evaluation.
Sources
- In XES, the Concept extension's name attribute usually stores the case ID on a trace and the executed activity's name on an event; the Lifecycle extension records the lifecycle phase. Source
- pm4py uses case:concept:name as its case identifier column and case: as the prefix for case attributes. Source
- pm4py's default XES keys are concept:name for the activity, time:timestamp, org:resource and lifecycle:transition. Source
- In OCEL 2.0, unlike traditional event logs, events can be related to multiple objects, and those relationships can be qualified. Source
- Google Analytics ends a session after 30 minutes of user inactivity by default, and the timeout can be adjusted. Source
- Zendesk ticket audits are a read-only history of all updates to a ticket; each audit is a single update that can contain one or more events. Source
- NLTK implements the WindowDiff (Pevzner and Hearst) and Pk (Beeferman, Berger and Lafferty) segmentation metrics. Source
- scikit-learn's cohen_kappa_score computes Cohen's kappa, a statistic that measures inter-annotator agreement corrected for expected chance agreement. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.