Skip to content

Systems and records

Support outcome measurement lag: how long to wait before labeling a resolution

By SourceX Editorial · Updated

9 sections

Short answer

Assign a resolution label only after the case has passed a follow-up horizon set for its issue type, long enough for the usual failure signals to appear: a reopen, a linked follow-up ticket, an escalation, a reversal or a repeat contact. Cases still inside their horizon at export are censored, not successes. Horizons come from your own reopen and follow-up timing, not from the helpdesk's close timer.

Record timeline

Support outcome measurement

This page covers support outcome measurement lag: how long to wait before labeling a resolution.Illustrative

Key takeaways

  • Solved and closed are workflow states; neither proves the customer's problem stayed fixed.
  • Helpdesk close timers and follow-up tickets can move late failures onto a new ticket ID.
  • Set a follow-up horizon per issue type from your own data on when failures show up.
  • Cases whose horizon has not elapsed at export are censored and should not be labeled resolved.
  • Link follow-ups to the original case before labeling, using keys that survive de-identification.
  • Show the share of censored cases and the horizon used, so a buyer can judge label maturity.

How long should support outcomes mature before resolution labels are assigned?

Long enough for the failures that usually follow that kind of issue to appear, measured from the solve time, and set separately for each issue type. A password reset proves itself or fails within hours. A billing correction is not proven until the next invoice goes out without the error. A bug workaround may hold until the next release changes the behavior again. One global waiting period either labels slow-failing cases too early or holds fast ones back for no reason.

The lag between solving a ticket and knowing whether the solution worked is the support outcome measurement lag. Ignoring it is the most common way an outcome-labeled support dataset overstates success: recent tickets look resolved because nothing has gone wrong yet, not because the fix worked.

The decision rule that follows is simple to state. A case gets a mature outcome label only when its issue type's follow-up horizon has fully elapsed before the data was extracted, and every failure signal inside that horizon has been checked, including signals recorded on other tickets. Otherwise the case keeps a provisional or censored state.

Why solved and closed are not the same as resolved

Helpdesk statuses describe the agent's workflow, not the customer's outcome, and their timers can move late failures out of sight. In Zendesk, a default automation closes a solved ticket four days after it is solved, and an admin can extend that window up to 28 days; a system rule closes tickets 28 days after they are solved. Once a ticket is closed, a customer reply does not reopen it. Zendesk creates a follow-up ticket that references the closed one.

Freshdesk behaves similarly with different timers. A reply within the configured threading interval reopens the original ticket, while a reply after it creates a new ticket. Closed tickets that stay inactive for 120 consecutive days are archived, and replies to archived tickets also create new tickets.

The consequence for labeling is direct. A reopen counter on the original ticket only sees failures that arrive before the ticket closes. Failures that arrive later show up as new tickets, often with a different subject line and agent. If the dataset reads the reopen counter alone, the cases that failed slowly look like clean first-contact resolutions.

Agent behavior adds a second layer. Some teams solve a ticket as soon as they reply, expecting the customer to reopen it if needed; others wait for confirmation. Some merge duplicates into an older ticket, which can carry the failure signal away from the case you are labeling. Before measuring anything, write down how your team uses solve, pending and merge, and whether that changed over the period you plan to export.

  • Reopen before close: visible on the original ticket's reopen count.
  • Reply after close or after the threading interval: a new or follow-up ticket that must be linked back.
  • Repeat contact on a new channel, such as chat after an email ticket: a new ticket with no link unless you build one.
  • Downstream evidence outside the helpdesk, such as a refund, a credit note or a reopened bug: invisible to ticket counters entirely.

Cases still inside their window are censored, not successes

A case whose follow-up horizon has not elapsed when the data is extracted has an unknown outcome, and it should be stored that way. Reliability engineering has a name for this: censoring. The NIST engineering statistics handbook describes a test that runs for a fixed time, records the failures observed, and treats the remaining units as survivors of the test period, not as units that will never fail.

Support data has the same structure. Every export has a cutoff date, and every case solved shortly before it has had less time to fail. If those cases are labeled resolved, the most recent months of the dataset look better than older months, and a model trained on it learns that recent-style resolutions work better than they do.

There are two honest ways to handle censored cases. Drop them from the outcome-labeled set and say how many were dropped, or keep them with a censored state and the number of days observed, so an analyst can use methods built for partial observation. Both are acceptable. Labeling them resolved is not.

The follow-up horizon and outcome maturity worksheet

Fill one row per issue type. The horizons shown are illustrative starting points to replace with your own measurements: the time by which most reopens and linked follow-ups for that issue type have already arrived in your history. Add rows for the issue types your queue actually uses.

To measure a horizon, take cases of one issue type that were solved long enough ago to have a full history, find every reopen and linked follow-up, and plot the days from solve to each failure. The horizon is the point after which few new failures appear. Where the curve never flattens, the issue type is probably two types sharing one label, and splitting it gives each part a usable horizon.

Horizons tied to an external event, such as the next invoice, the next release or the next renewal, should be stored as a rule rather than a fixed number of days. The case is mature when that event has happened plus the stated margin, which is different for every account.

The follow-up horizon and outcome maturity worksheet
Issue typeEarliest evidence the fix workedIllustrative horizon after solveFailure signals to check inside the horizonState if horizon not elapsed
Access or password resetSuccessful sign-in or no repeat contact3 daysReopen, follow-up ticket on the same accountProvisional
Configuration or how-to questionCustomer completes the task the question was about14 daysRepeat question on the same feature, follow-up ticketProvisional
Billing correctionNext invoice issued without the errorOne full billing cycle plus 7 daysCredit note, dispute, repeat billing ticketCensored
Bug with workaroundNo repeat report until the permanent fix shipsUntil fix release plus 14 daysLinked bug reopened, follow-up on the same symptomCensored
Bug fixed in a releaseCustomer on the fixed version without recurrence30 days after the customer upgradesLinked bug reopened, regression ticketCensored
Shipping or order problemDelivery confirmed or replacement receivedCarrier delivery window plus 7 daysSecond shipment, return, refundProvisional
Cancellation save or retention offerAccount still active at the next renewalThrough the next renewal dateCancellation, downgrade, churn flagCensored

The input dictionary: what each case needs before it can be labeled

A maturity decision needs a small set of fields per case, most of which the helpdesk already stores but not always in one place. Zendesk's ticket metrics, for example, include a reopen count that tracks moves from solved back to open or pending, the solve timestamp and full resolution time. Jira Service Management restarts its time to resolution SLA when an issue is reopened, and Atlassian recommends clearing the Resolution field on the reopen transition so the record reflects the reopened state.

At a glance

The input dictionary: what each case needs before it can be labeled

Collect the fields below before applying the rule. The extraction date matters as much as any ticket field, because it defines how long each case has been observed.

  • case_key: a stable identifier that groups the original ticket with its follow-ups, reopens and linked bugs.
  • issue_type: the category that selects the horizon, taken from a dated taxonomy so changes in categories are visible.
  • solved_at: the final solve time of the original ticket, and each earlier solve if it was reopened.
  • reopen_count and reopen times: from the platform's metrics or audit history.
  • linked_followups: follow-up ticket IDs, and repeat contacts matched by requester or organization and topic within the horizon.
  • downstream_events: refunds, credits, bug reopenings, cancellations or renewals that bear on the outcome, with dates.
  • extracted_at: when the data was pulled; observed_days equals extracted_at minus solved_at.
  • outcome_state: mature resolved, mature failed, provisional or censored, set by the rule, never typed by hand.

What CSAT can and cannot tell you about whether the fix held

CSAT measures how the customer felt soon after the solve, which is useful but not the same as whether the problem stayed fixed. Zendesk's default automation sends the satisfaction survey 24 hours after a ticket is solved, and admins can change that delay. A customer who rates the interaction well on day one can still come back on day ten when the workaround stops working.

Treat CSAT as a separate label with its own timestamp, not as a substitute for a mature outcome. Record when the survey was sent and answered relative to the solve time, and do not let a positive rating override a later follow-up ticket. Nonresponse is also information: response rates vary by issue type and customer segment, so a dataset that keeps only rated tickets can tilt toward the cases customers felt strongly about.

A practical combination is to keep three outcome fields side by side: the survey rating with its timestamp, the mature outcome state from the horizon rule, and the reason for any failure signal found. Disagreements between them are informative. A well-rated case that later fails points to a fix that only looked right; a poorly rated case that holds points to a process or tone problem rather than a wrong answer.

Illustrative scenario: a billing queue labeled too early

This scenario is illustrative and uses invented counts. A software company exports two years of billing tickets for an outcome-labeled dataset, with an extraction date at the end of a month. Its invoices go out monthly, so the worksheet horizon for billing corrections is one cycle plus seven days, roughly 38 days.

Of 1,200 billing tickets solved in the final 38 days before extraction, the first-pass labeling marks 1,140 as resolved because their reopen count is zero. Applying the rule, all 1,200 are censored: none has been observed through a full invoice cycle. In older months, where the horizon had elapsed, linking follow-up tickets by organization and topic moved about 9% of zero-reopen cases to mature failed, because the error recurred on the next invoice and the customer opened a new ticket after the original had closed.

The illustrative result is two corrections at once. The censored month is no longer counted as a success, and the older months show a lower but more honest resolution rate. A model trained on the corrected labels sees which billing fixes recurred, which is the pattern a support agent most needs to learn.

Which assumptions change the horizon, and how to show the uncertainty

Three assumptions move the answer more than any other, and each should be stated next to the dataset. Platform timers set where late failures land; if the close window or threading interval changed during the period, follow-up linkage must cover both regimes. Linkage quality decides whether follow-ups are found at all; matching by requester alone misses colleagues who report the same problem, and matching by organization alone can over-link unrelated issues. Product and billing cadence set how long a fix needs to prove itself, so a change from monthly to annual invoicing, or a slower release train, lengthens the horizon.

Show uncertainty in plain counts rather than a single resolution rate. For each issue type and month, report how many cases are mature resolved, mature failed, provisional and censored, the horizon applied and the linkage rule used. A reader can then see whether a low failure rate reflects good fixes or short observation.

Checklist · 4 items

  1. Measure the horizon from your history: the number of days after solve by which most reopens and linked follow-ups for that type have arrived.
  2. Recheck horizons when the close automation, threading interval, release cadence or billing cycle changes.
  3. Link follow-ups with pseudonymous keys created before de-identification, so linkage survives the privacy pass.
  4. Ask for review when an issue type has too few cases to estimate a horizon, or when failures arrive long after any reasonable window.

0 of 4 ticked. Tick each item as you go.

Mature outcomes in a licensing conversation

Outcome-labeled support histories are more useful to AI developers when the labels have had time to mature and the censored share is visible. As a general guide, buyers look for years of consistent history in one system, and a dataset with a stated horizon per issue type, linked follow-ups and a censored state for recent cases answers the first questions they ask about label quality.

Support teams that decide to license selected histories can do it through SourceX, which runs qualification, rights review, contracting, preparation, delivery and payment. Eligibility is checked with a brief company assessment, and no ticket export is needed for it.

Frequently asked questions

Does a closed ticket mean the customer's problem was resolved?

No. Closed is a workflow state set by timers and agent actions. In Zendesk and Freshdesk, a reply after a ticket closes or after the threading interval creates a new ticket, so a failure can arrive on a different ticket ID while the original still looks resolved.

How do I choose a follow-up horizon for a support issue type?

Look at your own history for that issue type and find the number of days after solve by which most reopens and linked follow-ups have already arrived. Use that as the horizon, and recheck it when release cadence, billing cycles or helpdesk timers change.

What should happen to tickets solved just before the data export?

Treat them as censored if their horizon has not elapsed. Either leave them out of the outcome-labeled set and report how many were removed, or keep them with a censored state and the days observed. Do not label them resolved.

Is CSAT a good enough resolution label on its own?

No. A satisfaction survey is usually sent soon after the solve, so it records how the customer felt before the fix had time to fail. Keep CSAT as a separate, timestamped label alongside a mature outcome state.

How can follow-up tickets be linked after customer data is de-identified?

Create pseudonymous keys for requester and organization before the privacy pass, and use them to group the original ticket with its follow-ups. The keys preserve linkage without carrying names or contact details into the dataset.

Sources

  • Zendesk's default automation closes a ticket four days after it is solved, and the window can be extended up to 28 days. Source
  • Zendesk closes tickets 28 days after they are solved, and replies to a closed ticket create a follow-up ticket instead of reopening it. Source
  • Zendesk ticket metrics include the number of reopens, the solve time and full resolution time. Source
  • Zendesk's default CSAT survey is sent 24 hours after a ticket is solved, and the delay can be customized. Source
  • In Freshdesk, a reply within the threading interval reopens the ticket, and a reply after it creates a new ticket. Source
  • Freshdesk archives closed tickets after 120 consecutive inactive days, and replies to archived tickets create new tickets. Source
  • Jira Service Management restarts the time to resolution SLA on reopen, and Atlassian recommends clearing the Resolution field on the reopen transition. Source
  • In censored observation, units that have not failed by the end of the observation period are survivors of that period, not proven never to fail. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies