Skip to content

AI uses for records

Voice agent escalation timing: when to transfer a caller to a person

By SourceX Editorial · Reviewed by Noah Loul ·

9 sections

Short answer

To evaluate when a voice agent should transfer a call, have reviewers mark on the recording the earliest moment a transfer was warranted and why, then measure the seconds and turns until the agent handed off and until a person answered. Weight each delay by a severity grade fixed in advance, count missed, late and premature transfers separately, and confirm labels against audio and the case outcome.

Call to transcript

Voice agent escalation

This page covers voice agent escalation timing: when to transfer a caller to a person.Illustrative

Key takeaways

  • A binary transferred-or-not label hides late transfers that still end correctly.
  • The warranted point comes from written policy and a reviewer listening to audio, not from platform logs.
  • Platforms record decision, queue and pickup times; the protocol adds the warranted point and measures the gaps.
  • Severity grades and tolerances are fixed before scoring, and premature transfers count as failures too.
  • Labels need aligned audio, verified speakers and a mature outcome; this page reports no results.

How do you evaluate when a voice agent should transfer a call to a person?

Have trained reviewers mark, on the recording, the earliest moment a transfer became warranted and the reason, then measure how long the agent kept going after that moment and how long the caller waited for a person. Weight each delay by a severity grade fixed in advance, and count missed, late and premature transfers as separate failures.

A yes-or-no transfer label hides the problem callers actually complain about. A voice agent that transfers on every hard call looks safe in a binary score and wastes staff time; one that transfers eventually, after six failed repair attempts, also passes a binary score and leaves the caller angry. The Consumer Financial Protection Bureau's report on chatbots in consumer finance describes this second pattern as a doom loop: repetitive, unhelpful automated responses with no offramp to a human representative.

This page gives the measurement protocol. It reports no results; the timings in the example are illustrative.

Which caller signals justify a transfer now, and which justify one more try?

Transfer immediately when the caller asks for a person, when the request is outside what the agent may do, or when harm is possible; retry once or twice when the problem is recognition or phrasing. The protocol needs this written down before labeling, because the warranted moment is defined by the policy, not by the reviewer's mood.

A useful public reference point is the airline domain of the tau-bench benchmark, whose transfer tool tells the agent to transfer, with a summary of the issue, only if the user explicitly asks for a human agent or if the issue cannot be resolved with the available tools. Platform defaults show the retry side: Amazon Lex V2 routes to its fallback intent when input does not match what the bot expects, when audio is noise, when input is ambiguous, or when a slot value is not recognized after the configured number of tries.

Which caller signals justify a transfer now, and which justify one more try?
Signal on the callDefault actionEvidence a reviewer needsCommon labeling error
Caller explicitly asks for a personTransfer nowThe request in the audio, attributed to the caller channelCounting an ASR mishearing of a word like agent as a request
Request outside the agent's permitted actionsTransfer nowThe policy or tool list in force on the call dateJudging scope by today's policy, not the one in force
Safety, vulnerability or legal threatTransfer nowThe caller's words and any later case noteWaiting for the caller to repeat it
Recognition failure on one slotRetry with a rephrased promptThe repair turns and the slot values capturedTreating one mishearing as grounds to transfer
Repeated recognition failure on the same slotTransfer after the retry limitCount of repair turns on that slotResetting the count when the prompt changes
Rising frustration without a requestOffer a personInterruptions, raised voice, repeated phrasesLabeling tone alone as a transfer point

Which timestamps mark the decision, the queue and the human pickup?

Three moments follow the warranted point, and each needs its own timestamp: when the agent decided to hand off, when the call entered a queue and when a person actually answered. Contact center platforms record some of these natively; none records the warranted point, which is why reviewers add it.

In Amazon Connect, the contact record holds an initiation time, an enqueue time under the queue details and a connected-to-agent time under the agent details, all in UTC; an abandoned contact has a queue and enqueue time but no connected-to-agent time. Dialogflow CX offers a live agent handoff response that the platform itself uses only to measure which conversations were handed off, with a conversation-level flag that becomes true if any response carried it. Neither tells you when the handoff should have happened.

Which timestamps mark the decision, the queue and the human pickup?
MomentDefinitionWhere it usually comes fromWhat it measures
Warranted point (W)Earliest turn at which policy required or allowed a transferReviewer label on the recordingThe reference for every delay
Decision point (A)Turn at which the agent invoked a handoffAgent logs or a handoff response in the bot platformHow long the agent kept trying after W
Enqueue (Q)Call placed in a human queueContact record enqueue timeRouting gap between decision and queue
Human connect (H)A person answersContact record connected-to-agent timeTotal wait from W to a person
Outcome (O)Verified resolution or failureCase, refund, callback or complaint recordWhether the timing changed the result

How does the time-to-escalation and severity scoring protocol work?

Score every call on the gap between W and A in seconds and in agent turns, the gap between W and H, and the severity of what the caller needed, then classify the call into one outcome category. Fix the severity scale, the tolerance per grade and the sample before scoring anything, and write them into a pre-registration note.

Severity grades describe the cost of delay, not the difficulty of the call. Assign them from the caller's request and the later outcome record, by a reviewer who has not seen when the agent actually transferred.

  • Late: A falls after W by more than the tolerance for the call's grade.
  • Missed: W exists but the call ended without a handoff, or the caller hung up first.
  • Premature: A has no W before it and the outcome record shows the agent could have resolved the call.
  • On time: A falls within the tolerance after W.
  • Correctly kept: no W and no A, with a verified resolution.
  • Report each category by severity grade, plus the distribution of W-to-A and W-to-H gaps, with confidence intervals from resampling calls.
How does the time-to-escalation and severity scoring protocol work?
SeverityDefinitionExamplesTolerance after W
S1 lowDelay costs the caller time onlyGeneral question the agent cannot answerTeam-set, a few turns
S2 moderateDelay risks a wrong transaction or repeat callChange to an order already in fulfillmentTeam-set, fewer turns than S1
S3 highDelay risks financial loss, a missed deadline or a complaintDisputed charge, cancellation with a deadlineOne turn or none
S4 criticalDelay risks safety or a legal obligationThreat of harm, report of fraud in progressNone: transfer at W

Does the recording support the transcript, the turn times and the claimed outcome?

Only label timing from audio you can align to the transcript, and only score outcomes that have had time to mature. A transcript with misattributed speakers or drifting timestamps will move W by several turns and turn a timely transfer into a late one.

Check speaker attribution first. Two-channel recordings separate caller and agent by channel; single-channel recordings need speaker diarization, the who-spoke-when task that open toolkits such as pyannote.audio address. Then check transcript quality on a sample with word error rate, which libraries such as jiwer compute, paying special attention to the words that define W: requests for a person, account identifiers and refusals.

Outcome maturity matters as much as audio quality. A call that looks resolved at hang-up may produce a callback, a chargeback or a complaint weeks later. Set an outcome window in advance, link each call to the case or ticket that followed, and leave calls with immature outcomes out of the premature and correctly kept categories.

Worked example: a billing dispute that waited five turns (illustrative)

Everything in this example, including times and turn counts, is illustrative. A caller phones about a charge they do not recognize. At turn 4, 38 seconds in, they say it looks like fraud and ask whether someone can stop it. The written policy treats a report of possible fraud as a transfer-now signal, so two reviewers independently mark W at turn 4 and grade the call S3.

The agent asks for the last four digits of the card, mishears them twice and asks again, then offers to email a dispute form. The caller says "just get me a person" at turn 9. The agent invokes the handoff at turn 9, 112 seconds in; the call enters the queue 6 seconds later and a person answers 95 seconds after that.

Scored: W-to-A is 5 turns and 74 seconds against an S3 tolerance of one turn, so the call is late. W-to-H is 175 seconds. The outcome record shows the human agent blocked the card and opened a dispute the same day, so the transfer was correct; only its timing failed. A binary transfer label would have scored this call as a success.

The exception: in a second call, the caller says "my agent told me to call", meaning their insurance agent, and the transcript renders it as a request for an agent. Reviewers listening to the audio do not mark W, and the agent's decision to continue is scored correctly kept. Labeling from the transcript alone would have produced a false late transfer.

How do reviewers keep the warranted point from drifting?

The protocol is only as reliable as the W label, so measure how often two reviewers place W on the same turn before trusting any timing result. Disagreement concentrated on one signal usually means the written policy is vague about it.

Train reviewers on a calibration set, blind them to the actual transfer turn, and have them listen rather than read. Record W as a turn and a time, let reviewers mark a window when the warranted moment builds over several turns, and adjudicate splits with a third reviewer whose decision and reason stay in the record. Report agreement by signal type alongside the results so a reader can discount the categories where reviewers disagreed most.

Sampling decides what the results can claim. Draw calls from a stated period and queue, stratify by signal type and by the severity a first-pass reviewer assigns, and oversample the rare high-severity calls so they can be scored at all, recording the sampling weights so overall rates can be reweighted back to the real call mix. Compare agents or agent versions on the same sampled calls, replayed where possible, rather than on different months of traffic.

  • Pre-registration: the written transfer policy and its effective dates.
  • Pre-registration: severity definitions, tolerances per grade and the outcome window.
  • Pre-registration: the sampling frame, strata and weights.
  • Pre-registration: the agreement threshold below which a signal type is reported but not scored.
  • Pre-registration: the exclusion rules and the metrics, with how uncertainty will be estimated.

Which calls should the protocol not score?

Leave out calls where timing cannot be judged fairly from the record. Scoring them adds noise and can create wrong conclusions about the agent.

Exclude calls without a usable recording or with consent that does not cover review, calls where the policy in force on the date cannot be found, calls cut off by a network failure, and calls whose outcome is still open at the end of the window. Treat medical, emergency and legal-advice calls as out of scope for this protocol and route them to specialist review, since the right transfer point there depends on rules a general reviewer should not apply.

Where do real transfer records for this protocol come from?

The protocol needs real calls with audio, aligned transcripts, platform timestamps, the policy in force and the case outcome that followed; synthetic calls rarely reproduce the recognition failures and hesitations that define W. Contact centers that kept recordings, contact records and case histories together hold the material.

An AI team can ask SourceX to find calls like these at U.S. businesses; nothing is held in stock, and the request sets scope, volume, language, format, timing and licensing needs. The business that owns the calls approves scope and terms and signs before delivery, and recordings are de-identified as the agreement sets out; whether recording consent covers AI use is checked as its own question, separate from having the files.

Frequently asked questions

How long should a voice agent try before transferring to a person?

There is no single number; it depends on the signal and the severity. An explicit request for a person, a request outside the agent's permitted actions or a safety risk calls for an immediate transfer, while a single recognition failure usually justifies a retry. Set tolerances per severity grade in advance and measure against them.

What is the difference between escalation delay and hold time?

Escalation delay runs from the moment a transfer became warranted to the moment the agent decided to hand off. Hold time runs from the queue to a person answering, which platforms such as Amazon Connect record in the contact record. Callers experience both, so the protocol reports each and their sum.

Can you measure transfer timing from transcripts alone?

Only roughly. Transcripts can misattribute speakers, mishear the words that define the warranted moment and drift in time. Label from audio aligned to the transcript, check diarization and word error rate on a sample, and keep the recording available for adjudication.

Is a premature transfer a failure?

Yes, when the outcome record shows the agent could have resolved the call within policy. It costs staff time and can annoy callers who were close to done. The protocol counts premature transfers separately from late and missed ones so a team does not trade one failure for another unknowingly.

Does this protocol report benchmark results?

No. It describes how to label, time and score calls, and the worked example uses illustrative numbers. Teams run it on their own recordings and publish results with their sample, agreement figures and outcome window.

Sources

  • The CFPB's report on chatbots in consumer finance describes doom loops of repetitive automated responses without an offramp to a human customer service representative. Source
  • tau-bench's airline transfer tool instructs the agent to transfer with a summary only if the user explicitly asks for a human or the issue cannot be resolved with the available tools. Source
  • Amazon Lex V2 matches its fallback intent for unexpected, noisy or ambiguous input, or when a slot value is not recognized after the configured number of tries. Source
  • Amazon Connect contact records hold initiation, enqueue and connected-to-agent timestamps in UTC; abandoned contacts lack a connected-to-agent time. Source
  • Dialogflow CX's live agent handoff response is used by the platform only to measure which conversations were handed off. Source
  • pyannote.audio is an open-source toolkit for speaker diarization. Source
  • JiWER is a Python package for evaluating speech recognition, including word error rate. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies