AI uses for records
Call transfer agent benchmark: can a voice agent carry context through a transfer?
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Build a set of multi-leg calls where the facts established before each transfer, including corrections, verification state and promises, are listed with their source turns. Capture what the spoken intro, call signaling, contact attributes and transfer summary carried, then grade the receiving leg fact by fact for re-asks, stale or invented details and dropped commitments, keeping verified case outcomes separate from how the call sounded.
Agent run
Call transfer agent
Key takeaways
- Context continuity means the receiving party starts with every established, shareable fact and nothing stale or invented.
- Context travels through speech, call signaling, contact-center attributes and tool summaries, and each channel must be captured.
- A context ledger with source turns and correction history is the gold standard the receiving leg is graded against.
- Score re-asks, stale and invented context, commitments and verification handling separately from verified case outcomes.
- Some re-asks are correct under policy, so write per-queue expectations with the operations owner.
What does carrying context through a transfer actually mean?
It means the party who takes over the call starts with everything the caller already established that is still true and allowed to be shared, and nothing that is wrong. The test is simple to state: after the transfer, did the caller have to repeat themselves, and did the new party act on stale or invented details?
SIP's transfer guidance names three roles: the transferee, who is being moved; the transferor, who starts the transfer; and the transfer target, the new party. A voice agent can play either of the last two. The common cases are a voice agent handing a caller to a person, a person handing back to an agent, and one specialist agent handing to another.
Context has more parts than a summary sentence suggests. It includes who the caller is and whether identity was verified, why they called, the facts collected, any corrections the caller made, what was already tried, commitments made to the caller, and signals such as urgency. A benchmark has to check each part separately, because agents tend to pass the intent and drop the corrections.
How does context travel from one leg of the call to the next?
Context moves through up to four channels at once, and a continuity benchmark must record what was in each. The spoken channel is the warm introduction a transferor gives the target. The signaling channel carries small data with the call itself. The platform channel stores attributes on the contact record. The tool channel is the summary an agent writes when it calls a transfer function.
On the signaling side, RFC 7433 defines a User-to-User header in SIP for a small, opaque payload that the initiating application inserts and the receiving application reads; SIP does not interpret it, the meaning is set by a named package, and the mechanism must not be used to carry a URL. For referrals the header can be escaped inside the Refer-To URI.
On the platform side, Amazon Connect documents two kinds of user-defined attributes. Contact attributes propagate immediately to every segment in a contact chain during transfers and conferences, and Amazon warns that a transfer overwrites previously set values, so per-leg data can be lost. Contact segment attributes stay specific to one segment. A benchmark that reads only the final attribute values may miss what the target saw at the moment of pickup.
On the tool side, the tau2-bench airline domain gives its agent a transfer_to_human_agents tool that takes a summary of the user's issue, and its policy allows a transfer only when the request cannot be handled within the agent's actions. That summary argument is exactly the artifact a continuity test should grade.
What should the transfer continuity evaluation set include?
Each item is one real or realistic call with at least two legs, the context state the caller established before the transfer, what each channel carried across, what the receiving party then did, and a later verified outcome. The table lists the fields. Keep the pre-transfer context as a structured list of facts, each with the turn and timestamp that established it, so the grader can check the receiving leg fact by fact.
The single most useful field is the corrections list. Callers often change a detail mid-call, such as the date of a charge or which account they meant, and a transfer is where the pre-correction value most often resurfaces.
| Field | What it holds | Why the evaluation needs it |
|---|---|---|
| call and leg ids | Call id, ordered legs with start and end times, transfer type: blind, warm or consultative | Lets every turn and attribute be placed on the right leg |
| roles | Transferee, transferor and target for each transfer, and whether each is a person or an agent | Separates agent-to-person from agent-to-agent behavior |
| audio and rights | Per-leg audio reference, channel layout, sample rate and the recording and use permission on file | Confirms the audio may be used and matches the transcript |
| aligned transcript | Speaker-attributed turns with start and end timestamps and repair turns marked | Shows who established each fact and when |
| context ledger | Facts established before the transfer, each with value, source turn, verification state and correction history | Gold list the receiving leg is graded against |
| carried context | What the spoken intro, signaling payload, contact attributes and tool summary each contained | Shows which channel dropped or distorted a fact |
| receiving-leg behavior | Facts re-asked, facts acted on, stale or invented details used, and time until the issue is restated | Raw material for the continuity metrics |
| policy expectations | Facts that must be re-verified or withheld after transfer under the operator's policy | Prevents scoring a required re-check as a failure |
| outcome and maturity | Resolution recorded in the ticket, order or case system, with date and how final it is | Separates what the call claimed from what happened |
| review record | Reviewer roles for audio, transcript and outcome labels, with dates | Makes every label auditable |
Which transfer scenarios should the set cover?
Cover the transfer patterns your deployment will meet, weighted toward the ones that fail quietly. A set made only of clean warm transfers will overstate continuity, because the hard cases are corrections, unverified callers and transfers that fail and come back.
The rows below are a starting matrix. For each one, include both successful and failed examples from real records where possible, and record the transfer type so results can be split by it.
| Scenario | What it tests | Continuity expected | Typical failure |
|---|---|---|---|
| Agent to person, warm | Summary and spoken intro quality | Target restates the issue correctly before asking anything new | Target asks for the account and reason again |
| Agent to person, blind | Whether data channels alone carry context | Contact attributes or the tool summary hold the key facts | Only the queue name arrives |
| Person back to agent | Whether the agent ingests what the person learned | Agent picks up from the person's last commitment | Agent restarts its script from the greeting |
| Agent to specialist agent | Context schema between agents | Specialist receives the ledger, not just free text | Specialist re-collects identity and intent |
| Transfer after a correction | Whether corrections survive | Corrected value is the one passed on | Pre-correction value reaches the target |
| Unverified caller | Verification state handling | Target knows verification is incomplete | Target treats the caller as verified |
| Failed transfer and callback | Recovery across a dropped leg | Callback leg starts from the saved context | Caller starts over from the beginning |
| Multi-hop transfer | Accumulation and overwrite across legs | Each leg sees facts from all earlier legs | Second transfer overwrites the first leg's facts |
How do you score what the receiving side knew?
Grade the receiving leg against the context ledger, fact by fact, and keep conversational signals apart from verified outcomes. A call can sound smooth and still end in the wrong refund, and an awkward call can be resolved correctly.
Weight the ledger facts before you aggregate. A dropped account number that the target can look up in seconds is a smaller failure than a dropped correction to the month of the disputed charge, and a single blended rate treats them the same. Agree the weights with the operations owner and publish them with the results.
Report each metric by scenario and transfer type, with counts and intervals, and run multiple trials for simulated items. tau2-bench evaluates agents in simulation with a language-model user and supports both turn-based text and full-duplex voice, which makes it a useful harness for the simulated portion; real recorded calls give the distribution of failures that simulation tends to miss.
- Re-ask rate: share of established, shareable facts the receiving party asked for again.
- Stale-context rate: share of items where a pre-correction value was passed or acted on.
- Invented-context rate: share of summaries or intros that state a fact the caller never gave.
- Commitment carryover: share of earlier promises, such as a callback time, that the receiving leg honored or acknowledged.
- Verification handling: whether the target knew the caller's verification state and followed the re-check policy.
- Time to restated issue: seconds from pickup until the target correctly states the caller's issue.
- Verified outcome: whether the case system later shows the issue resolved, kept separate from the call's own claim.
Can the recording back up the transcript and the case outcome?
It can, once three checks pass. Before any item becomes gold, confirm that the recording exists for every leg, that the transcript's speaker labels and timestamps line up with the audio, and that the outcome label comes from a later record in a system of record rather than from what was said on the call.
Transfers make alignment harder than single-leg calls. Speaker labels change as parties join and leave, hold music and recorded announcements interrupt the audio, and each leg may be stored as a separate file. Keep one clock for the whole call and map every leg onto it.
Audio conditions should be labeled because they change results. The tau2-bench voice module offers a control preset with clean audio and patient users and a regular preset with background noise, accents and interruptions, and it converts simulated speech to G.711 mu-law at 8 kHz, the telephone format. Label real calls with the same kinds of conditions so simulated and real results can be compared.
Permission to use the audio belongs in the item, not in a side file. Recording notices and consent rules differ by place and purpose, and whether a set of calls may be used for evaluation is a question for the records owner and counsel, so record the basis on file for each call and exclude calls without one.
Illustrative worked example: a disputed charge handed to a person
This example is illustrative; the business, counts and timings are invented to show the method, and no results are reported. A caller phones a utility's voice agent about a charge. In the first two minutes they give their account, say the charge is from March, then correct themselves to April, and say they already tried the online dispute form.
The agent verifies identity, finds it cannot reverse the charge under its policy and transfers to a person. Its transfer summary and the contact attributes are both captured, along with the spoken warm intro. The context ledger holds five facts: account, verified status, disputed charge in April, form already tried, and the caller's request for a callback if the dispute takes more than a day.
Grading the receiving leg: the person restated the issue 20 seconds after pickup, did not re-ask the account, but referred to a March charge, which came from the summary written before the correction, and did not mention the callback request. On the illustrative scorecard that is one stale fact and one dropped commitment out of five ledger facts, with no re-asks.
The case record two days later shows the April charge credited. The outcome is resolved, but the continuity failures still count, because in a different case the stale month would have led to the wrong credit.
When should context deliberately not carry over?
Context should not carry over when the operator's policy requires the target to re-verify identity, when a fact is too sensitive to put in a summary or attribute, or when the caller withdrew a request before the transfer. A receiving party who re-asks in these cases is following policy, and the benchmark must not count it as a failure.
Sensitive values are the clearest example. A full card number or a one-time passcode spoken on the first leg should not appear in a transfer summary or a contact attribute, and an item where it does should fail even if the call was resolved. Label these facts in the ledger as not shareable.
The policy expectations field exists for this exception. Write it with the operations owner for each queue, because the right answer differs between, say, a billing line that requires fresh verification and a technical support line that does not.
How should legs, roles and outcomes be packaged so the set stays usable?
Store each call as legs plus a transfer record that names the roles and points to the legs, not as one flattened transcript. Flattening hides which party knew what, which is the whole point of the test.
The IETF vCon working group's draft container format has a transfer dialog type that does this: it holds no conversation content itself, records which parties were the transferee, transferor and transfer target as indexes into the parties list, and points to the original, consultation and target dialogs. That draft, version -03, has since expired, with the working group's text continuing in a vCon core draft, so treat field names as subject to change; the structure is still a good model for packaging a continuity set.
Keep a per-call manifest beside the audio: leg files with their hashes, the transcript version, the attribute snapshots taken at each pickup, the transfer summary as written, and the case record id. When a reviewer disputes a label, the manifest lets them replay exactly what the receiving party had in front of them at that moment, rather than what the record looked like afterward.
Split by site, queue and time period rather than by call, because scripts and transfer habits repeat within a queue. Version the set and date the outcome labels, since case outcomes mature after the call. Know the limits: the test measures continuity on the scenarios and policies you wrote, not caller satisfaction, and not behavior on queues you left out.
Where can real transferred calls for a continuity test come from?
Real transferred calls with case outcomes are hard to build from public data, because they need multi-leg audio, aligned transcripts and a later record from a ticket or case system, all with a clear basis for use. Contact center operators, outsourced service providers and businesses with in-house support lines hold this material.
Transferred-call sets are assembled for a specific request rather than kept on a shelf: SourceX finds them at U.S. businesses, and the buyer sets out the scope, volume, language, format, timing and licensing terms, for example transferred calls with per-leg audio and the linked case records. No call leaves the supplying business until it has agreed the scope and terms and signed the agreement.
Frequently asked questions
What is a call transfer agent benchmark?
It is an evaluation set that tests whether context survives when a call moves between a voice agent and a person or another agent. Each item pairs the facts established before the transfer with what each channel carried and what the receiving party then did, and scores re-asks, stale or invented details and dropped commitments.
Is a warm transfer always better for context than a blind transfer?
Not necessarily. A warm transfer adds a spoken introduction, but it can still pass stale details, and a blind transfer can carry context well if the summary and contact attributes are complete. Measure both and report them separately.
Should the receiving agent re-verify the caller?
That depends on the operator's policy for the queue. The benchmark should record the policy per queue and score a required re-check as correct, while still checking that the target knew the verification state before asking.
Can simulated calls replace real transferred calls?
Simulation, such as the tau2-bench harness with voice conditions, is useful for controlled and repeatable tests. Real calls add the corrections, crosstalk and failed transfers that simulations tend to miss, so keep both and report them separately.
How should outcomes be labeled for transferred calls?
Take the outcome from a later record in the ticket, order or case system and date it, rather than from what was said on the call. Keep a maturity flag, because some cases are reopened after the first resolution.
Sources
- RFC 5589 describes SIP call transfer using REFER and Replaces and names the transferee, transferor and transfer target roles. Source
- RFC 7433 defines a SIP User-to-User header for a small opaque payload whose meaning is set by a named package, which must not carry a URL and can be escaped into a Refer-To URI. Source
- Amazon Connect contact attributes propagate to all segments of a contact chain during transfers and conferences, while contact segment attributes stay specific to one segment. Source
- Amazon Connect warns that contact attributes are shared across transfers and that a transfer overwrites previously set values. Source
- The tau2-bench airline domain's transfer_to_human_agents tool takes a summary of the user's issue, and the domain policy allows transfer only when the request cannot be handled within the agent's actions. Source
- tau2-bench simulates customer service conversations with a simulated user in text and full-duplex voice, with voice presets for clean and realistic audio and telephone-format 8 kHz audio. Source
- The IETF vCon container draft defines a transfer dialog that holds no content, records the transferee, transferor and transfer target, and points to the original, consultation and target dialogs. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.