AI uses for records
Multi-agent handoff evaluation: did agent B receive enough context?
By SourceX Editorial · Updated
9 sections
Short answer
To test whether one agent passed enough context to another, write a ledger of what the receiver must know (requirements, approvals, current shared state, open items), log the exact handoff payload, and plant probes after the handoff where forgetting is the easy path. Score the receiver's actions and the end state in the system of record, repeat runs, and compare with an oracle handoff to separate handoff loss from receiver error.
Agent run
Multi agent handoff
Key takeaways
- A handoff test needs a written ledger of what must survive, prepared before the run.
- Log the exact payload the receiver got; otherwise omitted and ignored context look identical.
- Plant six kinds of loss: requirements, authority, stale state, open items, repeated side effects and unowned checks.
- Score the end state in the system of record, never a status field or final message.
- An oracle-handoff control run shows how much of the failure the handoff itself caused.
How do you test whether one agent passed enough context to another?
Write down, before the run, everything agent B must know to finish correctly: the standing requirements, who approved what, the current value of every shared record, and the items still open. Then hide probes after the handoff where forgetting one of those items is the easy path, and score what agent B does at each probe and in the systems it touches, not what its final message says.
That is the whole method in two sentences. The rest of this page turns it into a task pack a team can build: a ledger of what must survive the handoff, a set of planted conflicts, linked evidence for every verdict, and a control run that separates a thin handoff from a weak receiver.
The distinction matters because a handoff failure and a receiver failure look the same from the outside. Agent B ships the wrong vendor terms either way. Only a test that knows what agent A was supposed to pass, and what it actually passed, can say which agent to fix.
What does a handoff actually carry in today's agent frameworks?
It depends on the transport, so the test has to record what crossed, not assume it. In the OpenAI Agents SDK, a handoff is exposed to the model as a tool, and by default the receiving agent sees the entire previous conversation history. Developers can change that with an input filter, and a built-in filter removes structured tool items from the history the next agent sees.
The same documentation notes two details that matter for evaluation. The filter that removes tool items does not redact tool arguments or results already copied into ordinary messages, and input guardrails run only for the first agent in a chain while output guardrails run only for the agent that produces the final output. A receiver can therefore act without the checks the first agent passed through.
The Agent2Agent (A2A) protocol takes a different shape. Agents exchange messages and tasks grouped by a contextId, a client can point at related work with referenceTaskIds, and a task can pause in an input-required or auth-required state until someone supplies input or authorization. Here each agent keeps its own internal state, so the handoff carries only what one agent chose to send.
Shared transcript or explicit message, the evaluation question is the same: of the items agent B needed, which ones arrived in a form it used? Logging the exact handoff payload for every run is the first requirement of the pack.
What should the handoff evidence and responsibility task pack include?
The pack has nine parts. Each one exists so that a reviewer can point to a record, not an impression, when deciding whether context was lost. Field names below are illustrative; map them onto your own trace and ticket schema.
| Component | What it records | Illustrative field | How it is used in scoring |
|---|---|---|---|
| Dependency graph | Which subtasks feed which, and which agent owns each | task_id, depends_on, owner_agent | Shows which handoffs a lost item would have to cross |
| Constraint ledger | Every requirement in force at the start, numbered | C1 payment terms as approved, source: approval email | Each ledger item becomes one retention check |
| Role and authority register | What each agent and person may approve, change or send | agent_b may create vendor, may not change bank details | Detects a receiver acting on authority it never held |
| Shared state snapshot | Value and version of each record read before the handoff | vendor_record v3, read at step 7 | Detects stale reads after someone else edits the record |
| Human change log | Edits a person made mid-task, with time and reason | Finance changed terms at step 9 | Tests whether late changes reach the receiver |
| Handoff packet as sent | The exact payload or history the receiver got | Serialized input after filters | Separates 'never sent' from 'sent and ignored' |
| Planted probes | Points after the handoff where a lost item causes a visible error | Step 14 asks agent B to set terms | Where retention is scored |
| Outcome check | The end state in the system of record | Vendor master terms field equals net 45 | Scores the task, not the status message |
| Counterexample run | The same task with the full ledger passed to the receiver | Oracle handoff variant | Isolates handoff loss from receiver error |
Which six kinds of context loss should the pack probe?
Six kinds of loss account for most of what a handoff test should probe. Plant at least one of each, place the probe after the handoff, and make sure the shortcut is tempting: the default value, the earlier state, the broader permission.
Failure research supports probing these separately. The MAST study of multi-agent LLM systems groups failures into three categories, one of which is inter-agent misalignment, with modes that include withholding information from another agent and ignoring another agent's input. The authors also point out that missing information can come from withholding, from ignoring input or from context mismanagement, which is why the packet as sent has to be logged.
| Loss type | What it looks like | Planted probe | Evidence that proves it |
|---|---|---|---|
| Dropped requirement | Receiver applies a default instead of an agreed term | Ask the receiver to fill the field the requirement governs | Ledger item, packet without it or with it, system field value |
| Authority inflation | Receiver performs an action only the sender or a person could approve | Offer a tool call that changes a protected field | Authority register, tool call span, approval record absent |
| Stale shared state | Receiver uses a value read before a later edit | Edit the record between handoff and probe | Record version read by receiver versus current version |
| Lost open item | A pending approval or callback is treated as done | Leave one approval pending at handoff time | Open-item list in packet, receiver's next action |
| Repeated side effect | Receiver redoes an action the sender completed | Sender completes a send or create step before handing off | Two create or send events for one intent |
| Misattributed responsibility | Each agent assumes the other owns a check | Assign a verification step to no one explicitly | Dependency graph owner field, check never executed |
How should the receiver be scored so a status field cannot pass it?
Score three layers and report them separately. First, retention: for each ledger item, did the receiver act consistently with it at its probe? Second, outcome: does the end state in the system of record match the target state written before the run? Third, reliability: does the receiver get it right on repeated runs of the same task?
The outcome layer follows the approach used in tau-bench, which grades an episode by comparing the database at the end of the conversation with an annotated goal state rather than trusting the agent's own account. The same paper introduced pass^k, the chance that an agent succeeds on all of k repeated trials, and its public repository reports results in pass^1 through pass^4 columns. Repetition matters for handoffs because a summary written by a model can drop a different item each run.
A completed status, a confident final message or a closed ticket counts for nothing on its own. A task whose message says the vendor was set up with net 45 terms, while the vendor master says net 30, is a failed retention check and a failed outcome check.
To attribute each action to the right agent, record traces with stable agent identifiers. The OpenTelemetry generative AI conventions, still marked as in development, define an invoke_agent operation and a gen_ai.agent.id attribute, which gives each span a durable owner a verdict can cite.
- Retention score: share of ledger items honored at their probes, per loss type.
- Outcome score: pass or fail against the target state, checked in the system of record.
- Reliability: pass^k over repeated runs of the same task and handoff.
- Attribution: for each failed item, sender omitted it, packet carried it but receiver ignored it, or receiver lacked authority.
- Control gap: oracle handoff score minus normal handoff score, the share of failures the handoff itself caused.
Worked case (illustrative): vendor intake handed to a setup agent
This example is invented to show the method; the counts are illustrative, not measured. Agent A runs vendor intake for a purchasing team: it collects the vendor's documents, checks the request against policy and hands off to agent B, which creates the vendor in the ERP and schedules the first purchase order.
Starting state: the ledger holds 5 items. C1, the payment terms the controller approved, which read net 30 when agent A opened the vendor record. C2, bank detail changes require a callback by finance before any payment setup. C3, the first purchase order may not exceed the approved budget line. C4, a pending tax form is still open. C5, agent B may create vendors but may not edit bank fields. At step 9 a person in finance changes those terms to net 45 with the controller's sign-off, after agent A first read the record.
Linked evidence: the packet agent A sent is logged; it carries C1 with the old value, C3 and C5, and omits C2 and C4. The trace shows agent B reading vendor record version 3, while version 4 holds the edit.
Decision: agent B sets net 30, marks the tax form complete because nothing told it otherwise, and does not call finance. Outcome check: the vendor master shows net 30 and an active payment profile without a callback record. Verdict: 3 of 5 items lost, two by omission from the packet and one by a stale read, and the task fails even though both agents reported success.
Counterexample: in the oracle variant, agent B receives the full ledger and the current record version, and keeps all 5 items in 4 of 5 illustrative repeats. The gap between the two variants is the handoff's share of the failure; the remaining miss belongs to the receiver.
Which records must a reviewer see before accepting a handoff verdict?
Every verdict in the pack should cite three things: the ledger item it scores, the trace span where the receiver acted, and the version of the business record that shows the result. A verdict missing any of the three is an opinion, and reviewers should send it back.
The proof of outcome lives in the system the task changed. For vendor setup that is the ERP vendor master and payment profile; for a support escalation it is the ticket's field history in a tool like Zendesk or Jira; for a sales handoff it is the opportunity record in Salesforce or HubSpot with its field audit trail. A transcript can explain why something happened, but only the record shows that it did.
Keep the human change log in the same timeline. Late edits by people are where stale-state failures come from, and they are often absent from agent traces entirely because the person changed the record directly in the business application.
Where do handoff tests mislead, and how do you keep them honest?
A handoff test built only from synthetic tasks tends to plant constraints that are easy to spot, so it can overstate retention. Real handoffs carry messier context: a terms change buried in an email thread, an approval given verbally and logged later, a permission that applies only to one vendor class.
Small packs also mislead. A pack with a handful of tasks per loss type can show a difference between two handoff designs that disappears on the next batch; report the number of tasks and runs beside every score, and treat single-digit counts as a smoke test.
Framework behavior moves. Default history passing, filter semantics and protocol task states are documented today, and a release can change them. Record the framework and protocol versions in each run so a later score drop can be traced to a change in the transport rather than the model.
Where do realistic handoff cases come from?
The best seeds for a handoff pack are records of people handing work to each other: ticket reassignments with their notes, shift handover logs, case transfers between teams, and the approvals and corrections that followed. They show which constraints people actually forget and which records prove the result.
A team building a handoff pack from real records can describe to SourceX the scope, volume, language, format, timing and licensing terms it needs; SourceX holds no ready-made stock and looks for U.S. businesses whose records fit. On the supply side, the company that owns the records approves the scope and terms and signs before any data is delivered.
Frequently asked questions
What is a multi-agent handoff evaluation?
It is a test of whether the context one agent passes to another is enough for the receiver to finish the task correctly. It plants requirements, permissions and shared state before the handoff and scores the receiver's actions and the end state afterward. A good one also logs the exact handoff payload so failures can be attributed.
Is passing the full conversation history enough context?
Not reliably. Full history can still contain a stale value that a person later changed in the business system, and a receiver may not treat an old message as a binding requirement. Some frameworks also let developers filter history, so the test should log what actually crossed rather than assume everything did.
How many tasks does a handoff evaluation pack need?
Enough per loss type to tell a real difference from noise, which depends on how often failures occur. Start with a small pack to debug the probes, then grow each loss type before comparing handoff designs. Always publish task counts and repeated-run counts next to scores.
How do you tell a bad handoff from a bad receiving agent?
Run a control where the receiver gets the full constraint ledger and current state, then compare it with the normal handoff. Failures that disappear in the control were caused by the handoff; failures that remain belong to the receiver. The logged packet then shows whether an item was omitted or ignored.
Can human handoff records be used to evaluate AI agents?
Yes, as seeds for realistic tasks. Ticket reassignments, shift notes and case transfers show which constraints get lost in practice and which records prove outcomes. They need rights review and de-identification before they leave the business that holds them.
Sources
- In the OpenAI Agents SDK, handoffs are exposed as tools, the receiving agent sees the previous conversation history by default, input filters change what it sees, and guardrails apply only to the first and final agents. Source
- The A2A specification groups related tasks and messages by contextId, lets clients reference related tasks, and defines input-required and auth-required task states. Source
- OpenTelemetry's GenAI agent span conventions, in development status, define an invoke_agent operation and a stable gen_ai.agent.id attribute. Source
- The MAST taxonomy groups multi-agent LLM failures into three categories, including inter-agent misalignment with information withholding and ignored input among its modes. Source
- tau-bench grades episodes by comparing the final database state with an annotated goal state and proposes pass^k to measure reliability across trials. Source
- The tau-bench repository reports leaderboard results as Pass^1 through Pass^4 for its airline and retail domains. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.