Skip to content

AI uses for records

Multi-agent handoff evaluation: did agent B receive enough context?

By SourceX Editorial · Updated

9 sections

Short answer

To test whether one agent passed enough context to another, write a ledger of what the receiver must know (requirements, approvals, current shared state, open items), log the exact handoff payload, and plant probes after the handoff where forgetting is the easy path. Score the receiver's actions and the end state in the system of record, repeat runs, and compare with an oracle handoff to separate handoff loss from receiver error.

Agent run

Multi agent handoff

This page covers multi-agent handoff evaluation: did agent B receive enough context.Illustrative

Key takeaways

  • A handoff test needs a written ledger of what must survive, prepared before the run.
  • Log the exact payload the receiver got; otherwise omitted and ignored context look identical.
  • Plant six kinds of loss: requirements, authority, stale state, open items, repeated side effects and unowned checks.
  • Score the end state in the system of record, never a status field or final message.
  • An oracle-handoff control run shows how much of the failure the handoff itself caused.

How do you test whether one agent passed enough context to another?

Write down, before the run, everything agent B must know to finish correctly: the standing requirements, who approved what, the current value of every shared record, and the items still open. Then hide probes after the handoff where forgetting one of those items is the easy path, and score what agent B does at each probe and in the systems it touches, not what its final message says.

That is the whole method in two sentences. The rest of this page turns it into a task pack a team can build: a ledger of what must survive the handoff, a set of planted conflicts, linked evidence for every verdict, and a control run that separates a thin handoff from a weak receiver.

The distinction matters because a handoff failure and a receiver failure look the same from the outside. Agent B ships the wrong vendor terms either way. Only a test that knows what agent A was supposed to pass, and what it actually passed, can say which agent to fix.

What does a handoff actually carry in today's agent frameworks?

It depends on the transport, so the test has to record what crossed, not assume it. In the OpenAI Agents SDK, a handoff is exposed to the model as a tool, and by default the receiving agent sees the entire previous conversation history. Developers can change that with an input filter, and a built-in filter removes structured tool items from the history the next agent sees.

The same documentation notes two details that matter for evaluation. The filter that removes tool items does not redact tool arguments or results already copied into ordinary messages, and input guardrails run only for the first agent in a chain while output guardrails run only for the agent that produces the final output. A receiver can therefore act without the checks the first agent passed through.

The Agent2Agent (A2A) protocol takes a different shape. Agents exchange messages and tasks grouped by a contextId, a client can point at related work with referenceTaskIds, and a task can pause in an input-required or auth-required state until someone supplies input or authorization. Here each agent keeps its own internal state, so the handoff carries only what one agent chose to send.

Shared transcript or explicit message, the evaluation question is the same: of the items agent B needed, which ones arrived in a form it used? Logging the exact handoff payload for every run is the first requirement of the pack.

What should the handoff evidence and responsibility task pack include?

The pack has nine parts. Each one exists so that a reviewer can point to a record, not an impression, when deciding whether context was lost. Field names below are illustrative; map them onto your own trace and ticket schema.

What should the handoff evidence and responsibility task pack include?
ComponentWhat it recordsIllustrative fieldHow it is used in scoring
Dependency graphWhich subtasks feed which, and which agent owns eachtask_id, depends_on, owner_agentShows which handoffs a lost item would have to cross
Constraint ledgerEvery requirement in force at the start, numberedC1 payment terms as approved, source: approval emailEach ledger item becomes one retention check
Role and authority registerWhat each agent and person may approve, change or sendagent_b may create vendor, may not change bank detailsDetects a receiver acting on authority it never held
Shared state snapshotValue and version of each record read before the handoffvendor_record v3, read at step 7Detects stale reads after someone else edits the record
Human change logEdits a person made mid-task, with time and reasonFinance changed terms at step 9Tests whether late changes reach the receiver
Handoff packet as sentThe exact payload or history the receiver gotSerialized input after filtersSeparates 'never sent' from 'sent and ignored'
Planted probesPoints after the handoff where a lost item causes a visible errorStep 14 asks agent B to set termsWhere retention is scored
Outcome checkThe end state in the system of recordVendor master terms field equals net 45Scores the task, not the status message
Counterexample runThe same task with the full ledger passed to the receiverOracle handoff variantIsolates handoff loss from receiver error

Which six kinds of context loss should the pack probe?

Six kinds of loss account for most of what a handoff test should probe. Plant at least one of each, place the probe after the handoff, and make sure the shortcut is tempting: the default value, the earlier state, the broader permission.

Failure research supports probing these separately. The MAST study of multi-agent LLM systems groups failures into three categories, one of which is inter-agent misalignment, with modes that include withholding information from another agent and ignoring another agent's input. The authors also point out that missing information can come from withholding, from ignoring input or from context mismanagement, which is why the packet as sent has to be logged.

Which six kinds of context loss should the pack probe?
Loss typeWhat it looks likePlanted probeEvidence that proves it
Dropped requirementReceiver applies a default instead of an agreed termAsk the receiver to fill the field the requirement governsLedger item, packet without it or with it, system field value
Authority inflationReceiver performs an action only the sender or a person could approveOffer a tool call that changes a protected fieldAuthority register, tool call span, approval record absent
Stale shared stateReceiver uses a value read before a later editEdit the record between handoff and probeRecord version read by receiver versus current version
Lost open itemA pending approval or callback is treated as doneLeave one approval pending at handoff timeOpen-item list in packet, receiver's next action
Repeated side effectReceiver redoes an action the sender completedSender completes a send or create step before handing offTwo create or send events for one intent
Misattributed responsibilityEach agent assumes the other owns a checkAssign a verification step to no one explicitlyDependency graph owner field, check never executed

How should the receiver be scored so a status field cannot pass it?

Score three layers and report them separately. First, retention: for each ledger item, did the receiver act consistently with it at its probe? Second, outcome: does the end state in the system of record match the target state written before the run? Third, reliability: does the receiver get it right on repeated runs of the same task?

The outcome layer follows the approach used in tau-bench, which grades an episode by comparing the database at the end of the conversation with an annotated goal state rather than trusting the agent's own account. The same paper introduced pass^k, the chance that an agent succeeds on all of k repeated trials, and its public repository reports results in pass^1 through pass^4 columns. Repetition matters for handoffs because a summary written by a model can drop a different item each run.

A completed status, a confident final message or a closed ticket counts for nothing on its own. A task whose message says the vendor was set up with net 45 terms, while the vendor master says net 30, is a failed retention check and a failed outcome check.

To attribute each action to the right agent, record traces with stable agent identifiers. The OpenTelemetry generative AI conventions, still marked as in development, define an invoke_agent operation and a gen_ai.agent.id attribute, which gives each span a durable owner a verdict can cite.

  • Retention score: share of ledger items honored at their probes, per loss type.
  • Outcome score: pass or fail against the target state, checked in the system of record.
  • Reliability: pass^k over repeated runs of the same task and handoff.
  • Attribution: for each failed item, sender omitted it, packet carried it but receiver ignored it, or receiver lacked authority.
  • Control gap: oracle handoff score minus normal handoff score, the share of failures the handoff itself caused.

Worked case (illustrative): vendor intake handed to a setup agent

This example is invented to show the method; the counts are illustrative, not measured. Agent A runs vendor intake for a purchasing team: it collects the vendor's documents, checks the request against policy and hands off to agent B, which creates the vendor in the ERP and schedules the first purchase order.

Starting state: the ledger holds 5 items. C1, the payment terms the controller approved, which read net 30 when agent A opened the vendor record. C2, bank detail changes require a callback by finance before any payment setup. C3, the first purchase order may not exceed the approved budget line. C4, a pending tax form is still open. C5, agent B may create vendors but may not edit bank fields. At step 9 a person in finance changes those terms to net 45 with the controller's sign-off, after agent A first read the record.

Linked evidence: the packet agent A sent is logged; it carries C1 with the old value, C3 and C5, and omits C2 and C4. The trace shows agent B reading vendor record version 3, while version 4 holds the edit.

Decision: agent B sets net 30, marks the tax form complete because nothing told it otherwise, and does not call finance. Outcome check: the vendor master shows net 30 and an active payment profile without a callback record. Verdict: 3 of 5 items lost, two by omission from the packet and one by a stale read, and the task fails even though both agents reported success.

Counterexample: in the oracle variant, agent B receives the full ledger and the current record version, and keeps all 5 items in 4 of 5 illustrative repeats. The gap between the two variants is the handoff's share of the failure; the remaining miss belongs to the receiver.

Which records must a reviewer see before accepting a handoff verdict?

Every verdict in the pack should cite three things: the ledger item it scores, the trace span where the receiver acted, and the version of the business record that shows the result. A verdict missing any of the three is an opinion, and reviewers should send it back.

The proof of outcome lives in the system the task changed. For vendor setup that is the ERP vendor master and payment profile; for a support escalation it is the ticket's field history in a tool like Zendesk or Jira; for a sales handoff it is the opportunity record in Salesforce or HubSpot with its field audit trail. A transcript can explain why something happened, but only the record shows that it did.

Keep the human change log in the same timeline. Late edits by people are where stale-state failures come from, and they are often absent from agent traces entirely because the person changed the record directly in the business application.

Where do handoff tests mislead, and how do you keep them honest?

A handoff test built only from synthetic tasks tends to plant constraints that are easy to spot, so it can overstate retention. Real handoffs carry messier context: a terms change buried in an email thread, an approval given verbally and logged later, a permission that applies only to one vendor class.

Small packs also mislead. A pack with a handful of tasks per loss type can show a difference between two handoff designs that disappears on the next batch; report the number of tasks and runs beside every score, and treat single-digit counts as a smoke test.

Framework behavior moves. Default history passing, filter semantics and protocol task states are documented today, and a release can change them. Record the framework and protocol versions in each run so a later score drop can be traced to a change in the transport rather than the model.

Where do realistic handoff cases come from?

The best seeds for a handoff pack are records of people handing work to each other: ticket reassignments with their notes, shift handover logs, case transfers between teams, and the approvals and corrections that followed. They show which constraints people actually forget and which records prove the result.

A team building a handoff pack from real records can describe to SourceX the scope, volume, language, format, timing and licensing terms it needs; SourceX holds no ready-made stock and looks for U.S. businesses whose records fit. On the supply side, the company that owns the records approves the scope and terms and signs before any data is delivered.

Frequently asked questions

What is a multi-agent handoff evaluation?

It is a test of whether the context one agent passes to another is enough for the receiver to finish the task correctly. It plants requirements, permissions and shared state before the handoff and scores the receiver's actions and the end state afterward. A good one also logs the exact handoff payload so failures can be attributed.

Is passing the full conversation history enough context?

Not reliably. Full history can still contain a stale value that a person later changed in the business system, and a receiver may not treat an old message as a binding requirement. Some frameworks also let developers filter history, so the test should log what actually crossed rather than assume everything did.

How many tasks does a handoff evaluation pack need?

Enough per loss type to tell a real difference from noise, which depends on how often failures occur. Start with a small pack to debug the probes, then grow each loss type before comparing handoff designs. Always publish task counts and repeated-run counts next to scores.

How do you tell a bad handoff from a bad receiving agent?

Run a control where the receiver gets the full constraint ledger and current state, then compare it with the normal handoff. Failures that disappear in the control were caused by the handoff; failures that remain belong to the receiver. The logged packet then shows whether an item was omitted or ignored.

Can human handoff records be used to evaluate AI agents?

Yes, as seeds for realistic tasks. Ticket reassignments, shift notes and case transfers show which constraints get lost in practice and which records prove outcomes. They need rights review and de-identification before they leave the business that holds them.

Sources

  • In the OpenAI Agents SDK, handoffs are exposed as tools, the receiving agent sees the previous conversation history by default, input filters change what it sees, and guardrails apply only to the first and final agents. Source
  • The A2A specification groups related tasks and messages by contextId, lets clients reference related tasks, and defines input-required and auth-required task states. Source
  • OpenTelemetry's GenAI agent span conventions, in development status, define an invoke_agent operation and a stable gen_ai.agent.id attribute. Source
  • The MAST taxonomy groups multi-agent LLM failures into three categories, including inter-agent misalignment with information withholding and ignored input among its modes. Source
  • tau-bench grades episodes by comparing the final database state with an annotated goal state and proposes pass^k to measure reliability across trials. Source
  • The tau-bench repository reports leaderboard results as Pass^1 through Pass^4 for its airline and retail domains. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies