Skip to content

AI uses for records

Agent trajectory branching decisions: three valid paths, one task

By SourceX Editorial · Updated

10 sections

Short answer

Store one task with many trajectories, and mark where they diverge. Each trajectory keeps its observations, tool calls, tool responses and state snapshots linked by step IDs, and each branch point records the options, the evidence seen and the choice made. Paths count as equivalent only when explicit rules agree: same verified end state, required messages sent, mandated steps present in order and no extra writes.

Agent run

Agent trajectory branching

This page covers agent trajectory branching decisions: three valid paths, one task.Illustrative

Key takeaways

  • Make the task the unit and attach every valid path to it, with branch points marked.
  • Link each step to its parent, observation, tool response and resulting state so divergence is visible.
  • Judge equivalence with written rules that combine end state and path constraints, never one alone.
  • Keep counterexamples: a path with the right end state and a skipped confirmation must fail.
  • Ship seeds, snapshots and rules so a second team can replay a path and reach the same verdict.

How should trajectory data represent several valid paths through one workflow?

Treat the task, not the trajectory, as the unit, and attach every observed path to it with its branch points marked. A dataset that stores one golden path per task teaches a model that the other correct paths are wrong, and it gives an evaluator no way to accept them. A dataset that stores many paths without rules for comparing them is worse: nobody can say which differences matter.

So a branching trace dataset needs two things the usual export lacks. First, a trace format in which every step links to its parent, its tool response and the state it produced, so the point of divergence is visible. Second, a short set of written equivalence rules that decide whether two paths count as the same solution. The rules are what turn a pile of logs into training and evaluation data.

This page works through one case with three valid paths and two counterexamples. It assumes you already know the difference between agent trajectories and workflow logs; the question here is narrower: how to capture and judge branching.

Why a single reference path mislabels correct work

Reference paths are convenient to write and misleading to grade against. The tau2-bench documentation is explicit about this: a task's actions list is one reference trajectory that solves the task, not the only correct one, and any sequence of tool calls producing an equivalent database end state passes its database check. The same page notes that an agent can match none of the listed reference actions and still be fully correct.

The documentation also explains when path matching is justified. Its action-based reward treats the listed actions as the assumed-unique correct trajectory, and the authors reserve it for tasks where all valid solutions have actually been enumerated. That is the decision every trajectory dataset has to make per task: is there one acceptable path, or a set of them, and who has enumerated the set.

Outcome-only checks have the opposite weakness. AgentDojo's task interface includes a check over function-call traces precisely because some tasks leave no trace in the environment. A path that reaches the right state while skipping a mandatory step is outcome-equivalent and still wrong. Equivalence rules have to cover both the end state and the path constraints that matter.

Three kinds of divergence, and which ones matter

Not every difference between two traces is a branch worth recording. Sort divergences into three kinds before labeling, because each kind needs a different rule.

At a glance

Three kinds of divergence, and which ones matter

  • Order-only divergence: the same calls in a different order, typically reads that do not depend on each other. Look up the order then the customer, or the reverse. Usually equivalent; record the order but do not create a separate branch.
  • Route divergence: different tools or inputs that reach the same end state, such as authenticating by email versus by name and postal code. A real branch; record the branch point and the reason, then test equivalence with the rules below.
  • Outcome divergence: different end states that the policy accepts, for example a return versus an exchange when the customer could choose either. Not equivalent; these are separate acceptable outcomes, and the task must list them as such rather than forcing one to match the other.

Worked case (illustrative): one address change, three valid paths

The task: a customer asks to change the shipping address on a pending order. The rules come from the tau2-bench retail policy, which requires authenticating the user by email or by name plus zip code even when the user offers an ID, allows changing the shipping address of a pending order, and requires listing the action details and getting an explicit yes before any update. All records, IDs and steps below are illustrative.

Path A authenticates by email, fetches the order the customer names, states the new address, gets a yes and calls the address update. Path B authenticates by name and zip code because the customer has no access to their email, lists the user's orders, identifies the pending one with the customer, then confirms and updates. Path C starts the same as A, but the order details show the new address is already the user's default; the agent points this out, the customer still wants the order updated, and the agent confirms and updates. All three end with the same order record and the same confirmation sent.

Two counterexamples show what the rules must reject. Counterexample X follows path A but writes before the customer says yes; its end state is identical, so only a path rule catches it. Counterexample Y follows path B but updates every pending order for the user, not just the one discussed; its end state differs by extra writes, so the end-state rule catches it.

Worked case (illustrative): one address change, three valid paths
StepPath APath BPath C
AuthenticateFind user by emailFind user by name and zip codeFind user by email
Locate the orderGet order by the ID the customer givesGet user details, list orders, agree which oneGet order by ID; note address equals the default
Branch pointNone after authenticationBP1: email unavailable, so name and zip routeBP2: possible no-op, so ask whether to proceed
ConfirmState new address, receive yesState new address, receive yesExplain no-op risk, restate, receive yes
WriteOne address update on the orderOne address update on the orderOne address update on the order
Verified outcomeOrder address equals the requested valueSame as ASame as A

The branching trace record, field by field

Every step in every path gets one row with the fields below. The point is linkage: a reviewer must be able to go from a decision to the observation that justified it and to the state it produced. Where you already emit OpenTelemetry traces, map onto their names; the generative AI conventions define an execute_tool span carrying a tool call identifier, a conversation identifier and opt-in arguments and results, and a plan span for the phase in which an agent formulates a strategy before executing it.

The branching trace record, field by field
FieldContentWhy it is neededOpenTelemetry analog
case_id and episode_idTask identifier and one identifier per attemptGroups all paths for one taskgen_ai.conversation.id
step_id and parent_step_idUnique step, and the step it followsMakes divergence a tree, not a listSpan ID and parent span
actor and timeAgent, user, tool or human reviewer, with timestampSeparates who acted from what happenedSpan attributes and timing
observation_refPointer to what the actor saw before actingTies each decision to its evidenceInput messages, where recorded
actionTool name, call ID and arguments, or message textThe decision itselfexecute_tool span, gen_ai.tool.call.id
tool_response_refPointer to the stored responseLets a replay compare responsesgen_ai.tool.call.result, opt-in
state_hash_afterHash of the normalized state after the stepLocates where paths reconvergeNo direct analog; add as custom attribute
branch_point_idSet when options diverge, with options and choiceRecords why paths splitPlan span, where reported

Equivalence rules: when do two paths count as the same solution?

Write the rules per task family and apply all of them; a path is equivalent to the reference only if every rule passes. The table is the rule set used for the illustrative case, with the check, the evidence each check reads and what a failure looks like.

Equivalence rules: when do two paths count as the same solution?
RuleCheckEvidence readFails when
E1 End stateNormalized final state equals the target statestate_hash_after of last step, volatile fields excludedCounterexample Y: extra orders changed
E2 Required messagesEvery required fact was communicatedAgent messages in the traceCustomer never told the new address
E3 Mandated steps in orderAuthentication before reads; confirmation before the writeStep order and actor on each stepCounterexample X: write before yes
E4 No extra writesWrite count and targets match the task scopeAll write actions in the traceSecond address update, or default address changed unasked
E5 Read-order freedomIndependent reads may appear in any orderRead actions and their dependenciesNever fails on order alone
E6 Evidence before decisionEach branch choice follows the observation it citesbranch_point_id and observation_refAgent picks an order before listing orders
E7 Accepted outcomes listedDistinct acceptable end states are enumerated by the task ownerTask definitionA valid alternative outcome graded as failure

Can another team replay a branch from the shipped data?

Only if the dataset ships what a replay needs, and the test is to have someone else try. tau-bench's reward function shows the minimum: reload the initial data, replay actions, hash the resulting state and compare. AgentDojo's checks receive the environment before and after the run, which is the same idea. Gymnasium adds the detail people forget: a seed passed to reset initializes the random generator, and a later reset without a seed does not re-seed it, so the seed must be recorded per episode.

Ship, per task: the initial state snapshot or a loader with its version, the seed, the user instruction or simulator script, tool versions, the normalization used before hashing and the equivalence rules. Then have a team that did not build the data replay one path from each branch and one counterexample, and confirm they reach the same verdicts. Disagreements usually point to a missing normalization or a rule that lives only in someone's head.

Normalization deserves its own written note. Fields such as update timestamps, generated confirmation numbers and audit sequence IDs differ between two correct runs by design. List the excluded fields per table, keep the list under version control and hash only what remains; otherwise rule E1 fails every path except the one that produced the target.

Proof of a branch choice versus proof of the result

The proof of a decision is the observation it cites; the proof of an outcome is the state, not the message. A final reply saying the address is updated proves nothing, and neither does a status field set to resolved. The verified outcome is the order record after the write, read back by the grader, with its hash matching the target.

Branch points need their own proof. For BP1 in the worked case, the evidence is the customer turn saying they cannot get into their email; without it, path B is an unexplained detour. When a branch point has no linked observation, label it as unexplained rather than dropping it, because it is often where models and humans differ most.

Limits of equivalence rules

Rules work when the end state is crisp and the mandated steps are written down. They weaken where the outcome is a document, a judgment or a message whose quality varies; there, an equivalence decision needs an expert reviewer or an agreed rubric, and the label should say so. Rules also age: a policy change can turn a valid path into a violation, so version the rule set with the policy it encodes. Finally, enumerating accepted outcomes is a business decision. A task expert, not the data engineer, should sign off rule E7.

Coverage is a separate limit. Three observed paths do not mean three exist; they mean three were recorded. Report how many distinct branch points the dataset contains per task, how many episodes passed through each, and which branch points have only one observed choice. A buyer comparing datasets learns more from that count than from the raw number of trajectories, because a thousand copies of path A teach less than a handful of paths through each branch.

Getting real branching from operating records

Business systems already hold many paths through the same workflow: the same request handled differently by different staff, with case histories, audit trails and outcomes attached. Those histories show which branches occur and why, which a synthetic generator can only guess.

Multi-path workflow records can be requested from SourceX by describing scope, volume, format and licensing terms; it then sources them from U.S. businesses, because it keeps no inventory to sell.

Frequently asked questions

What is a branching decision in an agent trajectory?

It is a step where more than one action was reasonable and the agent or person chose one, causing the trace to diverge from other valid traces. A useful record stores the options, the evidence seen and the choice, linked to the step that followed.

Should a dataset keep only the shortest successful path?

No. Keeping only one path hides valid alternatives and teaches a model to treat them as errors. Keep every path that passes the equivalence rules, and mark which one is the reference only if the task truly has a single acceptable path.

What should the branching trace example and equivalence rules include?

A trace in which each step links to its parent, observation, tool response and resulting state, plus written rules for end state, required messages, mandated step order, extra writes, evidence before decisions and the list of accepted outcomes. Include at least one counterexample per rule.

Is matching the final state enough to call two paths equivalent?

Not on its own. Two paths can reach the same state while one skips a required confirmation or authentication step, so path rules must be checked as well.

How do OpenTelemetry traces help?

They give a standard place for tool call identifiers, conversation identifiers and parent-child relationships between spans. You still need to add state hashes and branch-point records, which the conventions do not define.

Sources

  • tau2-bench documentation states that a task's actions list is one reference trajectory, not the only correct one, that equivalent database end states pass, and that action-matching reward assumes the listed path is the unique correct one. Source
  • The tau2-bench retail policy requires authenticating the user by email or by name plus zip code even when an ID is given, allows changing a pending order's shipping address, and requires explicit confirmation before updates. Source
  • tau-bench's reward function reloads data, replays the reference actions, hashes the resulting state and compares it with the state the agent produced. Source
  • AgentDojo user tasks check utility using the environment before and after execution, with an optional check over function-call traces for tasks that leave no trace in the environment. Source
  • AgentDojo's task documentation illustrates a trace-based check for a task whose increments and decrements leave counters unchanged before and after execution. Source
  • The OpenTelemetry GenAI conventions define an execute_tool span with tool call identifier, conversation identifier and opt-in tool call arguments and result. Source
  • The OpenTelemetry GenAI agent conventions define a plan span for the phase in which an agent formulates a strategy before executing it, reported only when instrumentation can distinguish planning. Source
  • Gymnasium's reset initializes the random generator from a passed seed and does not re-seed it when later called without a seed. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies