Skip to content

AI uses for records

Agent trajectory tool response preservation: why each action needs its result

By SourceX Editorial · Updated

10 sections

Short answer

An agent trajectory should link every tool response to the action that requested it because the response is the evidence the next decision was based on. Without it, a reviewer cannot tell a sound decision from a lucky one, separate tool errors from agent errors, replay the episode or verify the outcome. Keep the call ID, full response or a hashed pointer, error flag and resulting state on each step.

Agent run

Agent trajectory tool

This page covers agent trajectory tool response preservation: why each action needs its result.Illustrative

Key takeaways

  • The tool response is what the agent saw; a trajectory without it records choices but not the evidence behind them.
  • Model APIs and the Model Context Protocol already pair each call with its result by ID, so the link exists at run time and is usually lost during export.
  • Error results matter as much as successful ones, because they explain retries and changed plans.
  • A trajectory is complete when a second team, given only the record, can reconstruct each decision and verify the final outcome.
  • A status field or the agent's final message is not proof of the outcome; the record should link to the state that proves it.

Because the tool response is the evidence the agent's next decision rests on. An action list shows what the agent did; only the paired responses show what it knew when it did it. Strip them and every later step loses its justification.

The ReAct line of work made this loop explicit: the model alternates reasoning with actions, and each action brings back an observation from the environment that feeds the next reasoning step. A tool-calling agent works the same way. When a record keeps the action but not the observation, a reviewer can no longer check whether the agent read the result correctly, ignored it or acted on something it never received.

Four practical uses depend on the pairing. Grading a process, not just an end state, needs to know what the agent saw. Debugging needs to separate a tool that returned bad data from an agent that misread good data. Replay needs the recorded responses to stub tools that cannot be called again. Training on the trajectory needs the observations, or the model learns actions without the conditions that made them right.

The model APIs already pair every call with its result

Every major tool-calling interface links a result to its call through an identifier, so the pairing is available at run time. A dataset that lacks it lost it later, in logging or export.

In the Claude API, each tool use block carries an ID that is used to match up results; the result block names that ID in tool_use_id, can set is_error when the tool failed, and must immediately follow the tool use in the message history. OpenAI's function calling uses tool_call_id on tool messages in Chat Completions and call_id on function_call_output items in the Responses API. The Model Context Protocol adds a distinction worth preserving: protocol errors, such as an unknown tool, are JSON-RPC errors, while tool execution errors, such as a failed API call or invalid input, come back as results with isError set to true, and clients should pass those to the model so it can correct itself.

The practical consequence for buyers is simple. Ask for the native call ID on every action and every response, and ask for error results to be kept as results, flagged as errors, rather than dropped or merged into a generic failure message.

Parallel calls make the identifier essential rather than convenient. When an agent requests several tools in one turn, results can return in any order, and position in the log no longer says which result belongs to which request. Only the call ID keeps the mapping intact once records are exported, merged and sorted.

Where tool responses disappear between the run and the dataset

Responses are usually lost in the pipeline after the run, not during it. Each loss point below is common, and each one leaves a record that looks complete at a glance.

At a glance

Where tool responses disappear between the run and the dataset

Ask the supplier which of these steps their pipeline performs and test for each one in a sample. A pipeline description alone is not enough, because most of these losses are side effects of tools configured for monitoring or storage costs, not deliberate choices anyone remembers making.

  • Logging only assistant turns: the export keeps what the model said and called, but not the tool messages in between.
  • Observability defaults: in the OpenTelemetry GenAI conventions, the execute_tool span records gen_ai.tool.call.id, but tool call arguments and results are opt-in attributes marked as possibly sensitive, so a default setup can carry the ID without the content.
  • Dropping failed calls: error results are filtered as noise, so a later retry appears to come from nowhere.
  • Silent truncation: long responses are cut to a character limit with no flag, so the part the agent relied on may be missing.
  • Whole-response redaction: a privacy pass deletes the entire response instead of the personal fields inside it.
  • Re-sorting or deduplication: steps are ordered by timestamp across parallel calls or identical calls are merged, and the call-to-result mapping breaks.
  • Final status only: the export keeps the last system status and discards the intermediate state changes that explain it.

A paired trajectory: the same invoice task with and without tool responses

This worked case is illustrative. The task, tools and values are invented to show the method, and the tool names are generic rather than any vendor's API. An accounts payable agent is asked to decide whether supplier invoice INV-4471 can be paid. The policy says to pay only when invoice, purchase order and goods receipt quantities match. The table shows each step as the complete record holds it, what a stripped record keeps, and what a reviewer can no longer tell.

  • From the stripped record, a reviewer sees five calls, a hold and a polite message. The same record would look identical if the receipt had shown 50 units and the agent had held a payable invoice by mistake.
  • The error at step 3 is the clearest loss. Without it, step 4 looks like a redundant call and a grader might penalize it, when it was the correct recovery from a timeout.
  • The counterexample to keep in mind: had step 5 returned an error that the agent ignored, its final message could still claim the hold was placed. Only the linked response would show that it was not.
A paired trajectory: the same invoice task with and without tool responses
Step and actionComplete record: linked tool responseStripped record keepsMissing context (highlighted)
1. get_invoice(INV-4471)Call c1: 50 units billed, references PO-2209Action onlyWhy the agent fetched PO-2209 next
2. get_purchase_order(PO-2209)Call c2: 50 units ordered, status openAction onlyThat the order quantity matched the invoice
3. get_goods_receipt(PO-2209)Call c3: is_error true, receiving service timed outNothing, the failed call was filteredWhy step 4 repeats the same call
4. get_goods_receipt(PO-2209)Call c4: 40 units received, 10 pendingAction onlyThe quantity shortfall that drives the decision
5. set_invoice_status(INV-4471, hold)Call c5: status changed to on hold, hold reason recordedAction and final status on holdWhether the hold was justified or a guess
6. send_message(requester)Call c6: delivered, message cites 10 pending unitsFinal message textWhether the message matches the data
Outcome checkLater receipt record: remaining 10 units received, hold released, invoice paidStatus on hold at episode endProof that the hold was the correct decision

The fields every step record should carry

Ask for a step record that keeps the call, the response and the resulting state together, with identifiers that let any field be joined back to its source. This list is an editorial specification for tool-calling agents; adapt the names to your own schema.

At a glance

The fields every step record should carry

Two choices in this list deserve a decision up front. First, inline payloads or pointers: inline is simpler to audit, while pointers keep files small and let sensitive content sit under separate access control. Second, how retries are linked: a retry should name the failed call it repeats as its parent, so a grader can see the recovery as one decision rather than two unrelated calls.

The fields every step record should carry
FieldPurposeAcceptance check
Case ID, step ID and parent stepOrder steps and show which call a retry or follow-up belongs toNo orphan steps; parallel calls share a parent
Actor and timestampSeparate agent, human, simulated user and system events in timeEvery step has one actor and a monotonic time
Tool name, version and argumentsRecord exactly what was requested, against which tool definitionArguments validate against the logged tool schema
Native call IDJoin the action to its response as the API didEvery call ID has exactly one response
Response payload or pointerKeep what the agent saw, inline or as a hashed reference to stored contentPointer resolves and hash matches
Error flag and error classSeparate execution errors from protocol errors and successesErrors are present as steps, not dropped
Truncation and redaction flagsShow when content the agent saw is not fully in the recordFlags set whenever length or content changed
State change and artifact linksLink records created or modified by the callEach write action links to the changed record
Environment and policy versionTie the step to the tools, data and rules in forceVersion present on the episode header
Outcome verification recordLink the final state, or a later record, that proves success or failureOutcome cites evidence, not a status field alone

The handoff test: can an outside team rebuild every decision from the record?

The test of a complete trajectory is whether a team that was not present can rebuild every decision from the record alone. Give a second team the trajectory, the tool definitions and the policy, and nothing else. Ask them to state, for each step, what the agent knew, why the next action followed, and whether the final outcome was correct.

Existing agent tooling shows what is needed for this. SWE-agent's trajectory files keep, for each step, the raw model response, the parsed thought and action, the observation the action produced, extracted environment state and the exact input shown to the model. TheAgentCompany's evaluation documentation goes further for trajectories used in grading: whatever their format, they must record all steps conducted by the agent or human being evaluated, because graders use them to award partial credit.

Score the reconstruction step by step. A step passes when the second team can cite the linked response that justifies the next action. Any step they cannot justify points to a missing or broken link, and the trajectory should not be used for process grading or training until it is repaired. For episodes that must also be replayed, the recorded responses become the stubs, so a missing response blocks replay as well.

Proving the outcome: state changes and confirming records, not closing messages

The proof is the state the tools changed and any later record that confirms it, not the agent's final message or a status field. Messages can be wrong and statuses can be set without the work being done.

Benchmarks built on tool use grade this way. In the tau-bench reference code, an exception raised by a tool becomes an observation that begins with Error, so the agent sees the failure, and the task reward is computed by comparing a hash of the final data with the state produced by the gold actions; a confident closing message earns nothing if the data is wrong.

For trajectories derived from business records, the outcome record often arrives later than the episode, like the receipt that cleared the hold in the example. Link it explicitly, state the observation window, and mark episodes where no confirming record exists yet as unverified rather than successful.

Run a mechanical audit on every delivery before anyone reads an episode, because broken links are cheap to detect by machine and expensive to find by hand. The checks below take the step record above as their input and produce counts a supplier can be asked to fix.

At a glance

Auditing a delivered trajectory set for broken call-to-result links

Treat the results as acceptance criteria rather than quality scores. A delivery where call IDs do not join is not a slightly worse dataset; for process grading, replay and training it is a different and much weaker product, and the request or agreement should say so before records are prepared.

  • Join test: match every action to a response by call ID and count actions with no response and responses with no action. Both counts should be zero, apart from steps explicitly marked as cut off by the episode end.
  • Order test: confirm each response appears after its call and before any action that depends on it, including when the agent made parallel calls in one turn.
  • Error presence: compare the share of steps flagged as errors with what the source system's own logs show; a share of zero across thousands of tool calls usually means errors were filtered out.
  • Payload test: resolve a random sample of pointers, recompute their hashes and confirm that truncation and redaction flags are set wherever stored content is shorter than the logged length.
  • Outcome test: for a random sample of episodes, follow the outcome verification link to the confirming record and check that it supports the label given.
  • Reconstruction sample: hand a small random set of episodes to a reviewer outside the producing team and record the share of steps they can justify from the linked responses.

When a response is too large or too sensitive to store inline, keep the call ID, a content hash and a pointer, and redact fields inside the payload rather than deleting it. The link is what makes the step usable; the payload can be stored and access-controlled separately.

The OpenTelemetry conventions flag tool arguments and results as attributes that may contain sensitive information, and Anthropic's tool-use documentation warns that tool results often carry content from outside sources, such as web pages or inbound email, that should be treated as untrusted. Both points argue for the same handling: record what the agent saw, label where it came from, and apply privacy review to the stored payload.

Redaction must be visible. Set a flag on every step where content was removed or shortened, and keep the placeholder type, such as an account number or a person's name, so a reviewer knows what kind of information the agent had.

Requesting trajectories built from business systems

If your trajectories will come from real operational work, such as ticket, billing or procurement systems, write the step record and the outcome-verification rule into the request, because the system history that proves outcomes has to be selected before records are prepared and de-identified.

SourceX prepares licensed datasets from the operational records of U.S. businesses and sources them to order, not from stock, so a request can specify linked tool responses, error steps and outcome records alongside scope, volume, format, timing and licensing needs.

Frequently asked questions

What is an agent trajectory?

An agent trajectory is the step-by-step record of one task attempt: each observation, the action the agent chose, the tool response it received and the resulting state, ending in an outcome. For tool-calling agents, the tool responses are the observations. A trajectory without them is a list of actions.

Why keep failed tool calls in a trajectory?

Failed calls explain retries, changed plans and recoveries that would otherwise look like redundant or random steps. The Model Context Protocol reports tool execution errors as results with isError set to true, and keeping them as flagged steps lets graders distinguish a good recovery from wasted work.

What should the paired trajectory example with missing context highlighted include?

It should show one task recorded twice, once with each tool response linked to its action and once stripped, with the context lost at each step called out. Include at least one error result, one decision that depends on a response value and an outcome check that relies on a record outside the episode.

Can tool responses be summarized instead of stored in full?

Only if the summary is labeled as such and the full response remains retrievable through a pointer and hash. A summary can omit the exact value the agent relied on, which defeats grading and replay. Truncation and summarization should always be flagged on the step.

How do you check a trajectory dataset for preserved tool responses?

Join every action to a response by call ID and count orphans in both directions, confirm error results are present, check that truncation flags are set where content was shortened, and have a second team reconstruct a random sample of episodes from the record alone.

Sources

  • In the Claude API, tool results reference the tool use ID, can be flagged as errors, must immediately follow their tool use, and often carry untrusted outside content. Source
  • OpenAI function calling links tool results to calls through tool_call_id in Chat Completions and call_id in the Responses API. Source
  • The Model Context Protocol reports tool execution errors as results with isError true, separate from protocol errors, and clients should pass them to the model. Source
  • OpenTelemetry GenAI execute_tool spans record the provider's tool call ID, while tool arguments and results are opt-in and may contain sensitive information. Source
  • SWE-agent trajectory steps store the model response, thought, action, observation, environment state and exact model input. Source
  • tau-bench turns tool exceptions into error observations and grades by comparing final data state with the state produced by gold actions. Source
  • TheAgentCompany uses trajectories to grant partial credit and requires them to record all steps conducted by the examinee. Source
  • ReAct prompts language models to interleave reasoning traces with actions that return observations from external sources and environments. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies