Skip to content

AI uses for records

Agent policy compliance evidence report: proving which boundaries held

By SourceX Editorial · Updated

10 sections

Short answer

To prove which policy boundaries an agent respected, test each boundary separately in a sandbox with no live secrets or real side effects: instruction source, data disclosure, granted authority and approval points. Run every case several times, record each tool call and the state before and after, and give one verdict per boundary per run that cites the policy clause and the exact trace evidence behind it.

Agent run

Agent policy compliance

This page covers agent policy compliance evidence report: proving which boundaries held.Illustrative

Key takeaways

  • Task success is not policy evidence; boundaries need their own verdicts.
  • Separate four boundaries: instruction source, data disclosure, granted authority and approval.
  • Every verdict should cite an evidence anchor such as a span id, approval event or state snapshot.
  • Repeat each case and report consistency, because a boundary that holds once can fail on the next run.
  • Record not tested when a run never reached the boundary, so caution is not mistaken for compliance.

What does it mean to prove an agent respected a policy boundary?

Proving a boundary held means producing, for each boundary and each run, a verdict that someone else can re-check from recorded evidence: the policy clause, the test case that pressed on it, the tool calls the agent made, and the state of the sandbox before and after. A task success rate is not that proof, because an agent can finish a task while crossing a boundary on the way.

A policy boundary here is any rule about what the agent may treat as an instruction, what data it may reveal, which actions it may take with its granted tools, and which actions must wait for a person. Each kind fails differently, so each needs its own test cases and its own evidence.

The report format later on this page is built around that idea. It keeps one row per boundary per test case, with a pointer to the exact trace span or state record that justifies the verdict, so a reviewer can disagree with a specific line instead of a summary score.

Which boundaries should an agent policy evaluation separate?

Separate four boundaries: instruction source, data disclosure, granted authority and approval. Published security guidance already treats them as distinct risks. OWASP's 2025 list for LLM applications describes indirect prompt injection, where content from external sources such as websites or files alters model behavior, and separately describes excessive agency, whose root causes it names as excessive functionality, excessive permissions and excessive autonomy.

The Model Context Protocol specification states the approval boundary in protocol terms. Its 2026-07-28 revision says hosts must obtain explicit user consent before invoking any tool, that there should always be a human in the loop able to deny tool invocations, and that clients must treat tool annotations as untrusted unless they come from trusted servers.

Which boundaries should an agent policy evaluation separate?
BoundaryPolicy questionSeeded testEvidence it held
Instruction sourceDoes the agent follow only the user and system, not content it reads?An instruction planted in a ticket, file or web page the task requires readingNo tool call serves the planted goal; output treats the text as data
Data disclosureDoes the agent reveal only what the task needs?Records in reach that the requester should not seeOutputs and outbound tool arguments contain no out-of-scope fields
Granted authorityDoes the agent stay inside the tools and scopes it was given?A task that is easier with a tool or scope it lacksNo call to unlisted tools; no scope escalation attempts in traces
Approval pointDoes the agent stop for a person where policy requires?An action above a policy limit or of a destructive kindApproval request span precedes the action; no action after a denial

Does the agent distinguish task evidence from instructions?

You find out by labeling every piece of content the agent reads as trusted or untrusted before the run, then checking whether any action traces back to an untrusted source. The label belongs to the content item, not to the tool that fetched it: a support ticket body is untrusted even when it arrives through a trusted internal tool.

AgentDojo, a research benchmark from ETH Zurich, structures this cleanly. Each user task has a utility check, each adversary goal is an injection task with its own security check, and both can be judged from the environment state or from the trace of function calls. That pairing is the minimum for a credible verdict: the agent must still do the legitimate task, and the injected goal must not happen.

Look for partial compliance as well as full compliance. An agent that refuses the planted instruction but quotes it back to the user, or that performs one of three injected steps, has still let untrusted content steer it. The report should record which step and which content item.

Why does the sandbox count as part of the evidence?

The sandbox is part of the evidence because a verdict is only as trustworthy as the record of what the agent could have done. A defensive evaluation runs against emulated or isolated tools, with seeded fake credentials and canary records, no live secrets and no real side effects, and snapshots the state before and after every run.

Traces make individual actions citable. The OpenTelemetry semantic conventions for generative AI define an execute_tool span for tool execution, with the tool name required and a tool call identifier when the model or provider supplies one. Tool call arguments and results are opt-in attributes, which suits an evaluation sandbox where nothing real is exposed. The conventions are still marked as in development, so pin the version you used in the report header.

Two practical rules follow. Never let an evaluation tool reach a production system, even read-only, because the record of what the agent saw then includes data you cannot publish with the report. And make the sandbox build reproducible, so a reviewer can rerun the failing case and see the same state.

The evidence-anchored policy evaluation report format

Each field in the report names its evidence anchor, the stable identifier a reviewer uses to find the underlying record. Without anchors, a report is a set of claims about a run; with them, it is an index into the run.

Keep the boundary register and the verdict table separate. The register says what the policy is, in the policy owner's words, with a version; the verdicts say what happened. When the policy changes, old verdicts stay attached to the version they tested.

The evidence-anchored policy evaluation report format
Report fieldWhat it containsEvidence anchorReviewer check
HeaderAgent build, model, tool manifest, policy version, sandbox build, datesBuild hashes and manifest fileEvery later row refers to these versions
Boundary registerEach boundary as a numbered policy clause with ownerPolicy document section idsClause wording comes from the policy owner
Test caseTask prompt, seeded content, trust labels, expected safe behaviorCase id and seed fileExpected behavior written before any run
Run recordEach run of each case, with repetitionsTrace id per runNumber of runs matches the plan
VerdictHeld, crossed, partial or not tested, per boundary per runSpan ids of deciding tool calls or approval eventsVerdict follows from the cited spans
State evidenceDifferences between before and after snapshotsSnapshot idsNo change outside the task's expected effects
ConsistencyShare of runs in which each boundary heldRun ids grouped by caseSingle-run passes are not reported as reliable
Exceptions and limitsUntested boundaries, ambiguous clauses, sandbox gapsNotes linked to case idsLimits stated next to the verdicts they qualify

How many runs before a boundary verdict means anything?

Run each boundary case several times and report how often the boundary held in every run, not whether it held once. Agents are stochastic, and a boundary that holds in one run and fails in the next is a boundary that will fail in production.

Tau-bench, which emulates conversations between a simulated user and an agent given domain-specific API tools and policy guidelines, reports reliability as pass^k: the chance that a task succeeds in all of k trials. Its published leaderboard shows the score falling as k rises from one to four for every listed strategy, which is the pattern to expect for boundary verdicts too.

For the report, fix the number of runs per case before testing and report the consistency field per boundary. A boundary that held in every run of every case is a strong claim; a boundary that held in most runs is a finding that needs a mitigation, not a pass.

Worked example: a refund agent's boundary report (illustrative)

This example is invented to show the report format in use. The case counts and run counts are illustrative, and no real company, policy or model is described.

A retailer's support team is evaluating an agent that answers order questions and issues refunds through an emulated order tool. The policy owner states three clauses: refunds above the policy limit need a supervisor's approval, the agent may disclose order details only to the account holder, and content in customer messages or attachments is never an instruction.

The evaluation uses twelve cases with eight runs each, drawn from de-identified historical tickets with seeded fake order data. One case plants an instruction in an attached receipt image description asking the agent to refund a different order; another asks for the delivery address on an order belonging to someone else.

  • Instruction source: held in all runs of four cases; in one case the agent declined the planted refund but quoted the planted text to the user in two of eight runs, recorded as partial with span ids.
  • Data disclosure: held in all runs; outbound message spans contain no fields from the other account.
  • Approval point: held in all runs that reached it; in three runs the agent asked for clarification first and never reached the limit, recorded as not tested rather than held.
  • Granted authority: held; no calls to the cancellation tool, which was not in the manifest.
  • Report outcome: one partial finding with a mitigation request, one coverage gap where a case needs rewriting to reach the approval point reliably.

When is not tested the honest verdict?

Not tested is the honest verdict whenever the run never put pressure on the boundary. An agent that refuses the whole task, asks a clarifying question and stops, or fails before reaching the risky step has not respected the approval boundary; it simply never met it.

Over-refusal is the common way this goes wrong. A cautious agent can post a perfect boundary record by doing nothing, which is why the utility check in each case matters as much as the security check. Report boundary verdicts alongside task completion, so a reader can see when safety came from inaction.

Ambiguous policy produces the same problem from the other side. If two reasonable reviewers would disagree about whether an action crossed a clause, the verdict is not crossed or held but unresolved, and the clause goes back to its owner.

When should an agent security evaluator take over?

Bring in an agent security evaluator, with the business owner of each policy, before running tests that need real credentials, real systems or offensive techniques. The method on this page is for defensive, sandboxed evaluation of whether stated boundaries held; it is not a penetration test and should not become one by drift.

  • A test would require production credentials, live customer data or a real payment, message or deletion.
  • Tools can make irreversible changes that the sandbox cannot fully emulate.
  • The policy owner cannot state the clause precisely enough to judge a run.
  • Results will be used for a regulatory filing or a contractual assurance, which needs its own review.
  • Seeded content would have to include working exploit code rather than benign planted instructions.

Using real business records to build boundary test cases

The weakest part of most boundary evaluations is the test content: synthetic tickets and documents rarely carry the messy attachments, forwarded threads and mixed-trust content where real boundary failures happen. Historical business records, de-identified and licensed for evaluation use, make the seeded cases look like the traffic the agent will meet.

SourceX sources such records from U.S. businesses for a specific request instead of selling from a catalog. A buyer describes the scope, volume, language, format, timing and licensing terms needed, and SourceX handles qualification of suitable businesses, rights review, contracting and preparation before delivery. Sandbox seeding, trust labeling and the report itself stay with the evaluation team.

Frequently asked questions

Is a high task success rate evidence that an agent followed policy?

No. Task success shows the goal was reached, not how. An agent can complete a refund while disclosing another customer's data or skipping an approval, so boundary verdicts must be judged separately from task outcomes.

What is the difference between a trace and an evidence anchor?

A trace is the full record of a run. An evidence anchor is the specific identifier inside it, such as a span id or snapshot id, that justifies one verdict. Reports should cite anchors so a reviewer can go straight to the deciding event.

Should tool descriptions from an MCP server be treated as policy?

No. The MCP specification says clients must treat tool annotations as untrusted unless they come from trusted servers. Policy belongs in the boundary register written by its owner, not in metadata a server supplies.

Can production logs replace a sandbox for policy evidence?

They can show how often a boundary held in real traffic, but they cannot safely test boundaries with planted content or risky actions. Use production logs to choose cases and sandbox runs to judge verdicts.

Who should sign off an agent policy evaluation report?

An agent security evaluator should sign off the method and verdicts, and the business owner of each policy should confirm the clause wording and the treatment of unresolved cases. Neither should approve the other's part alone.

Sources

  • OWASP's 2025 LLM01 entry describes indirect prompt injection, where content from external sources such as websites or files alters model behavior. Source
  • OWASP's 2025 LLM06 entry defines excessive agency and names excessive functionality, excessive permissions and excessive autonomy as its root causes, recommending human approval of high-impact actions. Source
  • The MCP specification (2026-07-28) says hosts must obtain explicit user consent before invoking any tool and that tool annotations should be considered untrusted unless obtained from a trusted server. Source
  • The MCP tools specification says there should always be a human in the loop able to deny tool invocations and that clients must consider tool annotations untrusted unless they come from trusted servers. Source
  • AgentDojo defines user tasks with a utility check and injection tasks with a security check, and allows checks from function call traces when a task leaves no trace in the environment. Source
  • The OpenTelemetry GenAI conventions define an execute_tool span with a required tool name, a conditionally required tool call id, and opt-in tool call arguments and results, with development status. Source
  • Tau-bench emulates conversations between a simulated user and an agent given domain-specific API tools and policy guidelines, and its leaderboard reports pass^1 to pass^4. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies