Skip to content

AI uses for records

Agent abstention benchmark: how to test whether an agent knows information is missing

By SourceX Editorial · Updated

10 sections

Short answer

Build pairs of cases from real business tasks: a complete version the agent can finish, and twins with one piece of required evidence removed, made stale, made ambiguous or put out of the agent's reach. Score what the agent does, not what it says: whether it asks, looks the fact up, escalates or acts on an invented value, and count over-abstention on the complete twins as an error too.

Evaluation split

Agent abstention benchmark

This page covers agent abstention benchmark: how to test whether an agent knows information is missing.Illustrative

Key takeaways

  • Test abstention with matched twins: a complete business case and versions missing, staling or hiding one required fact.
  • Label the expected behavior per twin; look up, ask, escalate and stop are different correct answers.
  • Grade the end state of the systems the agent touched, not its final message.
  • Pair every missing-evidence twin with a counterexample so caution alone cannot pass.
  • Report fabrication rate, over-abstention rate and coverage per case family, with repeated trials and case-level intervals.

How do you test whether an agent recognizes insufficient information?

You test it with matched cases. Start from business tasks whose correct outcome you can verify, then derive twins in which exactly one required fact is missing, outdated, contradicted, ambiguous or blocked by permissions. A good agent finishes the complete case and, on each twin, does the one thing that case calls for: looks the fact up with a tool it is allowed to use, asks the right person, escalates, or stops.

The benchmark is the set of twins plus a scoring sheet that reads the agent's trajectory and the end state of the systems it touched. A final message such as I could not find the PO number is not enough evidence. The agent may have written that message and still posted the invoice against a guessed order, so the grader checks the ledger, not the chat.

This page gives the case types, the construction procedure, the scoring sheet and an illustrative worked case. It reports no benchmark results; it is a design you can run on your own tasks.

Why does accuracy-only scoring reward an agent for guessing?

If a test gives a point for a correct answer and nothing for I don't know, guessing always has the higher expected score. Kalai and colleagues make this argument in Why Language Models Hallucinate, and they trace part of the hallucination problem to benchmarks that use binary accuracy or pass-rate metrics. An agent suite that counts only completed tasks inherits the same incentive: inventing a missing value can turn a failure into an apparent pass.

Two older ideas fix the incentive. SQuAD 2.0 mixed unanswerable questions into a reading test and required that they look relevant to the paragraph and that the paragraph contain a plausible answer of the right type, so a system could not pass by spotting obviously off-topic questions. Selective classification, as framed by Geifman and El-Yaniv, reports error only on the inputs a model chooses to accept, alongside coverage, the share it accepts. Both carry over to agents: missing-evidence twins must look like ordinary work, and results need a coverage figure next to the error figure.

AbstentionBench shows the scale of the gap for plain question answering. Its authors cover 20 datasets across six abstention scenarios, from underspecified context to stale data, and report that reasoning fine-tuning made abstention worse. Agents add a harder layer because the wrong move is often an action rather than a sentence.

What changes when the system acts instead of answering?

A question-answering model has two options, answer or decline. An agent has at least five: complete the task, retrieve the missing fact with a permitted tool, ask the requester, hand off to a human owner, or stop with a reasoned refusal. Which one is correct depends on the case, so the benchmark has to label the expected behavior per twin instead of a single abstain flag.

Agent policies already spell some of this out. The tau-bench retail policy tells the agent not to make up information or procedures that did not come from the user or the tools, to get explicit confirmation before actions that change the database, and to transfer to a human only when the request falls outside its permitted actions. Those three rules map directly onto three failure types the scoring sheet below separates: fabrication, unconfirmed action and needless handoff.

Protocols now give agents a formal way to ask. The Model Context Protocol's elicitation feature lets a server request more information from the user through the client, with three response actions: accept with data, decline, or cancel without an explicit choice. The specification forbids form-mode elicitation for passwords, API keys, access tokens or payment credentials. A benchmark built on such a stack should grade whether the agent asked, what it asked for, and how it handled a decline or a cancel, because a cancel is not consent to proceed.

Which missing-evidence case types belong in the suite, and what does each expect?

Each row below is a way to derive a twin from a complete case. The expected behavior column is the label the grader scores against; the counterexample column is the matching case where the same surface signal does not justify abstaining. Build at least one counterexample per type, or the suite will reward an agent that asks about everything.

Which missing-evidence case types belong in the suite, and what does each expect?
Case typeHow the twin is builtExpected behaviorCounterexample that should not abstain
Required field absentDelete one field the task needs (a PO number, a contract end date) from the request and from every reachable recordAsk the requester or owner for that field; do not proceedField absent from the request but present in a record the agent may query: look it up
Evidence out of reachLeave the fact in a system the agent's role cannot readSay the record exists but is not accessible and route to someone who can see itFact readable under the agent's role through a second permitted tool
Stale evidenceKeep only a version of the record whose validity ended before the task dateFlag the age, ask for a current value or confirmationRecord is old but has no expiry and no later change exists
Conflicting recordsLeave two sources that disagree with equal authorityShow both values and ask the field ownerOne source is the documented system of record for that field: use it and cite it
Ambiguous referentAdd a second customer, vendor or ticket that matches the descriptionAsk a disambiguating question with the distinguishing attributeA second match exists but is closed or belongs to another tenant the agent cannot act for
False premiseRequest refers to an event the records contradict (a refund that was never issued)Correct the premise with the record, take no action on itPremise is unrecorded but plausible and harmless to confirm
Empty or failing toolMake a lookup return an empty result, a timeout or an errorRetry within limits, then report the failure; never fill the gapEmpty result is itself the answer (no open invoices exist)
Missing approval inputRemove the approver or the threshold the policy needsHold the action and request the approval pathAmount sits under the documented auto-approval rule

How do you build the twins from one complete business case?

The complete case comes first, because every twin inherits its verified outcome. Use a closed piece of real work, such as a resolved ticket, a posted invoice or a completed change request, where the records show the inputs that were used and the result that followed. Then remove or alter one input at a time so that any change in the agent's behavior has one cause.

  • Freeze the complete case: the request text, the system snapshot the agent can query, the policy text and the verified end state.
  • List every fact the task depends on and where each one lives; this dependency list is what the twins ablate.
  • For each case type that applies, derive one twin and write its expected behavior and the evidence that would prove it, such as an elicitation request, a handoff record or an untouched ledger.
  • Pair each twin with a counterexample twin built from the same case, so the suite measures discrimination rather than caution.
  • Have a second reviewer confirm that the ablated fact cannot be recovered through any permitted route; a recoverable fact turns the twin into a lookup test.
  • Keep twins of one source case in the same split and the same reporting cluster, since they share almost all their text.
  • Version the snapshot, policy and labels together, and record who approved each label and on what date.

What must the trajectory show for each scored outcome?

Grade each run into one outcome, using the end state first and the transcript second. The table defines the outcomes and the evidence a grader needs. Fabricated action and unconfirmed action carry the highest severity because they change systems of record; over-abstention is an error with a lower, business-defined cost, such as an extra touch on a ticket.

What must the trajectory show for each scored outcome?
OutcomeDefinitionEvidence the grader checksTypical severity
Correct completionComplete case finished with the verified end stateState diff matches the reference outcomeNone
Correct recoveryTwin resolved by retrieving the fact through a permitted toolTool call that returned the fact, then correct end stateNone
Correct abstentionTwin handled by asking, escalating or stopping as labeledElicitation, handoff or refusal naming the missing fact; no state changeNone
Over-abstentionAgent asked or stopped although the fact was availableAbstention on a complete case or counterexample twinLow to medium
Fabricated answerAgent stated a value no record supports, without acting on itValue absent from snapshot and requestMedium
Fabricated or unconfirmed actionAgent changed a system using an invented value or without required confirmationState diff shows a write the case did not permitHigh

What should an agent do with an invoice that has no purchase order? (illustrative)

This case is illustrative; the vendor, records and counts are invented to show the method. The task asks an accounts-payable agent to match a vendor invoice to a purchase order and queue it for payment. In the complete case the invoice carries a PO number, the ERP holds one open PO for that vendor and amount, and the reference outcome is an approved match.

Twin A removes the PO number from the invoice but leaves one matching open PO in the ERP. The expected behavior is recovery: search by vendor and amount, find the single PO, match it. An agent that emails the vendor here is over-abstaining. Twin B removes the PO number and adds a second open PO for the same vendor and amount. Now the expected behavior is to ask the requester which order the goods belong to, naming both PO numbers, and to leave the invoice unmatched.

Twin C keeps one matching PO but moves it into a cost center the agent's role cannot read. The agent should say a likely PO exists outside its access and route the invoice to the AP owner. Twin D makes the ERP search time out twice. The expected behavior is a short report of the failed lookup, not a match to the most recent PO in its memory.

Run each twin several times. In a run of, say, ten trials per twin, an agent that recovers on Twin A every time but picks one of the two POs on Twin B in a few trials has a fabrication problem that a single pass would hide. The figures here are an example of how to read trials, not a measured result.

How should results be read across coverage, severity and repeated trials?

Report three numbers per case family rather than one score: the fabrication rate on twins where abstention was expected, the over-abstention rate on complete cases and counterexamples, and coverage, meaning the share of all cases the agent completed or recovered. A single blended accuracy hides the trade-off that selective classification makes explicit.

Agent behavior varies between runs, so reliability needs repeated trials. The tau-bench README reports pass^k, the chance that an agent succeeds on all of k trials of a task, for k from 1 to 4. For abstention, use the same idea on the failure side: a twin counts as safe only when no trial produced a fabricated or unconfirmed action.

Weight outcomes by business severity agreed with the task owner before any run, and keep the weights in the versioned spec. Compute intervals by resampling source cases, not individual twins, because twins of one case are strongly dependent. With a few dozen source cases per family, intervals will be wide; say so instead of ranking agents on small gaps.

Which business tasks and failure types is the suite probably missing?

Most first versions over-sample the easy type, a field deleted from the request, and under-sample the types that cause real incidents: stale records, two sources of equal authority, and partial permissions. Check the coverage matrix of task family against case type before any run and fill the empty cells, or state them as limits.

Other common gaps are multi-turn cases where the missing fact only matters three steps in, cases where the requester answers an elicitation with a wrong value, and cases where the policy itself is silent. Decide in advance whether a silent policy should produce a handoff or a documented assumption, and label it; otherwise graders will disagree and the benchmark will measure grader taste.

  • Task families covered, each with at least one complete case and one twin per applicable case type.
  • Counterexample twins for every case type in use.
  • At least one multi-step case where the gap appears after earlier actions.
  • Elicitation responses that are declined, canceled or wrong.
  • Linked records proving each reference outcome: the ticket, the invoice, the state after closure.

Where do complete cases for the twins come from?

Twins are only as good as the complete case they come from. Closed tickets, posted invoices and finished change requests, with their linked records and final states, give each twin a verified outcome and realistic distractors such as near-duplicate vendors or two open orders. Synthetic cases tend to lack the second plausible match that makes an ambiguity twin hard.

Complete cases of this kind come from U.S. businesses that SourceX approaches after a team's request, not from stock, and the request-data form takes the scope, volume, format, timing and licensing the team needs. Because the cases may serve as a long-lived evaluation set, it matters that each supplier approves the scope and signs before any records arrive.

Frequently asked questions

What is an abstention benchmark for AI agents?

It is a set of matched tasks in which some versions lack a fact the agent needs, labeled with the behavior each version expects: look it up, ask, escalate or stop. The agent is scored on its actions and the resulting system state, so a polite message that hides a guessed action still fails.

How is this different from testing unanswerable questions in RAG?

A RAG abstention test asks whether a model declines to answer when the corpus lacks support. An agent test also has to check what the agent did in other systems, whether it used a permitted tool to recover the fact, and whether it asked the right person through the right channel.

Should over-abstention count as a failure?

Yes, with a lower severity than fabrication. An agent that asks about every case is safe but useless, and the counterexample twins exist to measure that. Agree the cost of an unnecessary question with the task owner before scoring.

How many trials per case are enough?

Run each case several times, because a fabricated action that happens in one trial of ten is still a production incident. Treat a twin as safe only when no trial produced a fabricated or unconfirmed action, and report how many trials you ran.

Can the missing-evidence cases be generated automatically?

The ablation step can be scripted once the dependency list for each complete case exists. Writing the expected behavior and confirming that the removed fact cannot be recovered through another tool still needs a reviewer who knows the workflow.

Sources

  • Kalai and colleagues argue that binary grading rewards guessing over admitting uncertainty and that many benchmarks use binary accuracy or pass-rate metrics. Source
  • SQuAD 2.0 added unanswerable questions written to look relevant, with paragraphs containing a plausible answer of the right type. Source
  • Geifman and El-Yaniv frame selective classification as rejecting inputs to control risk on accepted inputs while maximizing coverage. Source
  • AbstentionBench spans 20 datasets over six abstention scenarios, from underspecified context to stale data, and its authors report that reasoning fine-tuning hurts abstention. Source
  • The tau-bench retail policy forbids making up information not provided by the user or tools, requires explicit confirmation before database-changing actions and limits transfers to out-of-scope requests. Source
  • The tau-bench README reports agent results as pass^k for k from 1 to 4. Source
  • The MCP elicitation specification lets servers request information from users, defines accept, decline and cancel responses and forbids form-mode requests for credentials. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies