Skip to content

AI uses for records

Computer use agent confirmation data: did the save really happen?

By SourceX Editorial · Updated

10 sections

Short answer

To verify that a computer-use agent saved the intended change, read the record back from the system of record through a channel the agent did not use, after its session ends, and compare stored field values with the task's targets. Check that untouched records stayed unchanged. The agent's message, a toast or a screenshot are claims, not confirmation. Record the result as confirmation data linked to intent and actions.

Agent run

Computer use agent

This page covers computer use agent confirmation data: did the save really happen.Illustrative

Key takeaways

  • A save is confirmed by an independent read of the stored record after the session, not by the agent's message, a toast or a screenshot.
  • Build a persistence task set where each task tempts one failure: validation blocks, drafts, duplicates, partial saves, autosave delays, conflicts and retries.
  • Check whether your harness saves for the agent; some OSWorld spreadsheet tasks press Ctrl+S in the evaluator before grading.
  • Store a confirmation record per attempt that links intent, starting state, actions, verification channel and observed fields.
  • Report claimed success, verified success and the false confirmation rate, and use pass^k across repeated trials.

How do you verify that a computer-use agent saved the intended change?

Verify the save by reading the business record back through a channel the agent did not use, after the agent's session ends, and comparing the stored field values with the values the task asked for. The agent's final message, a green toast and the last screenshot are claims about the outcome. The stored record is the outcome.

Anthropic's guide to agent evaluations draws the same line. It defines the outcome as the final state in the environment at the end of the trial, and its example is a flight-booking agent that says the flight has been booked: what counts is whether a reservation exists in the environment's database. For a computer-use agent working in a CRM, an ERP or a help desk, that means a fresh read of the opportunity, the purchase order or the ticket, not a look at the screen the agent left open.

This page gives you the method as a reusable asset: a persistence verification task set, a ladder of evidence from weakest to strongest, and a confirmation record schema that links the agent's intent, its actions and the verified result. It complements state-based task design rather than repeating it.

Why is a click, a toast or a final message not confirmation?

Because each of them can appear while the stored record stays unchanged. A save button can be clicked while a required field fails validation, a toast can render before the server rejects the write, and an agent can report success because the last step looked routine. None of these is rare in business software, and none of them shows up if you only grade what the agent says.

Model behavior adds to the risk. Anthropic's computer use documentation notes that Claude sometimes assumes outcomes of its actions without explicitly checking their results, and suggests prompting the model to take a screenshot after each step and confirm the step succeeded before moving on. That prompt helps the agent, but a screenshot from inside the same session is still the agent's view of the interface, not proof that the record persisted.

Timing is the other trap. The Playwright documentation points out that most web pages show information asynchronously and that non-retrying assertions can make tests flaky; its auto-retrying assertions wait up to a default timeout of 5 seconds. A verifier that reads the record a few milliseconds after the click will report false failures on autosave forms and false passes on optimistic interfaces. Your check needs a defined wait, a retry window and a timestamp.

  • Validation blocked the write but the form kept the typed values on screen.
  • The interface updated optimistically and the server later rejected the change.
  • The change saved to a draft, a local cache or an unsubmitted modal.
  • The agent edited the wrong record, such as a duplicate account with the same name.
  • Only some fields saved, for example the amount but not the close date.
  • A concurrent edit by a person or an integration overwrote the agent's change.

What can an evaluation harness hide about saving?

A harness can save on the agent's behalf, which makes a persistence failure invisible. In the public OSWorld repository (commit of September 15, 2026, checked October 18, 2026), many spreadsheet task configurations include an evaluator postconfig step that sends a Ctrl+S keypress before the result file is compared with a gold copy. A count of the example files on that commit finds the step in 57 of 369 task files, mostly in LibreOffice Calc tasks.

That design is reasonable when a benchmark measures whether the agent can edit a spreadsheet correctly. It is the wrong design when you need to know whether the agent completes the job, including the save. If persistence matters, run each task in two variants: one where the harness saves, and one where it does not. The gap between the two pass rates is your measured persistence failure rate.

Where the check reads the result matters too. WebArena's HTML content evaluator can check the page the agent ended on, or navigate to a target URL and read the content there with a locator or an API helper. A fresh navigation is closer to a persistence check than reading the last page; a read through an API or database, outside the browser session, is closer still.

What should the persistence verification task set include?

It should include tasks that tempt each specific failure, a known starting state, an independent check and a pass condition stated at field level. The table below is the asset: copy the rows, swap in your application's objects and keep the trap column honest. The examples are illustrative and use common record types, not a specific vendor's behavior.

What should the persistence verification task set include?
Task familyStarting state and trapIndependent check after sessionPass condition
Plain field updateOpen opportunity; change close date and stageAPI read of the record by IDBoth fields equal target values; no other field changed
Validation blockStage change requires a field the agent must fill firstAPI read plus validation error log if exposedAgent fills the required field and the stage persists, or reports the block truthfully
Draft versus submitForm has Save draft and Submit buttonsStatus field read from the serverStatus equals submitted, not draft
Duplicate recordTwo accounts with near-identical namesRead by the unique ID named in the taskOnly the intended record changed; the twin is untouched
Partial saveMulti-tab form saves each tab separatelyRead every field the task touchedAll target fields persisted, not just the active tab
Autosave delayField autosaves after a short debouncePoll with a timeout and timestamp each readValue appears within the wait window; agent did not exit early
Concurrent editA scripted user edits the same record mid-taskRead final record and audit historyAgent detects the conflict and does not silently overwrite
Permission dropAgent role can view but not edit one fieldRead the field and the agent's final reportField unchanged and agent reports it could not edit
Undo and retryFirst attempt fails; agent retriesCount records or line items createdExactly one change applied; no duplicate entries
Downstream effectSaving a ticket should trigger an assignment ruleRead the assignment or workflow logThe expected downstream event exists with a later timestamp

What proves the intended business state changed rather than only the interface?

A read from the system of record, outside the agent's session, proves it; everything closer to the agent's own view is weaker. Rank your evidence and record which rung each verification reached, so a reviewer can tell a strong pass from a weak one.

tau-bench shows the strongest pattern for tool agents. Its reward function replays the task's gold actions against a fresh copy of the data, hashes the resulting database and compares that hash with the hash of the database the agent produced; if they differ, the reward is zero. That one comparison also catches collateral changes, because any extra edit changes the hash. For business software you rarely have a full database snapshot, so the practical version is a field-level read of the target record plus negative assertions on records the task should not touch.

What proves the intended business state changed rather than only the interface?
Evidence rungExampleWhat it provesWhat it misses
1. Agent statementFinal message says the record is updatedThe agent believes it finishedEverything about stored state
2. Same-session screenshotScreenshot shows new value in the formThe interface shows the valueUnsaved, optimistic or cached values
3. Reload in same sessionAgent refreshes and sees the valueValue survives a page loadClient-side caches and session-only state
4. Different session or userVerifier logs in separately and opens the recordValue is visible to othersFields the interface does not display
5. API or database readRead by record ID after the session endsStored field valuesDownstream effects and side effects
6. Downstream and negative checksWorkflow log, audit trail, untouched twin recordsIntended change and no collateral changeLittle, if checks are field-level

What confirmation record should each attempt leave behind?

Each attempt should leave a confirmation record that links the task's intent, the starting state, the actions taken and the independent verification. Without the link, a buyer or an evaluation team cannot tell whether a pass came from a real save or from a harness step, a lucky retry or a lenient check.

At a glance

What confirmation record should each attempt leave behind

Use the schema below as computer use agent confirmation data: one row per task attempt, stored next to the trajectory. It is deliberately small, and every field answers a question a reviewer will ask.

What confirmation record should each attempt leave behind?
FieldWhat it holdsWhy it matters
task_id and intentThe instruction and the target field valuesDefines what a correct save means
starting_state_refSnapshot ID or seeded fixture for the records involvedLets anyone reproduce the attempt
app_and_tool_versionsApplication build, browser, tool contract versionUI changes invalidate old checks
permission_contextRole and scopes the agent ran withSeparates a permission block from an agent error
action_log_refPointer to the trajectory of clicks, keystrokes or tool callsShows whether the save action was ever taken
verification_channel and rungAPI, database, second session; rung 1-6States how strong the evidence is
observed_fields and read_atStored values and the time of each readField-level proof with timing for autosave cases
negative_assertionsRecords and fields that must be unchanged, with resultsCatches collateral edits
retry_and_undoNumber of attempts and any reversalsDetects duplicate writes and masked failures
agent_claim and outcome_labelWhat the agent reported; verified pass, fail or blockedMeasures false confirmation directly

Worked case: the stage that never saved (illustrative)

This case is illustrative, built to show the method, not taken from a real deployment. The task asks an agent to move a CRM opportunity to the Negotiation stage and set the close date to the last day of the quarter. The starting state is a seeded opportunity in the Proposal stage, and a validation rule requires a next-step field before any stage after Proposal.

The agent changes both fields, clicks Save and sees a brief banner. Its final message says the opportunity was updated. The verifier waits for the defined window, reads the record through the API by ID and finds the new close date but the old stage, because the save attempt that carried the stage failed validation while an inline edit saved the date. The confirmation record shows rung 5 evidence, one negative assertion passed, agent_claim of success and outcome_label of fail. That is a false confirmation, and it is exactly the event this task set exists to count.

Now the counterexample. On an autosave form, the same verifier reading at once would have marked a correct run as failed, because the value lands after a short debounce. Playwright's expect.poll pattern, which polls a function until it returns the expected value or a timeout passes, is the right shape for this read. The lesson for the task set: every check gets an explicit wait window, and the window is recorded with the result.

How should retries, conflicts and consequential actions be handled?

Treat a retry as a new write that needs its own verification, and design the check so a double write fails. A retried submit that creates two line items, two notes or two payments is a persistence error even though the target value is present.

Concurrent edits need a rule as well. HTTP's If-Match precondition, defined in RFC 9110, exists to prevent the lost update problem when several clients act on the same resource; when the condition is false the server must not perform the method and may answer 412 Precondition Failed. You do not need your application to expose ETags to use the idea: seed a competing edit mid-task, then check that the agent noticed the conflict instead of silently overwriting it.

For consequential steps, add a human gate and test it. Anthropic's computer use documentation recommends asking a human to confirm decisions with meaningful real-world consequences, such as financial transactions, and notes that a batch of actions can complete a multistep action within one turn, so the confirmation check has to run before each block. A persistence task set should include at least one task where the correct behavior is to stop and ask.

How do you score persistence across repeated runs?

Score verified success, not claimed success, and repeat each task several times. Anthropic's evaluation guide notes that model outputs vary between runs, defines pass@k as the chance that at least one of k attempts succeeds and pass^k as the probability that all k trials succeed. For a business workflow where every save must stick, pass^k is the metric that matches the risk.

Report three numbers side by side: the claimed success rate from the agent's final messages, the verified success rate from rung 5 or 6 checks, and the false confirmation rate, meaning attempts where the agent claimed success and the stored state disagreed. The third number is the one a deployment owner cares about most, and it only exists if you keep the confirmation record. Then read a sample of transcripts; the same guide is blunt that you will not know whether your graders work unless you read transcripts and grades from many trials.

Where does real confirmation data come from?

Real workflow records are the best source of starting states and traps, because they show which validation rules, duplicates and handoffs people actually hit. Audit trails, field history tables and workflow logs in systems like Salesforce, Zendesk, Jira or NetSuite record what changed and when, which is the raw material for negative assertions and downstream checks.

Teams that want those traps from other companies' systems can request them through SourceX, which locates licensed workflow records at U.S. businesses on request instead of selling from a catalog, and handles the rights review, contracting, preparation and delivery. The request form asks for scope, volume, language, format, timing and licensing needs; name the audit and field-history tables your persistence checks will read. The supplying company signs its agreement before any records are delivered.

Frequently asked questions

What is computer use agent confirmation data?

It is the record that links a computer-use agent's task, its actions and an independent check of the stored result. A useful version holds the intent, the starting state, the app version, the agent's permissions, the verification channel, the observed field values and whether the agent's claim matched the stored state.

Is a screenshot enough to confirm a save?

No. A screenshot from the agent's own session shows what the interface displays, which can include unsaved, cached or optimistically updated values. Use it as a diagnostic, and confirm the save with a read from the system of record after the session ends.

How long should the verifier wait before reading the record?

Long enough for the application's normal save path, and the wait should be explicit. Poll with a timeout rather than reading once, record the time of each read, and treat a value that never appears within the window as a failure.

What is a false confirmation rate?

It is the share of attempts where the agent reported success but the independent check found the intended state missing or wrong. It is the most direct measure of whether an agent can be trusted to finish work without someone checking behind it.

Should the evaluation harness ever save on the agent's behalf?

Only when you are deliberately measuring something else, such as editing accuracy. If you need to know whether the agent completes the job, remove harness saves or run paired variants with and without them, and report the difference.

Sources

  • Anthropic defines an agent evaluation outcome as the final state in the environment at the end of the trial, uses a flight-booking example where the reservation must exist in the database, defines pass@k and pass^k, and advises running multiple trials and reading transcripts. Source
  • Anthropic's computer use documentation notes Claude sometimes assumes outcomes without checking, suggests a screenshot check after each step, and recommends human confirmation for consequential actions, checked before each batch of actions. Source
  • OSWorld example task configurations define a starting state and an evaluator; many spreadsheet tasks press Ctrl+S in the evaluator postconfig before comparing the result file with a gold file. Source
  • WebArena's HTML content evaluator can check the last page or navigate to a target URL and read content with a locator or helper function. Source
  • tau-bench computes reward by replaying the task's gold actions on fresh data and comparing a hash of the resulting database with the agent's database; a mismatch sets reward to zero. Source
  • Playwright's assertion docs note pages show information asynchronously, describe auto-retrying assertions with a 5 second default timeout, and document expect.poll for polling until a condition holds. Source
  • RFC 9110 describes If-Match as preventing the lost update problem on state-changing requests; a server must not perform the method when the condition is false and may respond 412 Precondition Failed. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies