Skip to content

AI uses for records

Computer use agent stale page state: can the agent notice old data?

By SourceX Editorial · Updated

10 sections

Short answer

To test whether a computer-use agent notices stale page data, change the record in the system of record after the page has loaded, give a task whose right action depends on the current value, and grade in the backend. Pair every stale run with a control run, classify outcomes from verified-and-correct to overwrote newer data, and keep the harness from refreshing on the agent's behalf.

Agent run

Computer use agent stale

This page covers computer use agent stale page state: can the agent notice old data.Illustrative

Key takeaways

  • Staleness is an input problem: the question is whether what the agent saw was still true when it acted.
  • Inject a scripted change after render and grade against the system of record, never the screen the agent may have trusted.
  • Pair each stale run with a control run of the same task so the score measures noticing, not a reload habit or luck.
  • Score five outcome classes, with overwriting newer data as the most serious and conditional writes as a way to detect it.
  • Auto-waiting and retrying harness code can hand the agent the fresh value and silently erase the test.

How do you test whether a computer-use agent notices stale page data?

Make the page wrong on purpose. Change the record in the system of record after the agent's page has loaded, give the agent a task whose correct action depends on the current value, and score the result in the backend: did the agent re-read, act on the fresh value, refuse to overwrite newer data, or escalate. Run the same task without the change as a control, so the score measures noticing rather than luck.

That design treats staleness as an input problem. Most computer-use evaluation asks whether an action took effect. This test asks something earlier: whether the agent's belief about the world was still true when it acted, and what it did when it was not.

The rest of this page gives the benchmark brief as a table of task families, the fields a brief must fix before anyone runs it, a scoring scheme with five outcome classes, the harness settings that quietly defeat the test, and an illustrative dispatch case.

Where stale page state comes from in everyday business software

Stale state appears whenever what a page displays was true at render time but the record has moved on since. In business software that is routine, not exotic: two people work the same queue, a background job updates a status, or a list view is served from a cache.

At a glance

Where stale page state comes from in everyday business software

HTTP gives the vocabulary for one of these sources. RFC 9111 calls a stored response fresh while its age has not exceeded its freshness lifetime and stale once it has, where age is the time since the origin server generated or last validated it. A page can also be stale with no cache involved at all, simply because the tab was opened before a colleague's edit.

  • Concurrent edit: a second user or session changes the record while the agent's tab still shows the earlier version.
  • Background update: an integration, scheduled job or webhook changes a status field without the open page re-rendering.
  • Cached view: a list, dashboard or search result is served from a cache or an index that lags the record.
  • Optimistic interface: the page shows a value the client assumed would save, before the server confirms or rejects it.
  • Long-lived session: the agent opened the page minutes ago, worked elsewhere, and returns to act on what it saw then.

Why the screen cannot be the grader for a staleness test

A staleness test has to be graded where the truth lives, because the stale screen is exactly what the agent might have trusted. If the grader reads the same screen, an agent that acted on old data and an agent that acted correctly can look identical.

Established benchmarks already grade by state rather than by appearance. The OSWorld paper pairs each of its 369 computer tasks with an initial state configuration and an execution-based evaluation script, and WebArena's repository tells evaluators to return the environment to its initial state after its 812 examples have run. A staleness benchmark needs the same two properties plus one more: a scripted change to the record at a known moment, logged so the grader knows which value was current when each action landed.

The stale UI and source of truth benchmark brief

The brief below defines six task families, each with a different staleness source and a different correct behavior. It is a template built for this page; replace the example systems with the applications your agent actually operates, and keep one control run per task with no injected change.

Every family names its source of truth explicitly. If a team cannot say which record or API is authoritative for a field, the task is not ready to run.

The stale UI and source of truth benchmark brief
Task familyInjected change and timingSource of truthCorrect behaviorScored from
Concurrent reassignmentSecond session reassigns a work order after the agent opens the dispatch boardWork order record in the field service systemRe-read before assigning; do not double-bookAssignment table and audit log
Status changed by a jobIntegration marks an invoice as paid while the agent's list shows it overdueAccounts receivable recordSkip the reminder or flag the mismatchOutbound message log and invoice status
Cached search resultSearch index lags a contact's new email address by a set delayContact record, not the search indexOpen the record before using the valueValue written to the downstream task
Optimistic save rejectedServer rejects a field update the page already displays as savedRecord after server responseDetect rejection and retry or reportField value and error log
Conflicting edit on writeRecord changes between the agent's read and its writeServer version or entity tag of the recordAccept a precondition failure, re-read, merge or escalateRecord history showing no lost update
Return after delayTicket closed by a colleague while the agent works in another appTicket recordRe-check before replying to the customerReply log and ticket state

What the brief must pin down before the first run

A brief is ready when two engineers who never spoke could build the same run from it. That means fixing the parts that most often change a staleness result silently.

At a glance

What the brief must pin down before the first run

  • Task intent in one sentence, written as the business outcome, not the clicks.
  • Starting state and environment version, with the reset procedure that restores it.
  • Injection mechanism: which actor changes which field, through which interface, at which step or elapsed time.
  • Screen context available to the agent: screenshots, accessibility tree, DOM, or a mix, recorded per step.
  • Tool contract: which reads and writes the agent may call, and whether writes are conditional on a version or entity tag.
  • Permissions: the agent's role, so a refusal caused by access is not scored as caution.
  • Allowed verification actions: page reload, opening the record, calling a read API, asking the user.
  • Success state in the system of record, plus the forbidden states, such as an overwritten newer value.
  • Retry and undo rules: how many attempts, and whether a reversal counts as recovery.
  • Logging: timestamps for the injection and every agent action, so the grader can tell which value was current at each write.

Noticing or lucky: how paired runs and outcome classes separate them

Pair each staleness run with a control run of the same task and seed but no injected change. An agent that reloads every page before acting will pass both; an agent that never re-reads will pass the control and fail the stale run. Only the difference between the two says anything about noticing.

Score each stale run into one of five outcome classes rather than a single pass or fail. Acted on fresh value after verifying is the target. Noticed the mismatch and escalated or abstained is acceptable when the task allows it. Acted on the stale value without harm happens when the change did not matter to the action and should be reported separately. Acted on the stale value with a wrong result is a failure. Overwrote newer data is the most serious class, because it destroys someone else's work.

Conditional writes make the last class detectable at the tool layer. Under RFC 9110, a request carrying If-Match is conditional on the server still holding a representation whose entity tag matches, and when the condition fails the server must not apply the method and may answer 412 Precondition Failed. If the application under test supports this, give the agent tools that send the tag it last read and record how the agent handles the 412. If it does not, the grader must compare the record's version history against the injection log.

Harness habits that hide staleness from the test

Test frameworks are built to wait out transient states, and the same convenience can erase the stale window the benchmark depends on. Check the harness before trusting any staleness score.

Playwright is a clear example of the behavior to watch for. Before a click, it auto-waits until the element resolves to exactly one node and is visible, stable, receiving events and enabled. Its web-first assertions retry, re-fetching the element until the condition holds or a timeout expires, and its documentation warns that non-retrying assertions tend to flake because pages load information asynchronously. Those are sound defaults for testing an application. In an agent harness, a retrying wait placed between the injection and the agent's next observation can deliver the fresh value for free, so the agent never faces the stale page.

Three settings keep the test honest. Do not let the harness reload or re-query on the agent's behalf; any refresh must be an action the agent chose and the log records. Time the injection relative to the agent's observations, not wall-clock alone. And snapshot exactly what the agent was shown at each step, so a reviewer can confirm the stale value was really on screen.

Worked case: a dispatch board that missed a reassignment (illustrative)

This case is invented to show the method; it describes no real company or system. The task: assign today's highest-priority repair job to an available technician on a dispatch board.

The agent loads the board, which lists 4 technicians as available. At step 3, a scripted second session assigns one of those technicians, call them T2, to a different job. The agent's screen still shows T2 as available. At step 5, the agent drags the repair job onto T2.

In the control run with no injection, assigning T2 is correct. In the stale run, the system of record now has T2 on two overlapping jobs. The grader reads the assignment table and the audit log, sees that the injection preceded the agent's write by two steps and that the agent made no read between them, and classes the run as acted on stale value with a wrong result.

Across an illustrative set of 20 paired runs, a team would report the count of stale runs in each outcome class beside the control pass count, for example 18 of 20 controls passed while 7 of 20 stale runs reached the target class. Those figures are placeholders that show the shape of the report, not a measurement. A useful variant repeats the task with a conditional assign tool: the same mistake now returns a precondition failure, and the score shows whether the agent re-reads the board or retries blindly.

Limits: when refreshing is not the right answer

Refreshing before every action is not always correct, so the brief should name the cases where the expected behavior is something else. Otherwise the benchmark rewards an agent for habits that are costly or harmful in production.

Some reloads discard work, such as an unsaved form, and the right move is to open the record in a second view instead. Some sources of truth are themselves eventually consistent, in which case the correct behavior is to wait for a confirmed version or report uncertainty, and the grader must allow for it. And when the team has no read access to the backend, a staleness benchmark cannot be graded honestly; narrow the claim to what the screen logs show and do not report it as a state-verified result.

Real change histories behind staleness tasks

The most realistic injections come from what actually happened: audit logs and record histories that show two people or a person and a job touching the same record within minutes. Those histories tell a benchmark team which fields change underneath users, how often, and with what lag, so the injected changes match real ones instead of guesses.

A benchmark team can ask SourceX for de-identified record histories from dispatch, billing or ticketing systems; once the request names the scope, volume, format and timing it needs, the records are sourced from U.S. businesses rather than drawn from a standing catalog. Any record history reaches SourceX only after its owner has approved scope and terms and signed, and user names in those histories are de-identified by SourceX through approved processing routes.

Frequently asked questions

What should the stale UI and source of truth benchmark brief include?

For each task: the intent, starting state and reset, the injected change with its timing and actor, the authoritative source of truth, the correct behavior, the allowed verification actions and the backend signal used for scoring. Add a control run with no injected change for every stale run.

How do you prove the record changed, and not just the screen?

Only a read of the system of record, compared against a log that says which value was current when each action landed. A confirmation message or a refreshed screen shows what the interface believes, which is exactly what a staleness test calls into question.

Should an agent always reload the page before acting?

No. Reloading can discard unsaved work and adds cost, and some sources of truth lag themselves. The brief should define when opening the record, calling a read API, using a conditional write or asking the user is the expected behavior.

How is a stale page different from an agent's stale memory?

A stale page is current-session input that changed after it rendered; stale memory is a fact the agent stored earlier that has since gone out of date. The tests overlap, but a stale-page benchmark injects changes during the episode and grades the very next actions.

Can conditional requests stop an agent from overwriting newer data?

They can when the application supports them. Under RFC 9110, a write sent with If-Match fails with 412 Precondition Failed if the record's entity tag has changed, so the agent cannot silently overwrite; the benchmark then measures what the agent does after the failure.

Sources

  • RFC 9111 defines a fresh response as one whose age has not exceeded its freshness lifetime and a stale response as one whose age has. Source
  • RFC 9110 makes a request with If-Match conditional on a matching entity tag; if the condition fails the server must not perform the method and may respond 412 Precondition Failed. Source
  • The OSWorld paper pairs each of 369 computer tasks with an initial state setup configuration and an execution-based evaluation script. Source
  • WebArena's repository instructs evaluators to reset the environment to its initial state after evaluating its 812 examples. Source
  • Playwright auto-waits for actionability checks such as visible, stable, receives events and enabled before performing actions like click. Source
  • Playwright's auto-retrying assertions re-fetch the element until the condition is met or a timeout is reached, and non-retrying assertions can be flaky on asynchronous pages. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies