Skip to content

AI uses for records

Computer use multi-tab context data: how to test state across tabs

By SourceX Editorial · Updated

9 sections

Short answer

To test a computer-use task that depends on several tabs, name one system of record for every fact the task touches, say which tab may show it and how fresh it must be, and check the record itself after each consequential action. Score the final stored state, not the last screen. Log tab focus, reloads, the tool or form used, the permission in force, and any retry or undo.

Agent run

Computer use multi tab

This page covers computer use multi-tab context data: how to test state across tabs.Illustrative

Key takeaways

  • A tab is a view, not the record; every multi-tab task needs a declared system of record for each fact it reads or writes.
  • Stale tabs are the main failure: same-origin tabs do not refresh each other's page content unless the application builds that in.
  • Score the end state stored in the system, as WebArena, OSWorld and tau-bench do in different ways, and treat the final screen as supporting evidence only.
  • Record tab focus, reloads, the permission in force and whether each write is safe to retry, so a duplicate submission is caught.
  • Some tabs are meant to stay stale, such as a saved report snapshot; the specification must say so or a correct agent will be marked wrong.

How do you test a computer-use task that spans several tabs?

Treat every tab as a possibly out-of-date view and test against the records behind it. Before the run, the task specification names the system of record for each fact, the tab expected to display it, and the freshness rule. After every action that writes, the checker queries the system of record and compares the stored values with what the task intended.

This matters because multi-tab work is how people actually do computer-use tasks in offices. An account manager keeps a CRM opportunity, a billing invoice and a support ticket open at once, copies a number from one, makes a change in another and confirms in the third. If the agent copies an amount from a tab loaded twenty minutes earlier, the screen in front of it can look perfectly consistent while the invoice is now wrong.

The rest of this guide gives a reusable specification for these tasks, explains which views count as evidence, shows how public benchmarks verify end state, walks through an illustrative three-tab case, and covers the tab that is supposed to stay stale.

Why can one tab look right while the record is wrong?

Because each open tab holds its own rendering of the data as of its last load, and nothing in the browser forces those copies to agree. Playwright, the browser automation library many agent harnesses use, models each tab or popup as a page inside a browser context, and one context can hold many pages. Each context gets its own cookies, local storage and session storage, so two contexts behave like two separate signed-in profiles.

At a glance

Why can one tab look right while the record is wrong

Within one context, tabs share an origin's local storage, but the page content does not update itself. MDN documents that a change to local storage fires a storage event in the other same-origin tabs, not in the tab that made the change, and that session storage changes never reach other tabs. The Broadcast Channel API lets same-origin tabs message each other, yet the message format is left to each application. Whether a second tab reflects an edit depends on what the application chose to build.

For a test designer, the conclusion is practical: never assume tab B saw the write made in tab A. The specification has to say whether the agent is expected to reload, wait for an in-app refresh, or ignore tab B for that fact. A checker that reads only the screen will reward an agent that never noticed the staleness.

  • Stale read: the agent copies a value from a tab loaded before another tab changed it.
  • Split write: the agent changes the same field in two tabs, and the later save silently overwrites the earlier one.
  • Wrong identity: two tabs are signed in as different users or tenants, and the write lands under the wrong permission.
  • Lost draft: an unsaved form in one tab is abandoned when the agent closes it to free focus.
  • Phantom success: a toast or banner says saved, but the request failed validation on the server.

What goes into a multi-view evidence and state task specification?

The specification is a single record per task that a second engineer could run without asking questions. It separates what the agent should achieve from what the agent can see, and it fixes the evidence that decides pass or fail before any run. The ten fields below are the minimum; a team can add domain fields but should not drop any of these.

What goes into a multi-view evidence and state task specification?
FieldWhat to recordHow the checker uses it
Task intentThe business outcome in one sentence, with the record identifiers and target values, written by the process ownerDefines the expected end state the final query must match
Tab mapEach tab's application, origin, signed-in user, role and the record it opens at startDetects wrong-identity writes and tabs the agent opened or closed
System of record per factFor every value read or written, the one system and field that is authoritativePoints every state check at storage, never at a tab
Freshness ruleWhether a tab must be reloaded after a write elsewhere, may rely on in-app refresh, or must be ignored for a factFlags stale reads by comparing the value used with the stored value at that moment
Write path and permissionThe form, API or tool allowed for each write, its validation rules, the role and scopes that should apply, and actions needing a human confirmationRejects writes made through an unapproved path or with broader access than intended, even if the value is right
Retry and undo ruleWhich writes are safe to repeat, how a duplicate is detected, and the approved reversal for each writeCounts duplicates and checks that any reversal left a clean audit trail
Per-step evidenceScreenshot, accessibility or DOM snapshot, active tab, open tab list and the stored-state query result after each consequential stepSupports diagnosis and partial credit; never overrides the end-state check
End-state check and invariantsThe exact query against the system of record with tolerances, plus facts that must not change, such as other customers' records, posted totals or audit fieldsDecides pass or fail; any unexpected diff outside the task fails the run
Exception ruleTabs that are meant to stay stale or belong to another actor, and whyPrevents a correct agent being failed for not refreshing them
Version stampApplication build or UI version, tool versions, date and specification versionLets results be compared only between runs on the same versions

Which views count as evidence in a multi-tab task?

Four views count, in a strict order of authority: the stored record, the tab list and focus history, the page structure, and the pixels. The stored record decides the outcome. The other three explain how the agent got there and where it went wrong.

Public browser environments already expose the middle layers. WebArena's action space includes opening a new tab, focusing a tab by index and closing a tab, so tab movements appear in the action log. BrowserGym returns the URLs and titles of every open page plus the index of the active one with each observation. Logging these fields at every step lets a reviewer see, for example, that the agent typed an amount while the billing tab was active but the value came from a CRM tab it had not reloaded since the start.

Page structure, whether a DOM snapshot or an accessibility tree, is the next layer. It shows which control was targeted and what text was present, which pixels alone cannot always settle. Screenshots remain useful for human review and for agents that act on pixels, but they are the weakest evidence of state, because a rendered value can be a cached copy.

  • Log the active tab index and the full open-tab list with every action, not only when the agent switches.
  • Record each reload or navigation as its own event with a timestamp, so freshness can be judged.
  • Store the stored-state query result next to the step that should have changed it.
  • Keep the agent's stated reason for switching tabs as a separate, attributable field when the harness captures one.

What proves the business state changed, not just the screen?

A query of the system of record after the task, compared with an expected state that was fixed before the run. A closing message, a success banner or a matching screenshot only shows that the interface said something; the stored record shows what the business now holds.

Public benchmarks reach this in different ways, and each suggests a design choice. WebArena's test configuration holds 812 tasks scored by three evaluator types: matching an answer string, matching the final URL, or running a program against page HTML to check content. An OSWorld task that spans a spreadsheet and a word processor saves the document and compares the resulting file with an expected file. Tau-bench replays the reference actions on a fresh copy of its database and sets the reward to zero unless the hash of the agent's final database matches.

For business tasks, the tau-bench pattern is the closest fit: compute the expected state by applying the intended change to a copy of the starting records, then compare the full set of fields the task is allowed to touch plus the invariants it must not touch. Checking only the target field misses the run that fixed the invoice and also edited a second customer by mistake.

What proves the business state changed, not just the screen?
Check typeWhat it provesWhat it missesUse in multi-tab tasks
Final answer stringThe agent reported the right valueWhether anything was storedOnly for read-only lookup tasks
Final URL or pageThe agent ended on the right recordField values and side effectsSupporting evidence
Program over page contentThe rendered page holds the expected textStaleness and other recordsUseful after a forced reload
File or export comparisonA saved artifact matches the expected oneChanges outside that artifactGood when the record is a document
Stored-state diff with invariantsThe intended fields changed and nothing else didWhy the agent chose its pathThe deciding check

How do tool contracts, permissions and retries change across tabs?

They multiply, because each tab can carry a different user, role and write path, and a retry in one tab can duplicate a write that already succeeded in another. The specification therefore names the approved write path and the permission for each change, and the checker fails a run that reaches the right state through the wrong door.

Agent tool protocols point the same way. The Model Context Protocol tools specification, in its June 2025 revision, says there should always be a human in the loop able to deny tool invocations, and that clients must treat a tool's behavior annotations as untrusted unless they come from a trusted server. For a test, that means a self-declared label such as read-only or safe to repeat is a claim to verify, not a fact to rely on.

Retries deserve their own rule. A credit, refund or status change submitted twice is often accepted twice. Record whether each write is safe to repeat, how a duplicate would be detected in storage, and which reversal is approved. A run that creates a duplicate and then reverses it can pass the end-state check yet still fail the audit-trail check, and the specification should say which outcome the team wants.

Worked example (illustrative): a credit across three tabs

This example is illustrative and uses generic systems, not any vendor's product. The task: a customer reported a duplicate charge; apply a credit on the open invoice equal to the duplicated line, mark the CRM account as credited, and close the support ticket with the credit reference. Three tabs are open at start, all signed in as the same billing specialist role.

The trap is built in. The CRM tab was loaded before a colleague corrected the duplicated line's quantity in billing, so the CRM summary shows an older figure. A passing agent reloads, or reads the line from billing, which is the declared system of record for amounts.

  • Illustrative failure 1: the agent uses the CRM figure from step 3; the credit amount is wrong and the run fails at step 4.
  • Illustrative failure 2: the credit form times out and the agent submits again; two credit rows exist, so the run fails the duplicate check even though the balance looks right on screen.
  • Illustrative failure 3: the agent closes the ticket before the credit is stored; the final state may match, but the step order breaks the process owner's rule and the run is marked a process failure.
Worked example (illustrative): a credit across three tabs
StepTab and actionState check after the stepEvidence kept
1Support: read the ticket and the invoice referenceNone; read onlyActive tab, ticket fields in the accessibility tree
2Billing: open the invoice and read the duplicated lineNone; read only, value logged with read timeLine values and load timestamp
3CRM: view the account summaryFreshness rule: CRM amount must not be usedTab load time earlier than the billing correction
4Billing: create the credit through the approved credit formOne new credit row, amount equals the corrected line, invoice balance reducedForm submission, stored credit row, duplicate count of zero
5CRM: reload, then set the account's credited flagFlag set on this account only; no other account changedReload event, stored flag, invariant diff
6Support: add the credit reference and close the ticketTicket status closed, reference matches stored credit idStored ticket fields
7Final checkFull diff against expected state plus invariantsPass or fail with the failing field named

When should a stale tab be left alone?

When the tab is a deliberate snapshot or belongs to someone else. A saved month-end report, an approval screen held open by a second person, or a read-only audit view are meant to show the state at a point in time. Forcing a reload can change what the task is testing, and in an approval flow it can hide that the approver acted on older figures.

The specification's exception rule names these tabs, says why they stay as they are, and tells the checker not to penalize the agent for leaving them. It also says what the agent should do instead, such as noting the difference in the ticket rather than editing the report. Without the rule, a reviewer will mark a careful agent wrong and a careless one right.

There are limits to the whole method. It assumes the team can query the system of record during evaluation, which is easy in a sandbox and hard against a production application with no read API. It also assumes the process owner can state one correct end state; tasks with several acceptable outcomes need an explicit list of them, or the check becomes a matter of opinion.

When does state checking stop working, and what should a supplier provide?

Stop and redesign when you cannot name a single authoritative system for a fact, when two applications both claim to own the same field, or when the only evidence available is the screen. Those tasks can still train perception, but they cannot be scored for business correctness, and mixing them into a state-checked set inflates the pass rate.

Multi-tab tasks are only as realistic as the records behind them. Synthetic tabs rarely carry the late corrections, duplicate charges and conflicting summaries that make real office work hard. When buying task data or the business records to build environments from, ask for the starting records, the change history that explains why tabs disagree, and the expected end states, with personal data handled before delivery.

For a multi-tab evaluation set, a team can describe the systems, record volumes, formats, timing and licensing terms on SourceX's request form; nothing is pre-packaged, and SourceX goes looking among U.S. businesses for records that match each request.

Frequently asked questions

What is multi-tab context data for computer-use agents?

It is task data in which the answer depends on information spread across several open tabs, along with a record of which tab was active, when each was loaded and what the stored records held at each step. Without the load times and stored-state checks, it is only a set of screenshots.

Do browser tabs update each other automatically?

Not on their own. A local storage change fires a storage event in other same-origin tabs, and the Broadcast Channel API lets tabs message each other, but the page only updates if the application handles those signals. Session storage changes do not reach other tabs at all.

Should an agent reload every tab before acting?

Not always. Reloading is the safe default for tabs the specification marks as needing fresh values, but snapshot reports and other people's approval screens should stay as they are. The task specification should say which rule applies to each tab.

Is a screenshot enough to verify a computer-use task?

No. A screenshot shows what was rendered, which may be a cached or stale copy. Use it to explain a run, and decide pass or fail with a query of the system of record plus a check that nothing outside the task changed.

How do you catch a duplicate submission in a multi-tab test?

Declare in the specification which writes are safe to repeat, and after each write count the stored rows that match the change. Two credit rows where one was intended fail the run even if the balance on screen looks correct.

Sources

  • In Playwright, a page is a single tab or popup window within a browser context, and one context can have multiple pages. Source
  • Playwright browser contexts are isolated, incognito-like profiles, each with its own local storage, session storage and cookies. Source
  • A localStorage change fires the storage event in other same-origin tabs but not in the window that made it; sessionStorage changes do not reach other tabs. Source
  • The Broadcast Channel API lets same-origin windows, tabs and frames communicate, and leaves the message protocol to each application. Source
  • WebArena's browser action space includes new_tab, tab_focus and close_tab actions. Source
  • BrowserGym observations include the URLs and titles of all open pages and the index of the active page. Source
  • WebArena's test configuration holds 812 tasks scored by string_match, url_match or program_html evaluators. Source
  • An OSWorld multi-app task spanning a spreadsheet and a word processor is scored by comparing the saved document with an expected file. Source
  • Tau-bench replays the reference actions on a fresh database and gives zero reward unless the agent's final database hash matches. Source
  • The MCP tools specification says a human should be able to deny tool invocations and that clients must treat tool annotations as untrusted unless the server is trusted. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies