AI uses for records
Computer use action rollback evidence: does undo restore the business state?
By SourceX Editorial · Updated
10 sections
Short answer
Test undo by snapshotting the records an action touches before it runs, performing the action and then its undo, and checking declared invariants against the business system rather than the screen. List every side effect the forward action caused, such as emails, refunds, webhooks or audit entries, and require each to be reversed, compensated or recorded as irreversible. A restored screen with a lingering side effect is a failed rollback.
Agent run
Computer use action rollback
Key takeaways
- Undo is a second business action; judge it by reads from the system of record, not by the restored page.
- Most business undo is compensation: new work that offsets the old, not a return to a snapshot.
- Write invariants before the action runs and exclude fields the system legitimately changes.
- Inventory side effects such as emails, refunds, webhooks and audit entries, and require each to be reversed, compensated or acknowledged.
- Grade recovery with explicit verdicts so a lucky screen does not count as a clean rollback.
How do you test whether undo restores business state rather than just the screen?
Treat undo as a second business action with its own expected outcome. Before the forward action, capture the state of every record it can touch and the list of downstream systems it can trigger. After the undo, read those records back from the system of record, check them against invariants written in advance, and walk the side-effect inventory item by item. The screen is only one of the places you look, and usually the least informative.
The gap between screen and business state is wider for undo than for the original action. A forward action changes one record and triggers its consequences. An undo button often reverses the visible field and nothing else: the customer email has gone, the payment provider has issued a refund that exists as its own object, the integration has already pushed the change to a warehouse system, and the audit log now holds two entries rather than none.
Timing adds a second gap. Many undo paths are open only for a while: a send can be cancelled for seconds, a deleted record can be restored for days, a refund can be issued but not withdrawn. A test that runs the undo immediately after the action can pass where a real recovery, made hours later after a human noticed the error, would fail. Record the delay between action and undo in every task, and build some tasks where the window has already closed so that the correct behavior is compensation or escalation rather than undo.
For computer-use agents this matters twice. Agents need to recover from their own mistakes, so training data should show what a correct recovery looks like in business terms. And evaluation tasks that grade recovery by the screen will reward agents for hiding errors rather than repairing them.
Five kinds of undo, and what each one actually reverses
Before writing a test, classify the undo the application offers, because each kind can restore different things. The examples come from current vendor documentation; behavior can change, so recheck the product's own pages when you build tasks on them.
| Undo kind | Real example | What it can restore | What it leaves behind |
|---|---|---|---|
| Local edit undo before save | Reverting a typed field before submitting a form | The unsaved field value | Nothing, if the form was never submitted; everything, if autosave already wrote it |
| Delayed commit | Gmail Undo Send, with a cancellation period of 5, 10, 20 or 30 seconds | The message, if cancelled inside the window | A sent message once the window closes |
| Soft delete and restore | Salesforce Recycle Bin, which keeps deleted records for 15 days | The record and, for a cascade delete, its master-detail children when the parent is restored | Children hard-deleted because they were deleted before their parent; anything purged after the window |
| Compensating action | A Stripe refund, created as a separate refund object against the original charge | The customer's money, fully or in partial refunds | The original charge and its history, plus the refund's own records and events |
| No undo | A filing submitted to a third party, a message delivered to a customer | Nothing in the system itself | The full effect; only a follow-up action such as a correction is possible |
Why rollback in business software is usually compensation
In most business systems an undo does not restore a past snapshot; it performs new work that offsets the old. Microsoft's Azure Architecture Center describes this as the Compensating Transaction pattern. It notes that you cannot always roll data back because other instances may have changed it since, that a compensating transaction is application specific, and that it does not necessarily return the data to its state at the start of the original operation. It also advises defining points of no return and identifying which steps are compensable and which are irreversible.
That has a direct consequence for test design. If the expected outcome of every undo task is the pre-action snapshot, field for field, the test will fail correct recoveries: the record now has a later modified timestamp, the refund carries its own ID, a colleague's unrelated edit has landed in the meantime. The test should instead state what business facts must hold after the undo, which is what invariants are for.
The same guidance recommends that compensation steps be idempotent, because a compensation can fail and need to be retried. For an agent, that means a recovery that is repeated after a timeout must not issue a second refund or restore a record twice.
Writing rollback invariants: what must be true after the undo
An invariant is a business fact that must be true after the undo regardless of how the system got there. Write them before the forward action runs, from the business owner's point of view, and make each one checkable by a read against the system of record.
At a glance
Writing rollback invariants: what must be true after the undo
Not every task needs all eight. A rename of a contact has no aggregates and no external effects; a cancelled shipment touches nearly all of them. Choose the invariants that the business owner would check by hand if a person had made the same mistake, and write down why the others do not apply, so a reviewer can tell an omission from a judgment.
- Record restoration: the target records' business fields equal the snapshot, excluding fields the system manages, such as last-modified time and version counters.
- Aggregate balance: totals that the action moved, such as stock on hand, an account balance or an open-ticket count, return to their pre-action values or are offset by a recorded compensating entry.
- Relationship integrity: child and linked records come back with the parent, and no orphaned links point at the restored record.
- No duplicates: the undo did not create a second copy of the record, the payment or the message.
- Concurrent work preserved: changes made by other users or processes between the action and the undo are still present.
- Side effects accounted for: every external effect on the inventory is reversed, compensated or explicitly acknowledged as irreversible, with the acknowledgement recorded.
- History kept: the audit trail shows both the action and the undo; a rollback that erases the log is a failure, not a success.
- Permission respected: the undo was performed with the same or narrower permissions than the action, without escalating to an admin path.
The rollback invariant and side effect verification task
This is the task specification to use as is. One task covers one forward action and its undo; chain tasks when an agent's recovery spans several steps. Fill the fields before running the agent, except the observed ones, which the harness fills after.
| Field | What it holds | Illustrative value | Filled by |
|---|---|---|---|
| intent_and_mistake | The goal and the deliberate or observed error to recover from | Cancel line 2 of an order; the agent cancelled line 3 instead | Task author |
| target_records | IDs of every record the forward action can change | Order, its lines, the stock reservation, the payment | Task author |
| pre_snapshot | Business fields of the target records and aggregates before the action | Line 3 quantity, reserved stock count, amount captured | Harness, by API read |
| side_effect_inventory | Each downstream effect the action can cause, with its channel and reversibility | Refund, customer email, warehouse webhook, audit entry | Task author with the process owner |
| undo_path_and_window | Which undo the system offers and how long it stays open | Reinstate line within the same business day; refund cannot be cancelled once issued | Task author from product documentation |
| invariants | Checks from the list above, each with its read query | Line 3 active with original quantity; stock reserved again; one refund compensated by a new charge or noted | Task author |
| concurrency_allowance | Fields others may legitimately change during the run | Shipping notes edited by the warehouse | Task author |
| observed_after_undo | Post-undo reads of every target record, aggregate and side-effect channel | Read results with timestamps | Harness |
| verdict_and_evidence | One of five verdicts with the read results that justify it | Compensated, with refund and new charge IDs | Grader |
Where side effects hide beyond the edited record
Most failed rollbacks fail outside the record the agent edited. Build the side-effect inventory by asking the business process owner what happens downstream when the action runs, then confirm each item in a sandbox before trusting it in a task.
Tool metadata is a starting point, not a substitute. The Model Context Protocol schema defines tool annotations such as destructiveHint, idempotentHint and openWorldHint, which describe whether a tool may make destructive updates, whether repeated calls have no additional effect and whether it reaches external entities. The same schema calls every annotation a hint that may not describe the tool faithfully, and that clients should never base tool-use decisions on annotations from untrusted servers. Record the annotations, then confirm reversibility by testing.
| Channel | Typical effect | Usually reversible? | Evidence to capture |
|---|---|---|---|
| Customer notifications | Confirmation or cancellation email or text | No, once delivered | Outbound message log entry with time and template |
| Payments | Refund, void or new charge | Only by a further compensating payment | Payment object IDs and their events |
| Integrations and webhooks | Change pushed to a warehouse, ERP or partner | Depends on the receiving system | Outbound webhook log and the receiver's record |
| Automations | Workflow or rule that fired on the change | Sometimes; the automation may not fire on the undo | Automation run history |
| Inventory and counters | Reservation released, sequence number consumed | Counts usually; consumed sequence numbers no | Before and after counts; the gap in the sequence |
| Audit trail | Entries for the action and the undo | Must not be reversed | Both entries, with actor and time |
| Search indexes and caches | Stale value shown after the undo | Yes, after refresh | A re-read after the system's refresh interval |
Worked example (illustrative): reinstating a cancelled order line
Everything here is invented to show the task in use. An agent asked to cancel line 2 of a customer order cancels line 3. The order system automatically issues a partial refund through the payment provider, releases the stock reservation and sends the customer a cancellation email. The agent notices the error and clicks Reinstate line.
The example also shows why the side-effect inventory must be written before the run. Without it, a grader comparing the order page with its earlier screenshot has no reason to look at the payment provider, the warehouse reservation or the outbound message log, and the three open effects would never surface. With it, each one is a line to check, and the agent's trajectory can be labelled with exactly which repairs it made and which it missed.
- Screen-only grading would score this recovery as a success: the order page looks exactly as it did before the mistake.
- Invariant grading scores it as partially compensated: two invariants hold, two fail, and one side effect is irreversible.
- A correct recovery adds three steps: re-reserve the stock, collect the difference through the payment flow the business uses, and send a correction to the customer, each logged.
- The step the agent should then take is line 2's cancellation, which was the original intent and still has not happened.
| Check | Screen after undo | System of record after undo | Result |
|---|---|---|---|
| Line 3 status and quantity | Active, original quantity | Active, original quantity | Invariant holds |
| Stock reservation | Not shown | Released; not re-reserved because reinstatement does not reserve stock | Invariant fails |
| Payment | Order total looks correct | Partial refund issued; no new charge; customer now underpays for line 3 | Invariant fails until compensated |
| Customer email | Not shown | Cancellation email delivered | Irreversible; needs acknowledgement and a correction message |
| Audit trail | Not shown | Cancel and reinstate entries, both attributed to the agent's run | Invariant holds |
Grading the rollback: five verdicts and the evidence each needs
Grade each rollback with one of five verdicts so datasets and evaluations can distinguish a clean recovery from a lucky one. Each verdict needs specific evidence from the system of record, not a screenshot.
Public benchmarks show the two ends of this spectrum. tau-bench scores an episode by hashing the environment's database after the agent's actions and comparing it with the hash produced by replaying the ground-truth actions, which works because the sandbox is deterministic. OSWorld tasks pair a setup configuration for the initial state with an evaluator that checks the result. Both are end-state checks on controlled environments; rollback tasks on business software usually need invariants instead of an exact hash, because IDs, timestamps and compensating records legitimately differ from any snapshot.
| Verdict | Meaning | Evidence required |
|---|---|---|
| Restored | All invariants hold and no side effect occurred or remains | Post-undo reads matching every invariant; empty side-effect log |
| Compensated | All invariants hold; side effects were offset by recorded compensating actions | Reads plus IDs of each compensating action |
| Partially compensated | Some invariants hold; others or some side effects remain open | Reads showing which checks failed and why |
| Irreversible, acknowledged | An effect could not be undone and the agent recorded it and took the agreed follow-up | Message or filing log, plus the follow-up action |
| Drifted | The undo itself caused new damage, such as a duplicate, a lost concurrent edit or an erased audit entry | Reads showing the new discrepancy |
Limits of the method, and when the business owner has to decide
Invariant testing has limits worth stating in every task set. It depends on the side-effect inventory being complete, and nobody knows every downstream integration in a large business; treat each inventory as a claim by the process owner and add channels when a test surfaces one. It also depends on read access to the systems where effects land, which a sandbox may not mirror.
Some decisions are not the agent's or the task author's to make. Whether a partial refund should be reversed or the customer simply re-billed, whether a correction email is appropriate, and whether a filing can be amended are business choices with customer, financial or legal weight. Mark these steps as requiring human approval in the task, grade the agent on stopping and asking at the right point, and route the policy question to the person who owns the process.
- Stop and escalate when the undo path would need elevated permissions the forward action did not use.
- Stop when a side effect reached an external party that the business, not the agent, must contact.
- Narrow the task when the system offers no read access to a channel on the inventory; grade only what can be read.
- Mark the task unverifiable rather than passing it when the evidence for an invariant is missing.
Records that already show reversals and their consequences
Operating businesses already hold real examples of this: orders cancelled and reinstated, tickets closed in error and reopened, refunds followed by re-charges, deletions restored from a recycle bin, each with the audit entries and downstream records that show what the reversal did and did not repair. Licensed with task context and de-identified, such records give agent builders realistic recovery cases instead of staged ones. Whether those records can be licensed is a separate question from whether they can be exported: privacy permission, ownership and the contract's licensing rights each need review, and de-identification alone does not create a right to license.
Reversal histories with their side-effect records can be requested from SourceX, which sources from U.S. businesses for each request rather than from stock; the request form covers scope, volume, language, format, timing and licensing needs, and that is where to ask for audit entries and downstream records to stay linked.
Frequently asked questions
What should the rollback invariant and side effect verification task include?
The intent and the mistake being recovered from, the target records, a pre-action snapshot, a side-effect inventory, the undo path and its window, written invariants with read queries, a concurrency allowance, the observed post-undo reads and a verdict with evidence. Fill everything except the observed reads before the agent runs.
What proves the intended business state changed back rather than only the interface?
Reads from the system of record after the undo that satisfy each invariant, plus evidence for every item on the side-effect inventory: message logs, payment object IDs, webhook logs and audit entries. A screenshot of the restored page proves only what the page displays.
Should a restored record match the pre-action snapshot exactly?
Its business fields should, but system-managed fields such as last-modified time, version counters and compensating record IDs legitimately differ. Exclude those from the comparison and declare which fields other users may change during the run.
Does an email client's undo send recall a delivered message?
Gmail's Undo Send cancels a message only during the cancellation period you set, of 5, 10, 20 or 30 seconds, because the message has not yet been sent. After that window the message is delivered, so a task should treat it as an irreversible side effect.
Can MCP tool annotations tell an agent whether an action is reversible?
They can suggest it. The MCP schema defines destructiveHint, idempotentHint and openWorldHint, but states that annotations are hints that may not describe the tool faithfully. Confirm reversibility by testing the tool in a sandbox and recording what its undo really restores.
Sources
- tau-bench computes a hash of the environment database after an episode and compares it with the hash produced by replaying the ground-truth actions; a mismatch sets the reward to zero. Source
- OSWorld task examples define a config of setup steps that create the task's initial state and an evaluator that holds the evaluation for the example. Source
- The MCP 2026-07-28 schema defines readOnlyHint, destructiveHint, idempotentHint and openWorldHint tool annotations and states that all annotations are hints that clients should not rely on from untrusted servers. Source
- The Compensating Transaction pattern says data cannot always be rolled back, compensation is application specific and may not restore the starting state, steps should be idempotent, and points of no return should be defined. Source
- Stripe refunds are created against an existing charge or PaymentIntent, and a charge can be partially refunded multiple times until it is fully refunded. Source
- Salesforce keeps deleted items in the Recycle Bin for 15 days, during which users can restore them, and then schedules them for permanent deletion. Source
- Salesforce restores master-detail children deleted in a cascade when the parent is restored, while a child deleted before its parent becomes hard deleted and must be recreated. Source
- Gmail's Undo Send lets users set a send cancellation period of 5, 10, 20 or 30 seconds. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.