AI data market
Cross-application agent data: the gap is real; is it a data gap?
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
Cross-application agent data is investable only if linked workflows close a gap that single-system datasets cannot. The gap is measured: in OfficeBench (2024), GPT-4 Omni passed 64.52% of single-app tasks but 21.43% of three-app tasks, and WindowsWorld (2026) found every tested agent below 21% on multi-application work. Whether real linked records, rather than simulated environments, close it is unproven; make suppliers show it.
Agent run
Cross application agent data
Key takeaways
- The cross-application gap is measured, not assumed: benchmarks from 2024 to 2026 show agents failing far more often once tasks span several apps.
- A measured gap is not yet a data gap; OfficeBench's small drop at two apps and cliff at three point to task length and planning as well as switching.
- Linked real records and simulated environments have opposite strengths: realism and observed handoffs on one side, verifiable outcomes and clean rights on the other.
- Platform terms can block the linking itself; Google's Workspace developer policy bars using API user data to train models beyond that user's personalized model.
- Underwrite a supplier on one comparison: real linked workflows against an equal budget of simulated tasks, scored on held-out cross-app work.
What would make cross-application agent data investable?
Cross-application agent data is investable when three things hold at once: agents fail measurably more often on work that spans applications, that failure comes from evidence single-system datasets lack, and someone can supply the linked evidence with rights intact at a cost below the alternatives. Public research now settles the first condition. The second and third are open, and they decide the thesis.
Cross-application agent data here means records of one task followed across the systems it touched: the request in email, the change in a CRM, the approval in a ticketing tool, the invoice in accounting, joined so the order of actions, the roles involved and the end state can be read together. A single-system dataset holds one of those slices, however long its history.
Work crosses applications constantly. In a study reported in Harvard Business Review in 2022, researchers who tracked 137 users on 20 teams at three Fortune 500 companies found people toggling between applications about 1,200 times a day. That makes the category plausible. Plausible is not investable, so the rest of this article tests each link from benchmark gap to paid data.
How large is the cross-application gap in public benchmarks?
Large, and consistent across independent benchmarks. Teams working on different platforms between 2024 and 2026 each found agent success falling once a task spans more than one application.
OfficeBench (Wang et al., 2024) split 300 office tasks by the number of applications involved. GPT-4 Omni, the best model it tested, passed 64.52% of the 93 single-app tasks, 60.00% of the 95 two-app tasks and 21.43% of the 112 three-app tasks, or 47.00% overall. OSWorld (Xie et al., 2024) counted 101 multi-app workflow tasks among its 369, or 27.4%, and its strongest baseline agent completed only 6.57% of them. WindowsWorld (2026) built 181 tasks across 17 desktop applications and reports that every computer-use agent it tested stayed below 21% success on multi-application tasks, far below single-app performance.
Two caveats keep the reading honest. The scores belong to the model generations tested and will move as models improve. And the WindowsWorld authors note that multi-application tasks make up under 28% of widely used benchmarks, so the field has measured this gap far less than single-app skill. The table sets out what each benchmark shows and what it cannot:
| Benchmark | Cross-app measure | Reported result | What it cannot show |
|---|---|---|---|
| OfficeBench, 2024 | 300 tasks split into one, two and three apps | GPT-4 Omni: 64.52%, 60.00%, 21.43% | Whether real business records would close the drop |
| OSWorld, 2024 | 101 multi-app workflow tasks out of 369 | Best baseline agent: 6.57% on workflows | Whether failures come from switching or from task length |
| WindowsWorld, 2026 | 181 tasks across 17 desktop applications | All tested agents below 21% on multi-app tasks | How newer agents score after publication |
| WorkBench, 2024 | 690 tasks over five business databases, graded on final state | ReAct with GPT-4: 43% of tasks correct | How a real company's systems differ from a sandbox |
Is the gap about missing data, or about longer tasks?
Partly longer tasks, and an investor should insist on separating the two before paying for data. OfficeBench's own numbers show why: going from one app to two cost GPT-4 Omni about 4.5 points, while going from two to three cost almost 39. If switching applications were the problem in itself, the first switch should hurt most. A cliff at the third app looks more like compounding steps, deeper planning and grounding actions across a larger combined action space, which are the skills the OfficeBench authors say their tasks demand.
The two explanations call for different fixes. If agents fail on long plans, more long single-system episodes may help as much as cross-app ones. If they fail on handoffs, such as finding the same customer under three identifiers or carrying an approval from a ticket into an ERP, only linked records or environments that model those handoffs will help. WorkBench shows the stakes: its authors note that agent errors can mean a wrong action taken, such as an email sent to the wrong person.
A supplier pitching cross-application data should name the failure its data targets and show task-level evidence for it, not only the headline drop.
The single-system gap ledger: six requirements, three kinds of evidence
The ledger scores how well each kind of evidence covers six things an agent task needs, from 0 for absent to 3 for fully observed. It compares a deep single-system export, linked records of the same tasks across applications, and a simulated multi-application environment of the kind benchmark teams build.
Read the profiles, not totals. Linked records lead where reality matters: handoffs observed as they happened and the real constraints and exceptions people worked under. Simulated environments lead where checking matters: a known starting state, replayable actions and an end state a script can verify, with no third-party rights to clear. The single-system export leads only on rights, since one owner and one set of terms is the simplest case to license. The scores are editorial judgments made for this article, illustrative and meant to be redone for any specific supplier:
| Requirement | Single-system export | Linked cross-app records | Simulated environment |
|---|---|---|---|
| Initial state across apps | 1: one app's state only | 2: rebuilt from records, gaps between snapshots | 3: set by the environment |
| Actions in order across apps | 0: other apps invisible | 2: ordered from timestamps and logs | 3: logged and replayable |
| Identity and handoffs | 0: no other system to hand off to | 3: observed as they happened | 2: authored by task designers |
| Real constraints and exceptions | 2: real but partial | 3: real, across the whole task | 1: written into the task |
| Verifiable outcome | 1: final state in one app | 2: downstream record, if retained | 3: checked by a script |
| Rights clarity | 3: one owner, one set of terms | 1: every app adds owners and terms | 3: no real records involved |
Where value sits in the cross-application chain
Value gathers at the links that are hardest to rebuild, and in this chain those are identity resolution and outcome verification, not access. The ledger's profiles predict it: what linked records uniquely supply sits in the middle of the chain, while access is rented from platforms and verification can be simulated.
At a glance
Where value sits in the cross-application chain
Follow one task from raw records to a paid dataset and five links appear, each with a rival that could take it:
- Access in each application: the export or API that gets records out. Platform owners control it and can change terms, so treat it as a cost, not a moat.
- Identity resolution and linking: proving that the customer, order and approver in one system are the same ones in the next. This is specialist work on the company's own reference data and the hardest link for an outsider to copy.
- Episode assembly: ordering states, actions and constraints into a task with a start and an end. Labs and environment builders can do this in house once they hold linked records.
- Verification: an end state or outcome a grader can check. Simulated environments supply it cheaply, which makes them the main substitute.
- Buyer acceptance: a held-out test showing the data improves agent success. Until a supplier passes it, every earlier link is cost.
The strongest alternative: environments, not records, may close the gap
The most serious objection is that benchmark builders already solve verification without licensing anyone's records. AppWorld (Trivedi et al., 2024) runs nine everyday apps through 457 APIs, filled with simulated digital activity for about 100 fictitious users, and grades 750 tasks with state-based unit tests that also check for collateral damage. WorkBench grades on the expected state of its sandbox databases. Neither needed a company's archive.
If environments like these, scaled up, teach agents cross-app work, the premium for real linked records shrinks to calibration: making simulated tasks resemble real ones. That is a smaller business than supplying training data. Simulation is not easy for agents either; at publication GPT-4o solved about 49% of AppWorld's normal tasks and about 30% of its challenge tasks.
The test that separates the two is concrete. Give one agent an equal budget of real linked workflows and another of simulated multi-app tasks, then score both on held-out tasks drawn from real cross-application work. If the real-records arm wins by more than run-to-run noise, the thesis holds for that task family. This article found no public study reporting that comparison for business workflows as of October 2026, which is both the opening and the risk.
What could make the cross-application thesis wrong
Five developments would weaken the thesis, and an investor can watch for each one:
- Platform terms close the linking route. Google's Workspace developer policy bars using user data from its APIs to create, train or improve a machine learning or AI model beyond that specific user's personalized model. Datasets assembled through third-party app access inherit limits like this; exports a company approves under its own authority are a different path, but every application still adds terms to read.
- Simulation proves sufficient. If simulated environments match real linked workflows on held-out real tasks, records become a calibration input rather than the product.
- Model progress closes the gap alone. If a later model generation scores as well on three-app tasks as on single-app ones without new cross-app data, the gap was about capability, not data.
- Linking multiplies privacy exposure. Each joined system adds attributes about the same people, so records that are harmless one system at a time can identify individuals once linked, and review costs rise with every application.
- Buyers do not budget for it. This article could verify no filing or announcement showing a model developer budgeting specifically for cross-application business records; demand is inferred from benchmarks, not observed in contracts.
Diligence questions for a cross-application data supplier, SourceX included
Ask questions that make a supplier show the link it claims to own, with denominators. The same list applies to SourceX.
SourceX covers a narrower slice than the whole thesis. It finds and qualifies companies that choose which of their records to license, reviews the rights, writes the contract, prepares the data, delivers it and pays the company, and that company approves scope and terms and signs before any records reach SourceX. Whether linked cross-application workflows from such companies beat simulated tasks is a result SourceX has not published. It shares company and commercial metrics with investors directly and is raising from accredited investors under Rule 506(c); materials are available on request through its investor page.
Questions worth asking, in order:
- Which cross-app failure does your data target, handoffs or long plans, and what task-level evidence shows it?
- In linked episodes, what share of joins rest on a shared identifier rather than inference, and how was each kind checked?
- How many applications does a typical episode span, and how many owners' terms had to be cleared for it?
- Has any buyer run real linked workflows against an equal budget of simulated tasks, and what was the held-out result?
- Which outcome in each episode can a grader verify without trusting your own labels?
- If one platform changed its API terms tomorrow, what share of your supply would stop?
Methods note: how the gap ledger was scored
Benchmark figures come from each paper's reported results: OfficeBench (arXiv 2407.19056), OSWorld (arXiv 2404.07972), WindowsWorld (arXiv 2604.27776), WorkBench (arXiv 2405.00823) and AppWorld (arXiv 2407.18901), checked on arxiv.org on 2026-10-27. Platform terms come from Google's Workspace developer policy page and the app-switching figure from Harvard Business Review (2022). Scores are not comparable across benchmarks, which differ in tasks, models and grading.
The gap ledger's 0 to 3 scores are editorial judgments for this article about typical cases of each evidence type; they are illustrative and should be redone for a specific supplier. Not measured: whether real linked records improve agent success over simulated tasks, buyer budgets for cross-application data, the cost of identity resolution per episode, and any SourceX supply, demand or performance figure.
Frequently asked questions
What is cross-application agent data?
Records of one task followed across every system it touched, such as email, CRM, ticketing and accounting, joined so an agent can see the order of actions, the roles involved and the end state. A single-system dataset shows only one of those slices.
Do AI agents really perform worse on multi-app tasks?
Yes, on every public benchmark that splits the two. OfficeBench (2024) reported GPT-4 Omni at 64.52% on single-app tasks and 21.43% on three-app tasks, and WindowsWorld (2026) found all tested agents below 21% on multi-application tasks.
Can simulated environments replace real cross-application records?
For verification, often yes: AppWorld and WorkBench grade agents on the final state of simulated apps without any company records. Whether simulation also captures real constraints, exceptions and handoffs well enough to replace real workflows has not been tested publicly, and that comparison should anchor any investment case.
What is the biggest risk for a cross-application data business?
Dependence on terms it does not control. Each application in a workflow brings its own owner and policies, and a platform can restrict how data obtained through its APIs is used, as Google's Workspace developer policy does for model training.
How should an investor evaluate a cross-application data supplier?
Ask for task-level evidence of the failure the data fixes, the share of joins that rest on shared identifiers, the number of owners whose terms each episode needed, and any held-out comparison against simulated tasks. A supplier that cannot answer with denominators is selling a hypothesis.
Sources
- In OfficeBench, GPT-4 Omni passed 64.52% of 93 single-app, 60.00% of 95 two-app and 21.43% of 112 three-app tasks, 47.00% overall; tasks require long-horizon planning, app switching and grounding in a large action space. Source
- OSWorld has 369 tasks, 101 of them (27.4%) multi-app workflows, on which the best baseline agent reached 6.57%. Source
- WindowsWorld's 181 tasks span 17 desktop applications; all tested computer-use agents scored below 21% on multi-application tasks, which make up under 28% of widely used benchmarks. Source
- WorkBench has 690 tasks over five databases, graded on the final database state; ReAct with GPT-4 completed 43%, and agent errors can mean wrong actions such as an email to the wrong person. Source
- AppWorld simulates nine apps via 457 APIs for about 100 fictitious users and grades 750 tasks with state-based unit tests; GPT-4o solved about 49% of normal and 30% of challenge tasks. Source
- Google's Workspace developer policy prohibits using user data to create, train or improve a machine learning or AI model beyond that specific user's personalized model. Source
- A 2022 study of 137 users on 20 teams at three Fortune 500 companies found workers toggling between applications about 1,200 times a day. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.