AI data market
Workflow versus document dataset value: a path test on 14,521 real cases
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
A workflow dataset beats documents when the work is fragmented. In two public event logs we measured on 2026-10-27, the most common path covered 49.7% of 1,434 permit-receipt cases but only 26.2% of 13,087 loan applications, where 28.7% of cases followed a path no other case took. Documents state the rules; logs show paths, timing and choices. Measure path concentration before paying a workflow premium.
Value drivers
Workflow
Key takeaways
- Path concentration, the share of cases on the most common paths, is a quick and comparable gauge of what a workflow dataset adds over a written procedure.
- In the permit-receipt log, 6 of 116 paths cover 80% of cases; in the loan log it takes 1,749 of 4,366.
- The 2,246 approved loan applications followed 2,067 distinct paths, and 28.7% of applications with an offer received more than one.
- Documents still supply what logs cannot: reasons, authority and rules for cases that have not happened yet.
- An event log alone trains forecasts; an agent also needs an environment to act in and outcomes tied to each decision.
What can a workflow dataset reveal that a document cannot?
A workflow dataset reveals how work actually moved: the order of steps, how long each case took, which branches people chose and how often, and where work looped back. A document, such as a procedure, policy or playbook, reveals what the organization intended, why, and what to do in situations no log has recorded yet. The first is evidence of behavior over time; the second is a statement of rules at a point in time.
For an investor the question is not which asset is better but when the workflow record deserves a premium. Agent builders need the states, actions, constraints and outcomes of real tasks. If a written procedure already predicts what happens in most cases, a workflow dataset adds little beyond confirmation. If cases scatter across many paths, the record carries information no document holds, and that information is scarce because it exists only inside the company that did the work.
That makes workflow versus document dataset value something to measure rather than argue. This article measures it on two public, real-life event logs, proposes path concentration as a comparable gauge, maps which agent inputs each asset supplies and names the findings that would make a workflow premium unjustified.
Two public event logs, measured the same way
We counted paths, durations and decision branches in two event logs that are public, real and documented, so anyone can rerun the numbers. Both were read on 2026-10-27 from the test data distributed with the open-source PM4Py process mining library, at repository commit 9521745 of October 19, 2026, and both carry their original 4TU.ResearchData metadata inside the file.
The first is the BPI Challenge 2012 log, which 4TU.ResearchData describes as an event log of a loan application process, first published online on April 23, 2012. Its embedded metadata names a proprietary workflow system as the source and lists 13,087 cases and 262,200 events, which our count matches; timestamps run from October 1, 2011 to March 14, 2012.
The second comes from the CoSeLoG research project and records the receiving phase of a building permit application process in an anonymous municipality in the Netherlands. Its metadata labels the process explicitly structured and the source a case handling system; it holds 1,434 cases and 8,577 events between October 2010 and January 2012. Comparing a loan process with a permit process is the point: two kinds of administrative work, measured with one method.
The path concentration test: one procedure, how many cases?
Path concentration asks how many distinct sequences of steps it takes to account for most cases. In the permit log, the most common path, six steps from confirming receipt to deciding whether a stop is needed, covers 49.7% of cases, and 6 of its 116 paths cover 80%. In the loan log the most common path covers 26.2% of cases, and it is an application submitted and declined within about two minutes, 38 seconds at the median. Half of the loan cases need 12 paths; 80% need 1,749 of 4,366.
Unique paths are the sharpest signal. In the loan log, 3,754 cases, 28.7% of the total, followed a sequence no other case followed; in the permit log the figure is 86 cases, or 6.0%. A written procedure describes one intended path per case type plus the exceptions its author foresaw. A log with that many singular paths is recording behavior that no procedure listed in advance.
The test reads differently by process, and that difference matters more for pricing than record counts. The permit log is the kind of process a maintained procedure can describe: one path and a short exception list cover most cases. The loan log is the kind where a document explains the rules but not the work. Split by path rank, the permit log's next nine paths hold 38.1% of cases, other shared paths 6.1% and unique paths 6.0%; in the loan log the same bands hold 23.1%, 22.0% and 28.7%.
Methods note for the test: counts come from our own parse of the two files on 2026-10-27, using each file's activity classifier (activity name plus lifecycle transition) to define a path. Durations run from a case's first to its last recorded event, with nearest-rank percentiles. The figures describe these two logs only; they are not estimates for any industry, and they say nothing about the value of any company's data.
| Measure | Permit receipt log | Loan application log | What it tells a buyer |
|---|---|---|---|
| Cases and events | 1,434 cases; 8,577 events | 13,087 cases; 262,200 events | Scale alone says little about information content |
| Distinct paths | 116 | 4,366 | How many ways the same work actually ran |
| Share on the most common path | 49.7% | 26.2% | The most a single written path could describe |
| Paths needed for 80% of cases | 6 | 1,749 | Whether a short exception list could cover the rest |
| Cases on a path no other case took | 86 (6.0%) | 3,754 (28.7%) | Behavior no procedure enumerated in advance |
| Median and 90th-percentile case duration | 48 minutes; 14.1 days | 0.81 days; 30.5 days | Timing spread a document rarely states |
Choices over time: what the loan log records that a policy page cannot
The loan log records decisions as they unfolded, and those decisions are what make it valuable. Of 13,087 applications, 2,246 (17.2%) reached approval, 7,635 (58.3%) were declined, 2,807 (21.4%) were cancelled and 399 (3.0%) had no final decision in the log. A credit policy can say who qualifies. It cannot say that the 2,246 approved applications followed 2,067 distinct paths, so that almost no two approvals moved through the institution the same way.
Offers show the temporal choices most clearly. Of the 5,015 applications that received at least one offer, 1,438 (28.7%) received more than one, so the offer changed while the case was open; 802 applications were declined after an offer had already been sent. Each is a state change made in response to something that happened mid-case, which is exactly the kind of decision an agent doing the same work would face.
Timing adds a dimension procedure pages rarely record. In the loan log, 51.9% of cases ended within a day, a share that includes the quick declines on the most common path, while one case in ten ran 30.5 days or longer. A model trained only on documents would not know the slow tail exists, let alone where in the sequence the elapsed time builds up.
What a document still reveals that a log cannot
Documents carry three things no event log does: the reason behind a choice, the authority that allowed it and the rule for a case that has not happened yet. The loan log records that an application was declined; it does not say why. It records which resource performed each step, as an anonymized identifier, but not whether that person could approve the amount involved. And a path that never occurred leaves no trace at all, even when the policy covers it.
The permit log shows the other side of the trade. Its metadata calls it an explicitly structured process, and the counts agree: one path covers about half of all cases and six cover 80%. For a process like that, the procedure holds most of the value, and the log mainly confirms it and quantifies the exceptions. Paying a workflow premium there buys little that a careful reading of the procedure would not.
The strongest asset is therefore the pair: the log joined to the procedure or policy version that governed each case. The document supplies reasons and rules for unseen cases; the log supplies frequencies, timing and the choices people actually made. An agent builder needs both, linked case by case.
Agent-task input map: which asset supplies each input
Map each input an agent task needs to the asset that can supply it, and the gaps show up at once. A supplier offering workflow data should be able to say which rows its records fill; a supplier offering documents should say which rows it leaves to the buyer.
Read the rows as a value chain as well. Documents are usually the easiest asset to obtain and say what should happen. Event logs add frequencies, order and timing. A linked case packet, meaning the log joined to the governing policy version and the later outcome, adds the evidence of whether a decision was right. A sandbox seeded from such packets lets an agent act, and an evaluation suite scores it on outcomes. Each link adds value and cost, and each can be a separate company's business.
The map below follows the states, actions, constraints and outcomes an agent must handle; the last column names what neither asset supplies without further linking:
| Agent input | A document supplies | An event log supplies | Neither supplies without linking |
|---|---|---|---|
| Starting state of a case | The intended entry conditions | The first recorded event and its time | The full file as it stood at that moment |
| Available actions | The steps the procedure permits | The steps people took, with frequencies | Actions considered but not taken |
| Constraints and rules | Rules, thresholds, authority and reasons | Only constraints that left a visible trace | Which policy version governed each case |
| Timing and sequence | Target timelines, where stated | Actual order, elapsed time and spread | Why a case waited |
| Branch choices | The conditions for each branch | How often each branch was taken, and when | What the decider could see at the branch |
| Outcome | The intended result | The final recorded event, such as approved or declined | What happened after the process ended |
| An environment to act in | Nothing | Nothing | A sandbox seeded from real cases |
Is an event log enough to train an agent?
No. An event log is enough to train forecasts, not actions, and this is the strongest objection to a workflow premium. Tax, Verenich, La Rosa and Dumas showed that LSTM neural networks trained on event logs can predict the next event of a running case and its timestamp, the full continuation of the case and its remaining time. That is useful for planning, but predicting what usually comes next is different from doing the next step well.
Agent benchmarks are built around environments for that reason. WorkArena is a remote-hosted benchmark of 33 tasks built on the ServiceNow platform, where the agent works inside the application itself, and its follow-up, WorkArena++, adds 682 compositional tasks aimed at planning and reasoning. A log of what employees once did is not an instance an agent can act in.
Two gaps follow. A record collection may lack the environment, so it can describe behavior but not let an agent attempt it. It may also lack outcome evidence: the loan log stops at the institution's decision, with no record of whether an approved loan was repaid or a declined applicant borrowed elsewhere. A workflow dataset earns an agent premium only when it can seed an environment or tie decisions to outcomes; otherwise it should be priced as forecasting data.
What could make a workflow premium unjustified
Four findings would shrink the premium for workflow records over documents, and a buyer or investor can check each one before paying for it:
- Fragmentation can be noise. Some unique paths reflect duplicated events, logging quirks or inconsistent step names rather than real choices; if cleaning collapses most singular paths, the log adds less than its raw count suggests. Ask for path concentration before and after normalization.
- Generated logs could substitute. If event logs simulated from a written procedure reproduce real path and duration distributions closely enough for agent work, the scarce part of the record disappears. Compare a supplier's real paths with a simulation of its own procedure.
- Public environments could absorb demand. WorkArena and WorkArena++ already offer interactive enterprise tasks built without any company's history; if such environments keep improving, buyers may pay for sandboxes rather than records.
- People are in the log. Even anonymized, event logs record which resource handled each step, and the loan log alone carries 69 resource identifiers, so rights, employee privacy and de-identification can narrow what a company may license.
Six measures that put two workflow datasets on equal terms, SourceX included
Ask every supplier for the same measures, computed the same way, and comparisons stop depending on adjectives such as rich or comprehensive. Then ask SourceX the same questions.
SourceX licenses operating records from companies to AI developers and sources each dataset after a buyer describes what it needs, so the path profile of any dataset depends on the company and process behind it. The company approves scope and terms and signs before any records are delivered to SourceX, and de-identification is carried out by SourceX or its processing partner. SourceX has not published path, duration or outcome-coverage figures for any dataset and shares company and commercial metrics with investors directly. It is raising from accredited investors under Rule 506(c); materials are available on request at sourcex.si/investors.
The six measures:
- Path concentration: the top path's share, the paths needed for 80% of cases and the share of unique paths, with the classifier named.
- Duration spread: median and 90th-percentile case duration, with cases still open reported separately.
- Branch counts: for each decision point, how many cases took each branch.
- Outcome coverage: the share of cases with a final outcome in the data, and whether anything after the process end is linked.
- Policy linkage: the share of cases tied to the procedure or policy version in force when they ran.
- Normalization effect: the same measures before and after duplicate and naming cleanup.
Methods note: files, definitions and what was not measured
Both logs were parsed on 2026-10-27 with a short Python script that streams each XES file and counts cases, events, paths, outcomes and durations; no process model was fitted and no benchmark code was run. Event logs were read from PM4Py's repository at commit 9521745; the loan log's description and publication date were confirmed on 4TU.ResearchData through a search excerpt, and both files' embedded metadata was read directly. Paper facts for Tax and colleagues and for WorkArena were confirmed through search excerpts on arxiv.org and should be opened by a reviewer before release.
Not measured: the value of either log to any buyer, how path concentration varies across industries, whether real paths beat simulated ones for agent training, and anything about SourceX datasets. The loan log's approved, declined and cancelled shares count cases containing the corresponding decision event; no case contains more than one of the three. The path concentration test and the input map are editorial frameworks.
Frequently asked questions
How does a workflow dataset differ from a document dataset?
A document dataset holds procedures, policies, manuals and similar text that state how work should be done. A workflow dataset holds the recorded events of work as it happened: which steps ran, in what order, when and with what result. Documents explain rules; workflow records show behavior across many cases.
What is path concentration in an event log?
It is the share of cases covered by the most common sequences of steps. High concentration means a written procedure describes most of the work; low concentration means cases scatter across many paths that only a log records. Report it as the top path's share, the number of paths covering 80% of cases and the share of unique paths.
Can an event log train an AI agent on its own?
It can train models that forecast the next step, its timing and the remaining time of a case. Acting well also needs an environment where actions change state and outcomes tied to each decision, which most logs lack. A log becomes agent data when it seeds such an environment or links to outcomes.
Are public process mining logs useful for AI training?
They are useful for testing pipelines and methods, as this article does, because they are real and openly documented. Each covers one organization and one process, is years old and lacks reasons and post-process outcomes, so it shows what workflow data looks like rather than supplying an agent-ready dataset.
Does SourceX publish path or outcome metrics for its datasets?
No. SourceX has not published path concentration, duration or outcome-coverage figures; datasets are sourced per buyer request and would be measured case by case. Company and commercial metrics go to investors directly, and materials can be requested at sourcex.si/investors.
Sources
- The BPI Challenge 2012 event log distributed in PM4Py's test data carries 4TU metadata: a real-life loan application process from a proprietary workflow system, 13,087 cases, 262,200 events and 4,366 distinct traces. Source
- The CoSeLoG receipt log in PM4Py's test data records the receiving phase of a building permit process in an anonymous Dutch municipality, labeled explicitly structured, with 1,434 cases and 8,577 events. Source
- 4TU.ResearchData describes BPI Challenge 2012 as an event log of a loan application process, first online on April 23, 2012. Source
- Tax, Verenich, La Rosa and Dumas show LSTM networks trained on event logs predicting the next event and its timestamp, a case's full continuation and its remaining time. Source
- WorkArena is a remote-hosted benchmark of 33 tasks on the ServiceNow platform. Source
- WorkArena++ adds 682 compositional planning and reasoning tasks to the WorkArena setting. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.