Deal economics
Data operations exception cost: budget the tail, not the average case
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
Data operations exception cost is the staff time spent on supplier cases that leave the standard path: unclear rights, broken exports and lost context. It concentrates; in one public data log, five of 102 sources produced 71.2 percent of 459 fixes. Ask for case-level hours on complete cohorts and a reserve sized to the 90th-percentile cohort: in an illustrative model, a budget at the average is overrun 47.1 percent of the time.
Value drivers
Data operations exception
Key takeaways
- Exception cost is the staff time on supplier cases that leave the standard path, and an average per supplier understates what a cohort consumes.
- In the JHU CSSE public data log, five of 102 sources produced 71.2 percent of 459 logged fixes, and by 2022 most entries were repeated corrections from those five.
- Corrections in that log reached back a median of 932.5 days, so the history that gives records value also sets the size of rework.
- In an illustrative 40-supplier cohort, a budget at the mean is overrun 47.1 percent of the time; a stop rule and a fix for standing exceptions raise capacity at 90 percent confidence from 40 suppliers to 50.
- Ask for exception hours per accepted dataset by cohort and channel, with every supplier that entered preparation in the denominator.
What is data operations exception cost, and why does the average hide it?
Data operations exception cost is the staff time a data company spends on supplier cases that leave its standard path: a rights question that needs counsel, an export that loses links or history, or records whose meaning has to be rebuilt with the people who made them. Each case is uncommon, and a few are very large, so an average per supplier understates what a cohort actually consumes.
The investor decision is whether a data company's operating leverage survives those cases. Three things settle it: whether exceptions are counted as cases with hours attached, whether they are budgeted apart from the standard path, and whether the budget is sized to a bad cohort rather than a typical one. A company that reports only an average preparation cost per dataset has answered none of the three.
Useful evidence covers complete supplier cohorts, meaning every supplier that entered preparation in a period, including those that later dropped out, with hours recorded per case, the event that opened each case, its type and its outcome: accepted, re-scoped or dropped.
Four ways a supplier case leaves the standard path
A supplier case becomes an exception when a named event forces work the standard path does not include. The opening event matters more than the label: without one, slow routine work gets relabeled as exceptional.
Exposure grows with the number of systems a supplier brings. SourceX's how-it-works page says qualifying companies list at least 5 systems, and each system is another place an export can fail. Dean and Barroso made the general point for computing in The Tail at Scale (Communications of the ACM, 2013): if one server answers slowly once in 100 requests, a request that waits on many such servers is slow far more often. If each system independently had a one-in-ten chance of an extraction exception (an illustrative rate), a five-system supplier would hit at least one 41.0 percent of the time.
The table gives each type its opening event and the illustrative rates the model later in this article uses. None of the rates or hours is a SourceX figure or a measured industry value:
| Exception type | Opening event | Chance per supplier (illustrative) | Typical and heavy case hours (illustrative) | What a heavy case usually means |
|---|---|---|---|---|
| Rights | A customer clause, notice or piece of third-party content needs a decision from counsel | 15% | 8 typical, 40 heavy; 1 case in 5 is heavy | The same clause has to be read across many customer accounts |
| Extraction | An export drops links, history or attachments, or a system has no usable export | 20% | 10 typical, 60 heavy; 1 case in 4 is heavy | A custom extraction or a rebuild of links between systems |
| Context recovery | Codes, fields or definitions changed during the history and their meaning must be rebuilt | 25% | 6 typical, 30 heavy; 1 case in 5 is heavy | Working sessions with the supplier's staff to decode retired codes |
| Standing exception | The same correction recurs on every re-export or refresh of one source | 5% | 8 repeats of 12 hours each | A source quirk that is never fixed upstream, so every pass repeats the work |
What a public exception log shows: 459 fixes, three years, five sources
In the public exception log examined for this article, exceptions concentrated in a handful of sources, shifted from one-off surprises to standing corrections, and usually reached back across most of the history. That is the pattern an exception budget has to absorb.
The log belongs to the Johns Hopkins CSSE COVID-19 repository, which compiled daily counts from public health sources around the world. Its data README keeps a section titled Data modification records that, in its own words, "will contain any modifications to our datasets as well as the reason for the change." At the repository's final commit (4360e50, March 10, 2023) the section holds 461 entries. Dropping two exact duplicate lines leaves 459, dated from February 13, 2020 to March 9, 2023, with at least one entry in each of the 38 calendar months.
Concentration is extreme. The entries name 102 distinct locations, and 79 of them appear once. The five most frequent, Ohio, California, Tennessee, North Dakota and the United Kingdom, account for 327 entries, or 71.2 percent; Ohio alone accounts for 113, or 24.6 percent. The README explains Ohio's run: to match the state's new reporting standard, its death series was overwritten with date-of-death data, "and the entire time series will be updated with each time Ohio updates their death data." One upstream change became a standing exception: 112 of Ohio's 113 entries contain that note's phrase "new reporting standards".
The mix moved over time. In 2020, 68 of 73 entries came from sources outside that top five. In 2022, the five accounted for 212 of 228 entries, or 93.0 percent, most of them repeated notes such as Tennessee's "Corrected from 2020-03-12 to 2022-02-26". Monthly counts had a median of 10 and a 90th percentile of 23, so a budget sized to a typical month would have been exceeded in 18 of the 38 months.
Corrections also reached far back. Of the 156 entries that name a dated correction range, the median start lay 932.5 days before the entry, about two and a half years, and 137 reached back more than two years. That matters for licensing because history is what buyers pay for: as a general guide they look for years of consistent history, typically 2 years or more. The depth that makes records valuable is the depth a correction may have to cross.
The log has limits: it comes from a public-data aggregator, not a licensing business; it records entries, not hours; and it holds only what the team chose to write down. It shows the shape of exception load, not its cost. Entries by year:
| Year | Entries | From the five most frequent sources | From the other 97 sources | Share from the five |
|---|---|---|---|---|
| 2020 | 73 | 5 | 68 | 6.8% |
| 2021 | 104 | 63 | 41 | 60.6% |
| 2022 | 228 | 212 | 16 | 93.0% |
| 2023, to March 9 | 54 | 47 | 7 | 87.0% |
The exception budget model: one cohort, four planning rules
In an illustrative cohort, a budget set at the average exception load is overrun almost half the time, while a reserve at the 90th percentile, a stop rule for one-off cases and a fix for standing exceptions together let the same team take more suppliers at 90 percent confidence. Every number below comes from a script and labeled assumptions.
The cohort has 40 suppliers entering preparation, 30 standard-path hours each and a team capacity of 2,000 hours, with the exception rates from the table above. The script computes the exact distribution of total exception hours, with no simulation. Mean exception load is 566.4 hours, 14.2 per supplier and 32.1 percent of all hours. Heavy one-off cases and standing exceptions make up 27.7 percent of expected cases but 74.2 percent of exception hours; the typical cases, nearly three in four of the total, take the remaining 25.8 percent.
The tail sets the budget. The 90th percentile is 790 hours, and a budget held at the mean is overrun in 47.1 percent of cohorts. Planning on the average admits 45 suppliers and leaves a 44.4 percent chance of overrunning capacity. Holding a 90th-percentile reserve admits 40.
Two controls recover that capacity. A stop rule sends any one-off case that reaches 16 hours to a scope decision, which cuts the 90th percentile to 610 hours at the cost of about 5.2 re-scoped cases per cohort. Fixing standing exceptions at the source, modeled as a 24-hour fix plus 2 hours per later repeat in place of 96 hours, brings it to 396. At 90 percent confidence the team then takes 50 suppliers instead of 40. The four planning rules side by side:
| Planning rule (illustrative) | Exception hours: mean and 90th percentile | Budget held | Chance the budget is overrun | Suppliers a 2,000-hour cohort can take |
|---|---|---|---|---|
| Plan on the average | 566.4 and 790 | 566 hours | 47.1% | 45, with a 44.4% chance of overrunning capacity |
| Hold a 90th-percentile reserve | 566.4 and 790 | 790 hours | 9.8% | 40 |
| Add a 16-hour stop rule for one-off cases | 421.6 and 610 | 610 hours | 9.8% | 44 |
| Stop rule plus a fix for standing exceptions | 305.6 and 396 | 396 hours | 10.0% | 50 |
Rules that turn a reserve into a budget
A reserve becomes a budget when it comes with rules for opening cases, stopping them and deciding what happens once it is spent, and public precedents exist for both halves.
At a glance
Rules that turn a reserve into a budget
The U.S. Freedom of Information Act lets agencies run multitrack processing of requests "based on the amount of work or time (or both) involved in processing requests," and it names the unusual circumstances that justify more time, among them the need to collect records "from field facilities or other establishments that are separate from the office processing the request" and to examine "a voluminous amount of separate and distinct records." Those are extraction and volume exceptions written into law, with their own lane. Google's SRE Workbook gives the other half: its sample error budget policy halts changes and releases, apart from the most urgent fixes, once a service has spent its four-week budget, and it calls for a postmortem when a single incident uses more than 20 percent of that budget (paraphrased). The rules below adapt both:
- Open a case only on a recorded trigger from the exception table, with the date, supplier, system and type. Work without a trigger stays on the standard path.
- Keep two budget lines. One-off cases draw on a pooled reserve sized to the 90th-percentile cohort; standing exceptions get a line per source, because in the public log five sources produced most of the entries.
- Stop each one-off case at a set hour mark for a scope decision: narrow the dataset, drop the record family, or continue with a named approver who accepts the cost.
- When the reserve is spent, new heavy cases wait for the next cohort instead of borrowing standard-path hours, so the standard path keeps its pace.
- After the third repeat, a standing exception is fixed at the source, built into that supplier's standard path or ended. It never stays in the reserve.
- Record history depth and system count at intake, because correction reach-back and fan-out both scale with them.
- Reconcile monthly: hours by type against budget, and a short review of any single case that took more than a fifth of the reserve.
Strongest alternative: more applications buy more exceptions, not more supply
The strongest competing explanation for rising preparation cost is simpler than exceptions: more applications reach preparation, and more of them are suppliers that will never close. In that case exception hours grow with volume while accepted supply stays flat, and an exception budget only manages the symptom.
Separate the two by tying hours to outcomes. Report exception hours per accepted dataset, not per supplier, by intake channel and cohort, and report the share of exception hours spent on suppliers that never reached acceptance. If applications double while accepted datasets hold steady and the second share rises, growth is buying exceptions rather than supply.
Concentration sharpens the test. If a data company's heaviest exception sources are also its least likely to close, most of its exception spending buys nothing, and the fix belongs at intake, not in a larger reserve.
What could make an exception budget misleading
An exception budget can look disciplined and still fail. Four developments would make it misleading, and each can be checked against records a company should already hold:
- Tails heavier than the model. Flyvbjerg and colleagues analyzed 5,392 IT projects (Journal of Management Information Systems, 2022) and found that cost overruns follow a power-law distribution with a fat tail, making such projects far riskier than decision makers normally assume. Under tails like that, a 90th percentile estimated from a few cohorts is unstable, and per-case stop rules carry more of the protection than the size of the reserve.
- Correlated exceptions. The model treats suppliers as independent. A platform change, a new rule or one upstream reporting change can hit many suppliers in the same month, as one reporting change in Ohio produced 112 entries in the public log, and a reserve sized for independent cases will be too small.
- Stop rules that cut the valuable cases. Hard cases are often the rare records buyers want most. A stop rule should force a decision with a named approver, not an automatic drop, and the re-scoped datasets should still be tested against the buyer's acceptance criteria.
- Unrecorded exceptions. If staff absorb exceptions without opening cases, the budget looks healthy while the standard path slows. Compare time records with the case log before trusting either.
Exception cost questions for any data company, SourceX included
Ask for case-level evidence from complete cohorts, on your definitions, and ask how the company decides when to stop. SourceX belongs on the same list.
SourceX's published process shows where exception cases would open. Its first step asks a few questions and takes no files: the applicant names its systems, years of history and what it can export, and SourceX then checks fit, builds out the inventory together with the company, reviews the rights and picks a processing route. Before the opportunity is presented externally, both sides sign a Dataset Deal Confirmation that fixes the exact dataset, the payout, the rights, the processing route and the deadlines, and after SourceX gives the go-ahead the data is extracted, transferred and de-identified through approved processing routes. Rights, extraction and context cases would surface at those steps.
SourceX publishes no exception, cohort or cost figures; its investors page says it shares company and commercial metrics with investors directly. Accredited investors looking at its Rule 506(c) raise can request materials there and put these questions to the company:
- What share of preparation hours went to exception cases in each of the last four cohorts, with every supplier that entered preparation in the denominator?
- Which recorded events open an exception case, and how many cases opened without one?
- What is the 90th-percentile cohort exception load, and how was the reserve sized against it?
- At what hour mark does a case go to a scope decision, who decides, and how many re-scoped datasets were later accepted?
- Which sources produced repeated exceptions, and what happened after the third repeat?
- What share of exception hours went to suppliers that never reached acceptance, by intake channel?
Methods note: the log count, the model and what neither measures
The public log count reads one file, csse_covid_19_data/README.md, in the JHU CSSE COVID-19 repository at commit 4360e50239b4eb6b22f3a1759323748f36752177. The script counts lines that begin with an asterisk and a space between the Data modification records heading and the next section heading; the section has no indented bullets and no prose entries apart from its opening description and format note, three lines the script prints. An entry's date is the leading month name, day and four-digit year followed by a colon or vertical bar, checked against the calendar, using the first day of a day range. Exact duplicate lines are dropped (two). A location label is the text between the date and the first vertical bar, case-folded with spaces collapsed; spellings such as Massachusetts and Massachusetts, US are not merged, which if anything understates concentration. Months run from February 2020 to March 2023 with empty months counted, and none was empty. Reach-back uses entries containing a range written as one date, the word to, and a second date, in year-month-day or month-day-year form with both dates valid, measured from the earliest range start; ranges without a year are ignored. Repeat notes are counted as a label's entries containing the phrase new reporting standards, case-insensitive. Percentiles use the nearest-rank method, the median of an even count is the mean of the two middle values, and percentages are rounded once, half up.
The exception budget model is an editorial construction computed by a second script with exact integer arithmetic and no random draws. Its inputs are the illustrative rates and hours in the exception table, 40 suppliers, 30 standard hours per supplier, 2,000 hours of capacity, a 16-hour stop rule that caps every one-off case at 16 hours, and a standing-exception fix of 38 hours in place of 96. Percentile h is the smallest total with at least that share of probability at or below it; supplier counts are the largest cohort, searched from 30 to 60 suppliers, whose standard hours plus the mean, or plus the 90th percentile, fit within capacity, and the overrun chance is the probability that exception hours exceed capacity minus standard hours. The 41.0 percent fan-out figure is 1 minus 0.9 to the fifth power. Both scripts and their outputs are kept for recount; the other sources were read on 2026-10-28.
Not measured: hours per exception at SourceX or any other data company, real exception rates by type, correlation between suppliers, and whether re-scoped datasets keep their value to buyers. None can be estimated from outside without company records that have not been published.
Frequently asked questions
What is an exception budget in data operations?
It is a reserve of staff hours set aside for supplier cases that leave the standard path, together with rules for opening, stopping and closing those cases. The reserve is sized to a bad cohort, not an average one, and it is tracked apart from routine preparation so exceptions cannot quietly slow the standard work.
How large should an exception reserve be?
Large enough to cover the cohort the company should expect about one time in ten, which means estimating the 90th percentile of cohort exception hours from case-level records. In this article's illustrative model that is 790 hours against a mean of 566.4, and a budget held at the mean is overrun in 47.1 percent of cohorts.
Why do standing exceptions need their own budget line?
Because they recur at one source instead of arriving at random, and they can absorb a pooled reserve. In the public JHU log, five of 102 sources produced 71.2 percent of 459 entries, mostly repeated corrections. A recurring exception should be fixed, built into that source's standard path or ended.
Does a stop rule mean throwing away hard cases?
No. A stop rule forces a decision at a set hour mark, and continuing is one of the options when a named approver accepts the cost. Hard cases can be the rare records buyers value most, so the rule exists to make that trade visible rather than to drop them by default.
Does SourceX publish exception or cohort metrics?
No. SourceX has not published exception rates, cohort hours or preparation costs, and its investors page says company and commercial metrics are shared with investors directly. The model in this article is illustrative and contains no SourceX data.
Sources
- The JHU CSSE COVID-19 repository README (csse_covid_19_data/README.md, commit 4360e50) keeps a Data modification records section listing changes to the datasets and their reasons; counts in this article come from that section. Source
- Dean and Barroso (Communications of the ACM, 2013) show that variability in individual components is magnified at the service level: a server slow on one request in 100 makes one user request in 100 slow, and far more once a request fans out across many servers. Source
- The Freedom of Information Act lets agencies provide for multitrack processing of requests based on the amount of work or time involved in processing them. Source
- The FOIA statute text published by the Justice Department lists unusual circumstances, including collecting records from field facilities separate from the processing office and examining a voluminous amount of separate and distinct records. Source
- Google's SRE Workbook sample error budget policy halts changes and releases, apart from the most urgent fixes, after a service exceeds its four-week error budget, and requires a postmortem when one incident uses more than 20 percent of the budget. Source
- Flyvbjerg and colleagues (Journal of Management Information Systems, 2022) analyzed 5,392 IT projects and found cost overruns follow a power-law distribution with a fat tail, making IT projects far riskier than decision makers normally assume. Source
- SourceX's how-it-works page describes a first step of a few questions with no files, a Dataset Deal Confirmation fixing the exact dataset, payout, rights, processing route and deadlines, processing through approved routes, and says qualifying companies list at least 5 systems. Source
- SourceX's investors page says it shares company and commercial metrics with investors directly and offers investor materials on request. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.