Deal economics
Dataset pricing for linked versus unlinked records: rows or complete cases?
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Ten thousand complete business cases are more useful than one million rows when the buyer's task needs the path from request to outcome, such as agent evaluation or outcome prediction. A million unlinked rows win for per-record tasks like classification or extraction. Dataset pricing for linked versus unlinked records follows usable units for the buyer's task, linkage rate, rights in every system and preparation effort, not row count.
Value drivers
Dataset pricing for linked
Key takeaways
- The buyer's task decides whether complete linked cases or a large set of unlinked rows is more useful.
- Compare packages built from the same source, period, rights and de-identification so only packaging differs.
- Linkage rate and whether linked cases skew toward easy work can change the answer.
- Every linked system's contracts apply to the case, and linking raises re-identification risk.
- A linked core plus an unlinked tier, each documented and rights-checked separately, often serves more buyers.
When are 10,000 complete business cases more useful than one million rows?
Ten thousand complete cases are more useful when the buyer's task needs the path from a request to its outcome: training or testing agents, predicting how a case ends, or grading decisions against what actually happened. One million unlinked rows are more useful when each row stands on its own, such as classifying messages, extracting fields from documents or learning domain language. The deciding factor is the buyer's task, not the row count.
That is also how dataset pricing for linked versus unlinked records works in practice. A buyer values the units it can use for its task. For an agent evaluation team, a row that cannot be joined to the request, the decision and the result is not a usable unit, however many of them there are. For a team training a text classifier, a large, varied, well-labeled set of standalone records may be exactly the unit it wants.
As a general guide, AI buyers look for years of consistent history, typically two or more, and tens of thousands of records or more in one system, while smaller sets can qualify when the work is rare; every dataset is assessed deal by deal. This page gives you a method to compare the two packagings of your own records on equal terms before anyone talks about value.
Why does linkage change what a dataset is worth to a buyer?
Linkage changes worth because it preserves the relationships a model needs to learn cause and outcome, and flattening those relationships away cannot be undone by the buyer. A support ticket alone shows a question; the same ticket joined to the bug, the fix, the release and the closing reply shows what happened and why.
Research on relational data makes the same point in technical terms. The RelBench paper, a benchmark for deep learning on relational databases, notes that multi-table data is usually flattened into a single table through manual feature engineering and that this loses predictive signal. RelBench itself packages each dataset as tables, keys, a foreign-key graph and time splits, and hides rows after the test timestamp to prevent leakage.
Documentation standards ask the question directly. Datasheets for Datasets, the widely used documentation template, asks whether relationships between individual instances are made explicit and, if so, how. The MLCommons Croissant metadata format gives a field-level answer with its references property, which it describes as the equivalent of a foreign key reference in a relational database. A linked package that cannot answer these questions in its documentation is priced like an unlinked one.
What should the matched dataset packaging comparison include?
It should compare two packages built from the same source records, the same period, the same rights and the same de-identification, so that the only difference is packaging. The table is the template. Fill in the middle columns from your own counts; the last column says how to measure each line.
| Dimension | Package A: wide, unlinked rows | Package B: complete linked cases | How to measure |
|---|---|---|---|
| Unit of value | One record, such as a line item or message | One case from request to outcome | Name the unit the buyer's task consumes |
| Unique count | Distinct records after deduplication | Distinct cases with every required link | Count after dedup, not raw exports |
| Coverage per unit | Fields in one system | Systems and record types joined per case | Share of cases with each link present |
| Outcome coverage | Outcome field if the system has one | Recorded result per case | Share of units with a verifiable outcome |
| Time span and consistency | Years covered, schema changes | Years covered, schema changes in every system | Gaps and breaks by month |
| Tasks supported | Per-record tasks | Sequence, outcome and agent tasks | List the buyer tasks each package can serve |
| Privacy linkage risk | Lower; fewer attributes per person | Higher; more quasi-identifiers per case | Re-identification review on a sample |
| Preparation effort | One extract, one redaction pass | Joins, crosswalks, consistent pseudonymous keys | Hours estimated per step |
| Permitted uses and exclusivity scope | Rights in one system's contracts | Rights in every linked system's contracts | Rights review per source system |
| Delivery timing | Shorter preparation path | Longer preparation and QA path | Calendar estimate from the plan |
How do you run the comparison so the result is fair?
Hold everything constant except packaging, then test both packages against the same buyer task. If Package B comes from a cleaner period or a better-documented system, you are comparing data quality, not linkage.
- Fix the source: the same systems, the same date range and the same record selection rules for both packages.
- Build both from one extract, so deletions, corrections and de-identification apply identically.
- Record the linkage rate: the share of eligible cases that join end to end, and the reasons the rest do not.
- Check whether linked cases differ from unlinked ones, for example only simple jobs having invoices, and note any bias.
- Write down the buyer task in one sentence, then pick a sample from each package that a buyer could pilot on that task.
- Document both packages with the same datasheet questions, including how relationships are made explicit.
- List every assumption next to the result, and date it, because buyer requirements change.
Which assumption decides whether the linked package wins?
The buyer's task definition has the largest effect, followed by the linkage rate and whether every linked system's records can be licensed. If the task needs outcomes, an unlinked package fits poorly at any size; if a single linked system's contracts forbid the use, the whole linked case may drop out.
Preparation effort matters for net proceeds as well, because linked packages take more joining, crosswalk and de-identification work before delivery. That cost is real, but it is usually smaller than the gap in buyer fit when the task needs linked cases. Treat any figure as a conditional range built from these assumptions, not a promise.
| Assumption | If it holds | If it fails | How to test it |
|---|---|---|---|
| The buyer task needs outcomes | Linked cases fit; unlinked rows are context at most | Unlinked rows may fit as well or better | Ask for the task in one sentence and a sample schema |
| Most eligible cases link end to end | Package B keeps most of the volume | Package B shrinks and may skew to easy cases | Measure linkage rate and compare linked and unlinked cases |
| All linked systems are licensable | Linked case can be offered whole | Restricted system must be dropped or the case narrowed | Rights review of each system's contracts and notices |
| De-identification holds after linking | Linked case can be delivered | More generalization or removal is needed | Re-identification review on a linked sample |
| History is consistent over time | Cases compare across years | Buyer must handle schema breaks | Profile fields by month for breaks |
How does linking records change privacy and preparation?
Linking records raises privacy risk because each case gathers more attributes about the same people, so de-identification has to be planned for the linked case, not system by system. NIST's report on de-identification of personal information, NISTIR 8053, notes that direct identifiers can be systematically replaced with pseudonyms so that records referencing the same individual can still be matched. That is exactly what makes linked cases useful, and exactly why they need closer review.
NIST SP 800-188 frames re-identification risk as a function of the adverse impact if re-identification occurred and its likelihood. A complete case that joins a customer's request, a technician's notes and a billing record usually carries more quasi-identifiers than any single row, so expect more generalization, more removal and a re-identification review on a sample.
Keep four permissions separate: having access to the systems, having privacy permission to use the personal data for this purpose, owning the records, and holding contractual rights to license them. De-identification alone does not create a right to license. Linked packages touch more systems, so they touch more contracts, customer agreements and notices that counsel should review.
Worked example: a field service history packaged two ways (illustrative)
This example is illustrative, with invented counts, and does not describe a real company or any price. A field service company has about a million work order line items across several years, and roughly ten thousand jobs that link from the customer request through dispatch, technician notes and the invoice to whether a callback followed.
Package A, the line items, suits a buyer training a model to normalize parts descriptions or classify job types from text. Package B, the linked jobs, suits a buyer evaluating a dispatch agent, because each job shows the request, the decision and the outcome that can be checked. Built from the same extract and de-identified the same way, the two packages answer different buyer tasks, and neither is simply bigger or better.
The exception is a biased linkage. Suppose only straightforward jobs reached an invoice in the billing system, while complex jobs were billed through a separate project tool. Package B would then overrepresent easy work and could mislead an evaluation. The right move is to repair the link to the project tool, document the gap, or offer the linked cases as a narrower, clearly described scope.
Can you offer both packages?
Often yes: a linked core of complete cases plus an unlinked tier of standalone records, each described with its own unit, counts, coverage and permitted uses. Tiering lets a buyer pilot the linked core for agent or outcome tasks while using the wider tier for language or classification work.
Tiers need separate documentation and separate rights checks, and the exclusivity and term for each package are set in that deal's signed agreement rather than assumed. Do not describe the unlinked tier as linked because a few fields overlap; a buyer will test the joins, and a package that overstates linkage loses trust for the whole offer.
Describe each tier in the same terms a buyer will use to test it: the unit, the unique count after deduplication, the share of units with each link, the share with a recorded outcome, the date range and the systems involved. A short data profile that states these numbers, without sharing files, lets a buyer judge fit before any sample is prepared, and it keeps the conversation about the records rather than about raw size. If a buyer asks for a pilot, offer the same small sample from each tier so the comparison stays matched.
When should a linked package wait for counsel or drop a system?
Stop and get review when linking requires joining systems governed by different contracts or notices, when linked cases include health, financial or other sensitive records, or when the linkage rate is low and the linked share is clearly unrepresentative. In each case the risk of a wrong decision is higher than the cost of a short delay.
Narrow the scope when one system's rights are unclear: offer the cases without that system and say so. Bring in counsel for contracts and privacy, and a finance reviewer for any valuation range, so assumptions about rights, timing and preparation cost are written down before the comparison is shared.
Bringing a matched packaging comparison to a licensing conversation
For a company deciding how to package its own history, SourceX covers the licensing path for selected business records, from qualification and rights review through contracting, preparation, delivery and payment. It starts with a brief assessment of the company itself, so neither package has to be built or uploaded to learn whether the company qualifies.
Whichever tier goes forward, the company approves its scope and terms and signs before any records move. The delivery date, exclusivity and term are written into each deal's signed Dataset Deal Confirmation, and SourceX performs the de-identification through approved processing routes. The matched comparison above gives that conversation concrete units, counts and link rates to work from.
Frequently asked questions
How much is 100 GB of data worth?
Size alone does not set value. A buyer values the units it can use for its task, so 100 GB of unlinked logs and a much smaller set of complete, outcome-linked cases can be worth very different amounts to different buyers. Value depends on the task, coverage, outcomes, rights and preparation, and is assessed deal by deal.
Are linked records always worth more than unlinked records?
No. Linked records fit tasks that need the path from request to outcome, such as agent evaluation. For per-record tasks like classification or extraction, a large, varied set of standalone records can fit better.
What is a complete case?
A complete case is one unit of work with every record the buyer's task needs joined together, for example a request, the decision, the work done and a recorded outcome. Define the required links before counting, and report the share of eligible cases that meet them.
Does linking records make de-identification harder?
Yes. A linked case gathers more attributes about the same people, so it usually needs more generalization or removal and a re-identification review on a sample. Plan de-identification for the whole case with consistent pseudonymous keys, not system by system.
Can a company license linked records if one system's contract is restrictive?
Sometimes, by dropping that system or narrowing the cases. The rights of every linked system apply to the case, so counsel should review each system's contracts and notices before a linked package is offered.
Sources
- The RelBench paper notes that multi-table data is usually flattened into a single table through manual feature engineering, which loses predictive signal. Source
- RelBench packages each dataset with tables, keys, a foreign-key graph and time splits, and hides rows after the test timestamp. Source
- The Croissant metadata format's references property links a field to another record set's field, the equivalent of a foreign key in a relational database. Source
- Datasheets for Datasets asks whether relationships between individual instances are made explicit and, if so, how. Source
- NISTIR 8053 explains that direct identifiers can be systematically replaced with pseudonyms, allowing records referencing the same individual to be matched. Source
- NIST SP 800-188 describes re-identification risk as typically a function of the adverse impacts if re-identification occurred and the likelihood of occurrence. Source
See if your company qualifies
A short company assessment. No data uploads are needed.