Skip to content

Deal economics

Dataset pricing for linked versus unlinked records: rows or complete cases?

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

Ten thousand complete business cases are more useful than one million rows when the buyer's task needs the path from request to outcome, such as agent evaluation or outcome prediction. A million unlinked rows win for per-record tasks like classification or extraction. Dataset pricing for linked versus unlinked records follows usable units for the buyer's task, linkage rate, rights in every system and preparation effort, not row count.

Value drivers

Dataset pricing for linked

This page covers dataset pricing for linked versus unlinked records: rows or complete cases.Illustrative

Key takeaways

  • The buyer's task decides whether complete linked cases or a large set of unlinked rows is more useful.
  • Compare packages built from the same source, period, rights and de-identification so only packaging differs.
  • Linkage rate and whether linked cases skew toward easy work can change the answer.
  • Every linked system's contracts apply to the case, and linking raises re-identification risk.
  • A linked core plus an unlinked tier, each documented and rights-checked separately, often serves more buyers.

When are 10,000 complete business cases more useful than one million rows?

Ten thousand complete cases are more useful when the buyer's task needs the path from a request to its outcome: training or testing agents, predicting how a case ends, or grading decisions against what actually happened. One million unlinked rows are more useful when each row stands on its own, such as classifying messages, extracting fields from documents or learning domain language. The deciding factor is the buyer's task, not the row count.

That is also how dataset pricing for linked versus unlinked records works in practice. A buyer values the units it can use for its task. For an agent evaluation team, a row that cannot be joined to the request, the decision and the result is not a usable unit, however many of them there are. For a team training a text classifier, a large, varied, well-labeled set of standalone records may be exactly the unit it wants.

As a general guide, AI buyers look for years of consistent history, typically two or more, and tens of thousands of records or more in one system, while smaller sets can qualify when the work is rare; every dataset is assessed deal by deal. This page gives you a method to compare the two packagings of your own records on equal terms before anyone talks about value.

Why does linkage change what a dataset is worth to a buyer?

Linkage changes worth because it preserves the relationships a model needs to learn cause and outcome, and flattening those relationships away cannot be undone by the buyer. A support ticket alone shows a question; the same ticket joined to the bug, the fix, the release and the closing reply shows what happened and why.

Research on relational data makes the same point in technical terms. The RelBench paper, a benchmark for deep learning on relational databases, notes that multi-table data is usually flattened into a single table through manual feature engineering and that this loses predictive signal. RelBench itself packages each dataset as tables, keys, a foreign-key graph and time splits, and hides rows after the test timestamp to prevent leakage.

Documentation standards ask the question directly. Datasheets for Datasets, the widely used documentation template, asks whether relationships between individual instances are made explicit and, if so, how. The MLCommons Croissant metadata format gives a field-level answer with its references property, which it describes as the equivalent of a foreign key reference in a relational database. A linked package that cannot answer these questions in its documentation is priced like an unlinked one.

What should the matched dataset packaging comparison include?

It should compare two packages built from the same source records, the same period, the same rights and the same de-identification, so that the only difference is packaging. The table is the template. Fill in the middle columns from your own counts; the last column says how to measure each line.

What should the matched dataset packaging comparison include?
DimensionPackage A: wide, unlinked rowsPackage B: complete linked casesHow to measure
Unit of valueOne record, such as a line item or messageOne case from request to outcomeName the unit the buyer's task consumes
Unique countDistinct records after deduplicationDistinct cases with every required linkCount after dedup, not raw exports
Coverage per unitFields in one systemSystems and record types joined per caseShare of cases with each link present
Outcome coverageOutcome field if the system has oneRecorded result per caseShare of units with a verifiable outcome
Time span and consistencyYears covered, schema changesYears covered, schema changes in every systemGaps and breaks by month
Tasks supportedPer-record tasksSequence, outcome and agent tasksList the buyer tasks each package can serve
Privacy linkage riskLower; fewer attributes per personHigher; more quasi-identifiers per caseRe-identification review on a sample
Preparation effortOne extract, one redaction passJoins, crosswalks, consistent pseudonymous keysHours estimated per step
Permitted uses and exclusivity scopeRights in one system's contractsRights in every linked system's contractsRights review per source system
Delivery timingShorter preparation pathLonger preparation and QA pathCalendar estimate from the plan

How do you run the comparison so the result is fair?

Hold everything constant except packaging, then test both packages against the same buyer task. If Package B comes from a cleaner period or a better-documented system, you are comparing data quality, not linkage.

  • Fix the source: the same systems, the same date range and the same record selection rules for both packages.
  • Build both from one extract, so deletions, corrections and de-identification apply identically.
  • Record the linkage rate: the share of eligible cases that join end to end, and the reasons the rest do not.
  • Check whether linked cases differ from unlinked ones, for example only simple jobs having invoices, and note any bias.
  • Write down the buyer task in one sentence, then pick a sample from each package that a buyer could pilot on that task.
  • Document both packages with the same datasheet questions, including how relationships are made explicit.
  • List every assumption next to the result, and date it, because buyer requirements change.

Which assumption decides whether the linked package wins?

The buyer's task definition has the largest effect, followed by the linkage rate and whether every linked system's records can be licensed. If the task needs outcomes, an unlinked package fits poorly at any size; if a single linked system's contracts forbid the use, the whole linked case may drop out.

Preparation effort matters for net proceeds as well, because linked packages take more joining, crosswalk and de-identification work before delivery. That cost is real, but it is usually smaller than the gap in buyer fit when the task needs linked cases. Treat any figure as a conditional range built from these assumptions, not a promise.

Which assumption decides whether the linked package wins?
AssumptionIf it holdsIf it failsHow to test it
The buyer task needs outcomesLinked cases fit; unlinked rows are context at mostUnlinked rows may fit as well or betterAsk for the task in one sentence and a sample schema
Most eligible cases link end to endPackage B keeps most of the volumePackage B shrinks and may skew to easy casesMeasure linkage rate and compare linked and unlinked cases
All linked systems are licensableLinked case can be offered wholeRestricted system must be dropped or the case narrowedRights review of each system's contracts and notices
De-identification holds after linkingLinked case can be deliveredMore generalization or removal is neededRe-identification review on a linked sample
History is consistent over timeCases compare across yearsBuyer must handle schema breaksProfile fields by month for breaks

How does linking records change privacy and preparation?

Linking records raises privacy risk because each case gathers more attributes about the same people, so de-identification has to be planned for the linked case, not system by system. NIST's report on de-identification of personal information, NISTIR 8053, notes that direct identifiers can be systematically replaced with pseudonyms so that records referencing the same individual can still be matched. That is exactly what makes linked cases useful, and exactly why they need closer review.

NIST SP 800-188 frames re-identification risk as a function of the adverse impact if re-identification occurred and its likelihood. A complete case that joins a customer's request, a technician's notes and a billing record usually carries more quasi-identifiers than any single row, so expect more generalization, more removal and a re-identification review on a sample.

Keep four permissions separate: having access to the systems, having privacy permission to use the personal data for this purpose, owning the records, and holding contractual rights to license them. De-identification alone does not create a right to license. Linked packages touch more systems, so they touch more contracts, customer agreements and notices that counsel should review.

Worked example: a field service history packaged two ways (illustrative)

This example is illustrative, with invented counts, and does not describe a real company or any price. A field service company has about a million work order line items across several years, and roughly ten thousand jobs that link from the customer request through dispatch, technician notes and the invoice to whether a callback followed.

Package A, the line items, suits a buyer training a model to normalize parts descriptions or classify job types from text. Package B, the linked jobs, suits a buyer evaluating a dispatch agent, because each job shows the request, the decision and the outcome that can be checked. Built from the same extract and de-identified the same way, the two packages answer different buyer tasks, and neither is simply bigger or better.

The exception is a biased linkage. Suppose only straightforward jobs reached an invoice in the billing system, while complex jobs were billed through a separate project tool. Package B would then overrepresent easy work and could mislead an evaluation. The right move is to repair the link to the project tool, document the gap, or offer the linked cases as a narrower, clearly described scope.

Can you offer both packages?

Often yes: a linked core of complete cases plus an unlinked tier of standalone records, each described with its own unit, counts, coverage and permitted uses. Tiering lets a buyer pilot the linked core for agent or outcome tasks while using the wider tier for language or classification work.

Tiers need separate documentation and separate rights checks, and the exclusivity and term for each package are set in that deal's signed agreement rather than assumed. Do not describe the unlinked tier as linked because a few fields overlap; a buyer will test the joins, and a package that overstates linkage loses trust for the whole offer.

Describe each tier in the same terms a buyer will use to test it: the unit, the unique count after deduplication, the share of units with each link, the share with a recorded outcome, the date range and the systems involved. A short data profile that states these numbers, without sharing files, lets a buyer judge fit before any sample is prepared, and it keeps the conversation about the records rather than about raw size. If a buyer asks for a pilot, offer the same small sample from each tier so the comparison stays matched.

When should a linked package wait for counsel or drop a system?

Stop and get review when linking requires joining systems governed by different contracts or notices, when linked cases include health, financial or other sensitive records, or when the linkage rate is low and the linked share is clearly unrepresentative. In each case the risk of a wrong decision is higher than the cost of a short delay.

Narrow the scope when one system's rights are unclear: offer the cases without that system and say so. Bring in counsel for contracts and privacy, and a finance reviewer for any valuation range, so assumptions about rights, timing and preparation cost are written down before the comparison is shared.

Bringing a matched packaging comparison to a licensing conversation

For a company deciding how to package its own history, SourceX covers the licensing path for selected business records, from qualification and rights review through contracting, preparation, delivery and payment. It starts with a brief assessment of the company itself, so neither package has to be built or uploaded to learn whether the company qualifies.

Whichever tier goes forward, the company approves its scope and terms and signs before any records move. The delivery date, exclusivity and term are written into each deal's signed Dataset Deal Confirmation, and SourceX performs the de-identification through approved processing routes. The matched comparison above gives that conversation concrete units, counts and link rates to work from.

Frequently asked questions

How much is 100 GB of data worth?

Size alone does not set value. A buyer values the units it can use for its task, so 100 GB of unlinked logs and a much smaller set of complete, outcome-linked cases can be worth very different amounts to different buyers. Value depends on the task, coverage, outcomes, rights and preparation, and is assessed deal by deal.

Are linked records always worth more than unlinked records?

No. Linked records fit tasks that need the path from request to outcome, such as agent evaluation. For per-record tasks like classification or extraction, a large, varied set of standalone records can fit better.

What is a complete case?

A complete case is one unit of work with every record the buyer's task needs joined together, for example a request, the decision, the work done and a recorded outcome. Define the required links before counting, and report the share of eligible cases that meet them.

Does linking records make de-identification harder?

Yes. A linked case gathers more attributes about the same people, so it usually needs more generalization or removal and a re-identification review on a sample. Plan de-identification for the whole case with consistent pseudonymous keys, not system by system.

Can a company license linked records if one system's contract is restrictive?

Sometimes, by dropping that system or narrowing the cases. The rights of every linked system apply to the case, so counsel should review each system's contracts and notices before a linked package is offered.

Sources

  • The RelBench paper notes that multi-table data is usually flattened into a single table through manual feature engineering, which loses predictive signal. Source
  • RelBench packages each dataset with tables, keys, a foreign-key graph and time splits, and hides rows after the test timestamp. Source
  • The Croissant metadata format's references property links a field to another record set's field, the equivalent of a foreign key in a relational database. Source
  • Datasheets for Datasets asks whether relationships between individual instances are made explicit and, if so, how. Source
  • NISTIR 8053 explains that direct identifiers can be systematically replaced with pseudonyms, allowing records referencing the same individual to be matched. Source
  • NIST SP 800-188 describes re-identification risk as typically a function of the adverse impacts if re-identification occurred and the likelihood of occurrence. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies