Skip to content

Deal economics

Data refresh subscription vendor comparison: price per usable record

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

A data refresh subscription vendor comparison should rank vendors by what each actually delivers: new or changed records that pass validation and fall within your permitted use, with your own checking cost per refresh added to the price. Normalize frequency, scope, validation and permitted use first, then compare. In this article's illustrative three-vendor example, the cheapest plan costs the most per usable record.

Value drivers

Data refresh subscription

This page covers data refresh subscription vendor comparison: price per usable record.Illustrative

Key takeaways

  • Price per record delivered rewards full snapshots: in the illustrative example, the quarterly snapshot vendor looks more than twelve times cheaper per delivered record than one rival and ends up the most expensive per usable record.
  • Four normalizers decide the comparison: frequency, scope, validation and permitted use, and each needs a contract term, not a product page.
  • Frequency is not free: weekly refreshes multiply the buyer's own checking cost, which in the example hands first place from Vendor Z to Vendor X.
  • Marketplace terms show what to check: on AWS Data Exchange, revision contents are up to the provider, revisions can be revoked at any time, and access to view or export ends when the subscription expires.
  • Catalog size and security badges are not refresh evidence; run two pilot refreshes through your own acceptance test before signing.

How should a buyer compare data refresh subscriptions across vendors?

Compare the price of usable refreshed records, not the price of the plan. For each vendor, count the new or changed records a year that pass your validation test and fall within your permitted use, add your own checking cost per refresh to the plan price, and divide. Then score the terms a price cannot capture, such as revocation and what you keep after the term ends.

Refresh terms differ more than headline prices do. One vendor ships a full snapshot every quarter, another a monthly delta, a third a weekly feed; one validates every batch, another leaves validation to you; one licenses training and evaluation, another training only. Until those differences are normalized, a lower subscription price says nothing about cost.

This guide sets out the four normalizers, works a usable-refresh price through three illustrative vendors, shows when paying for frequency makes sense, lists the terms that sit outside the formula, and ends with a scorecard and a decision memo a procurement lead can file with the purchase.

Why the plan price is the wrong first number

A plan price buys access to whatever the vendor publishes, on the vendor's schedule, under the vendor's terms. Unless the contract fixes all three, the price has no stable unit to compare.

Public marketplace documentation makes the gap concrete. AWS Data Exchange describes all of its products as subscription-based, with offer terms covering price, duration, payment schedule, a data subscription agreement and a refund policy. Its user guide adds that a revision may contain all of a data set's data or only what is new since the previous revision, and that what each revision contains is up to the provider. Two subscriptions with the same price and the same stated cadence can therefore deliver very different amounts of new information.

Even a machine-readable data contract can price the wrong unit. The Open Data Contract Standard lets a contract state a subscription price per unit of measure, giving megabyte and gigabyte as examples, next to a frequency-of-update property and a latency property in its service-level section. Bytes and cadence are easy to write down; usable records are what a training or evaluation team is actually buying.

Catalog size, security badges and selected case studies do not close the gap. None of them tells you how many records in the next refresh will be new, valid and licensed for your use.

Four normalizers: frequency, scope, validation and permitted use

Normalize every vendor to the same four terms before comparing any price. The table gives the question each term answers, the unit to normalize to, the evidence that settles it and the trap that makes different offers look alike.

Labels are not evidence for the last row. Across the more than 1,800 text datasets in the Data Provenance Initiative's audit, widely used hosting sites omitted the license 70 percent of the time or more and got it wrong 50 percent of the time or more, so read the grant in the agreement for every refresh rather than the license field on a listing.

Four normalizers: frequency, scope, validation and permitted use
NormalizerQuestion it answersNormalize toEvidence that settles itTrap
FrequencyHow often does a refresh arrive, and is the cadence an obligation?Refreshes a year the contract commits toSchedule clause and a log of past refresh datesA stated update frequency the agreement does not require
ScopeWhat does each refresh contain?New or changed records a year, with duplicates and unchanged rows removedPer-refresh manifest counting new, changed and deleted recordsA full snapshot counted as fresh volume
ValidationWho checks each batch, against what, and what happens on failure?Share of delivered records that pass your acceptance testBatch acceptance rule, replacement path and two pilot refreshesChecks run only on the vendor's own criteria
Permitted useMay you use each batch for training, evaluation and retrieval, and after the term?Share of records usable for your intended usesLicense grant per batch, survival terms and revocation termsTraining rights in the demo, access-only rights in the contract

The usable-refresh price, worked through three vendors

Usable-refresh price is the annual plan price plus your own checking cost, divided by the new or changed records a year that pass validation and fall within permitted use. In the illustrative example below, the cheapest vendor changes at almost every step.

At a glance

The usable-refresh price, worked through three vendors

The three vendors are illustrative, and prices are index points a year, not real prices. Vendor Y's quarterly snapshot delivers 800,000 records a year, which makes it look more than twelve times cheaper per delivered record than Vendor Z. But only 10 percent of each snapshot is new or changed, a quarter of those fail the buyer's test, and its license excludes evaluation, which accounts for 30 percent of the buyer's planned use. Per usable record, Y is the most expensive of the three.

Frequency changes the answer once your own work is counted. Vendor Z's weekly delta is the cheapest per usable record on plan price alone, at 1.39 points per 1,000 records, but 52 refreshes at 0.4 points of checking each add 20.8 points a year, and Vendor X's monthly delta, which ships validation evidence with every batch, finishes first at 1.55 against Z's 1.61. If your task needs weekly recency, Z's premium may still be worth paying, which the next section tests.

The usable-refresh price, worked through three vendors
Step (Illustrative)Vendor XVendor YVendor Z
Annual plan price, index points10080130
Refresh unit and cadenceMonthly delta, 6,000 recordsQuarterly full snapshot, 200,000 recordsWeekly delta, 2,000 records
Records delivered a year72,000800,000104,000
New or changed records a year72,00080,000 (10% of each snapshot)104,000
Usable: pass validation and within permitted use68,400 (95% and 100%)42,000 (75% and 70%)93,600 (90% and 100%)
Buyer checking a year, index points6 (12 x 0.5)12 (4 x 3)20.8 (52 x 0.4)
Plan price per 1,000 records delivered1.390.101.25
Plan price per 1,000 usable records1.461.901.39
All-in price per 1,000 usable records1.552.191.61
A slope chart ranking three illustrative vendors at four steps: plan price, price per delivered record, price per usable record and all-in price per usable record, with the cheapest plan, Vendor Y, falling to last.
How the cheapest vendor changes as each normalizer is applied. Illustrative vendors and index prices.

When paying more for refresh frequency is the right call

Pay for frequency when your task's data goes stale faster than the cheaper cadence refreshes it, or when an evaluation set has to stay unseen by the models it tests. Otherwise a slower, validated refresh usually costs less per usable record.

Staleness is common and measurable. Vela and colleagues, writing in Scientific Reports in 2022, ran four standard model types on 32 datasets from healthcare operations, transportation, finance and weather and found temporal quality degradation in 91 percent of the 128 model and dataset pairs. The pace differs by task, so time your own model's decline on recent held-out records before paying for weekly delivery.

Evaluation buyers have a second reason. GSM1k shows the risk: a set of new grade school math problems, built to match the public GSM8k benchmark in difficulty, lowered leading models' accuracy by as much as 8 percent, and several model families appeared to have overfit the public set (arXiv 2405.00332). For evaluation data, the value of a refresh comes from novelty the tested models have not seen, not from volume.

A workable rule: estimate how many usable records each cadence delivers inside the window in which your task still benefits from them, and compare prices on that count. Records that arrive after the window has closed are volume, not value.

Terms the formula cannot price: revocation, expiry and change notice

Some terms change what you keep rather than what you pay, so score them beside the price instead of folding them into it. Three matter most in refresh subscriptions.

Revocation. AWS Data Exchange lets a provider revoke subscriber access to a specific revision at any time, an action it says is typically taken for compliance reasons, after which subscribers cannot export or query that revision's data. Ask any vendor what happens to records already in your training sets, evaluation sets or retrieval indexes if a batch is withdrawn, and require notice that lists record identifiers.

Expiry. On the same marketplace, once a subscription expires the subscriber can no longer view or export the data sets, and an offer can grant anywhere from no historical revisions to all of them. A license that vests each batch once it is delivered and paid behaves very differently at the end of the term from one that ends every right at once.

Change notice. Datasheets for Datasets asks dataset creators whether a dataset will be updated, how often, by whom and how updates will reach users, and it recommends issuing an updated datasheet with each updated version while noting that its workflow can strain with dynamic datasets. A vendor that cannot answer those questions for each refresh cannot tell you what changed between the batch you tested and the batch you trained on.

Can catalog size or a security badge stand in for refresh evidence?

No. Catalog size measures what a vendor lists, a badge measures how it secures its systems, and a case study shows its best customer; none measures how many records in your next refresh will be new, valid and licensed for your use. This is the strongest alternative to the method above, and it fails on its own terms.

The test is to ask which decision each artifact would change. A larger catalog matters only if your task needs categories the smaller vendor lacks. A security attestation belongs in the security review, where it is necessary but not sufficient. A case study counts only if it reports acceptance results on a task like yours, with its denominator. If none of those holds, the artifact is marketing, and the comparison should rest on two pilot refreshes run through your own acceptance test.

Pilots expose the normalizers directly. Two consecutive refreshes show the real cadence, the share of new records, the pass rate on your test and whether the license text matches the sales description, which is enough to fill every row of the usable-refresh table with observed values instead of claimed ones.

A refresh vendor scorecard that keeps unknowns visible

Score each vendor on observed evidence, with pass, partial, fail or unknown in every cell, so that no single total hides a failed term. The rows are the four normalizers plus the terms outside the formula, and the columns are the illustrative vendors from the worked example.

Read across each row before reading the price. Vendor Z's missing revocation notice is a fail that a lower price cannot offset if your models are slow to retrain; Vendor X's partial on post-term rights matters only if you plan to retrain on old batches after the term ends. The decision follows your intended use, not the lowest number.

A refresh vendor scorecard that keeps unknowns visible
Criterion (Illustrative)Vendor XVendor YVendor Z
Cadence committed in the contractPass: monthly, in the schedulePartial: quarterly stated, not committedPass: weekly, in the schedule
Manifest of new, changed and deleted recordsPassFail: full snapshot onlyPass
Batch validation and replacementPass: evidence and a replacement pathFail: buyer tests alonePartial: automated checks, no replacement path
Permitted use covers training and evaluationPassFail: training onlyPass, plus retrieval
Rights in each batch after the termPartial: models survive, raw records deletedUnknown: agreement silentPass: each batch vests on payment
Revocation notice with record identifiersPassUnknownFail: no notice period
All-in price per 1,000 usable records1.552.191.61

What goes in the refresh purchase decision memo

The memo records the decision, the evidence behind it and what would change it, on one page filed with the pilot results. It lets a later reviewer, or the team handling the renewal, see why a vendor won without rerunning the comparison.

Fill these fields, each with a date and a pointer to the document behind it:

  • Decision: buy, buy a narrower scope, extend the pilot or decline, with the named owner.
  • Intended uses: training, evaluation, retrieval, and whether each must survive the term.
  • Normalized terms per vendor: committed cadence, new records a year, validation pass rate and permitted-use share.
  • Usable-refresh price per vendor, plan-only and all-in, with the checking-cost assumption stated.
  • Scorecard results, with every unknown listed beside the question sent to the vendor.
  • Pilot evidence: the two refreshes tested, their manifests and your acceptance results.
  • Limits: what the pilot could not show, such as behavior through a schema change.
  • Reopen triggers: a missed cadence, a revoked batch, a failed refresh or a change in how fast your task drifts.

Writing refresh terms into a request to SourceX

Put the refresh terms in the request rather than leaving them for the negotiation. SourceX sources datasets for AI buyers on request from U.S. businesses instead of holding stock, and its request form asks for scope, volume, language, format, timing and licensing needs; a recurring need, its cadence, the new-record unit and the acceptance test belong in that same request.

Whether a supplying business can support recurring refreshes is assessed deal by deal. Three terms frame every deal: the supplying company signs, with scope and terms approved, before anything is delivered; SourceX or its processing partner carries out de-identification; and each deal's signed agreement states exclusivity and term, so cadence and post-term rights are settled in writing rather than assumed. A buyer can run the scorecard above against that agreement as it would against any vendor's.

Frequently asked questions

What is a data refresh subscription?

It is a license that delivers new versions or batches of a dataset on a schedule for a recurring price, rather than one delivery for a one-time fee. Terms vary on how often refreshes arrive, what each contains, who validates it and what you may do with each batch during and after the term.

How do you compare data subscription prices fairly?

Convert each vendor to a price per usable refreshed record: new or changed records a year that pass your validation test and fall within your permitted use, with your own checking cost added to the plan price. Then score revocation, expiry and change-notice terms separately, because a price cannot capture them.

Is a full snapshot refresh better than a delta?

Not by itself. A snapshot repeats records you already hold, so measure the share that is new or changed; a delta needs a reliable record of deletions and corrections. Choose the unit that lets you count new records and carry withdrawals through to your own systems.

What happens to refreshed data when a subscription ends?

It depends on the agreement. On AWS Data Exchange, for example, subscribers can no longer view or export data sets once a subscription expires, while some licenses vest each batch permanently once it is delivered and paid. Check whether trained models, evaluation sets and retrieval indexes are treated differently.

How many pilot refreshes should a buyer test before signing?

Test at least two consecutive refreshes, so you can compare them and see the real cadence, the share of new records and whether validation holds from one batch to the next. One pilot batch shows the data; two show the refresh.

Sources

  • AWS Data Exchange products are subscription-based, with offer terms covering price, duration, payment schedule, a data subscription agreement and a refund policy; a revision may contain all of a data set's data or only what is new, what each revision contains is up to the provider, offers grant zero to all historical revisions, and once a subscription expires subscribers can no longer view or export the data sets. Source
  • AWS Data Exchange lets a provider revoke subscriber access to a specific revision at any time, typically for compliance reasons; subscribers then cannot export or query the data within the revoked revision. Source
  • The Open Data Contract Standard's service-level section lists frequency (frequency of update) and latency, described as preferred to freshness, among its recommended SLA properties. Source
  • The Open Data Contract Standard's pricing block states a subscription price per unit of measure, with megabyte and gigabyte as example units. Source
  • The Data Provenance Initiative's audit of more than 1,800 text datasets found license omission of 70 percent or more and error rates of 50 percent or more on widely used dataset hosting sites. Source
  • Vela and colleagues (Scientific Reports, 2022) tested four standard model types on 32 datasets from healthcare operations, transportation, finance and weather and found temporal quality degradation in 91 percent of the 128 model and dataset pairs. Source
  • The GSM1k paper (arXiv 2405.00332, current version) reports accuracy drops of up to 8 percent for leading models on new problems built to match GSM8k, with several model families showing systematic overfitting. Source
  • Datasheets for Datasets asks whether a dataset will be updated, how often, by whom and how updates will be communicated, and recommends updated datasheets with updated versions while noting its workflow may pose problems for dynamic datasets. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies