Deal economics
Data refresh subscription vendor comparison: price per usable record
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
A data refresh subscription vendor comparison should rank vendors by what each actually delivers: new or changed records that pass validation and fall within your permitted use, with your own checking cost per refresh added to the price. Normalize frequency, scope, validation and permitted use first, then compare. In this article's illustrative three-vendor example, the cheapest plan costs the most per usable record.
Value drivers
Data refresh subscription
Key takeaways
- Price per record delivered rewards full snapshots: in the illustrative example, the quarterly snapshot vendor looks more than twelve times cheaper per delivered record than one rival and ends up the most expensive per usable record.
- Four normalizers decide the comparison: frequency, scope, validation and permitted use, and each needs a contract term, not a product page.
- Frequency is not free: weekly refreshes multiply the buyer's own checking cost, which in the example hands first place from Vendor Z to Vendor X.
- Marketplace terms show what to check: on AWS Data Exchange, revision contents are up to the provider, revisions can be revoked at any time, and access to view or export ends when the subscription expires.
- Catalog size and security badges are not refresh evidence; run two pilot refreshes through your own acceptance test before signing.
How should a buyer compare data refresh subscriptions across vendors?
Compare the price of usable refreshed records, not the price of the plan. For each vendor, count the new or changed records a year that pass your validation test and fall within your permitted use, add your own checking cost per refresh to the plan price, and divide. Then score the terms a price cannot capture, such as revocation and what you keep after the term ends.
Refresh terms differ more than headline prices do. One vendor ships a full snapshot every quarter, another a monthly delta, a third a weekly feed; one validates every batch, another leaves validation to you; one licenses training and evaluation, another training only. Until those differences are normalized, a lower subscription price says nothing about cost.
This guide sets out the four normalizers, works a usable-refresh price through three illustrative vendors, shows when paying for frequency makes sense, lists the terms that sit outside the formula, and ends with a scorecard and a decision memo a procurement lead can file with the purchase.
Why the plan price is the wrong first number
A plan price buys access to whatever the vendor publishes, on the vendor's schedule, under the vendor's terms. Unless the contract fixes all three, the price has no stable unit to compare.
Public marketplace documentation makes the gap concrete. AWS Data Exchange describes all of its products as subscription-based, with offer terms covering price, duration, payment schedule, a data subscription agreement and a refund policy. Its user guide adds that a revision may contain all of a data set's data or only what is new since the previous revision, and that what each revision contains is up to the provider. Two subscriptions with the same price and the same stated cadence can therefore deliver very different amounts of new information.
Even a machine-readable data contract can price the wrong unit. The Open Data Contract Standard lets a contract state a subscription price per unit of measure, giving megabyte and gigabyte as examples, next to a frequency-of-update property and a latency property in its service-level section. Bytes and cadence are easy to write down; usable records are what a training or evaluation team is actually buying.
Catalog size, security badges and selected case studies do not close the gap. None of them tells you how many records in the next refresh will be new, valid and licensed for your use.
Four normalizers: frequency, scope, validation and permitted use
Normalize every vendor to the same four terms before comparing any price. The table gives the question each term answers, the unit to normalize to, the evidence that settles it and the trap that makes different offers look alike.
Labels are not evidence for the last row. Across the more than 1,800 text datasets in the Data Provenance Initiative's audit, widely used hosting sites omitted the license 70 percent of the time or more and got it wrong 50 percent of the time or more, so read the grant in the agreement for every refresh rather than the license field on a listing.
| Normalizer | Question it answers | Normalize to | Evidence that settles it | Trap |
|---|---|---|---|---|
| Frequency | How often does a refresh arrive, and is the cadence an obligation? | Refreshes a year the contract commits to | Schedule clause and a log of past refresh dates | A stated update frequency the agreement does not require |
| Scope | What does each refresh contain? | New or changed records a year, with duplicates and unchanged rows removed | Per-refresh manifest counting new, changed and deleted records | A full snapshot counted as fresh volume |
| Validation | Who checks each batch, against what, and what happens on failure? | Share of delivered records that pass your acceptance test | Batch acceptance rule, replacement path and two pilot refreshes | Checks run only on the vendor's own criteria |
| Permitted use | May you use each batch for training, evaluation and retrieval, and after the term? | Share of records usable for your intended uses | License grant per batch, survival terms and revocation terms | Training rights in the demo, access-only rights in the contract |
The usable-refresh price, worked through three vendors
Usable-refresh price is the annual plan price plus your own checking cost, divided by the new or changed records a year that pass validation and fall within permitted use. In the illustrative example below, the cheapest vendor changes at almost every step.
At a glance
The usable-refresh price, worked through three vendors
The three vendors are illustrative, and prices are index points a year, not real prices. Vendor Y's quarterly snapshot delivers 800,000 records a year, which makes it look more than twelve times cheaper per delivered record than Vendor Z. But only 10 percent of each snapshot is new or changed, a quarter of those fail the buyer's test, and its license excludes evaluation, which accounts for 30 percent of the buyer's planned use. Per usable record, Y is the most expensive of the three.
Frequency changes the answer once your own work is counted. Vendor Z's weekly delta is the cheapest per usable record on plan price alone, at 1.39 points per 1,000 records, but 52 refreshes at 0.4 points of checking each add 20.8 points a year, and Vendor X's monthly delta, which ships validation evidence with every batch, finishes first at 1.55 against Z's 1.61. If your task needs weekly recency, Z's premium may still be worth paying, which the next section tests.
| Step (Illustrative) | Vendor X | Vendor Y | Vendor Z |
|---|---|---|---|
| Annual plan price, index points | 100 | 80 | 130 |
| Refresh unit and cadence | Monthly delta, 6,000 records | Quarterly full snapshot, 200,000 records | Weekly delta, 2,000 records |
| Records delivered a year | 72,000 | 800,000 | 104,000 |
| New or changed records a year | 72,000 | 80,000 (10% of each snapshot) | 104,000 |
| Usable: pass validation and within permitted use | 68,400 (95% and 100%) | 42,000 (75% and 70%) | 93,600 (90% and 100%) |
| Buyer checking a year, index points | 6 (12 x 0.5) | 12 (4 x 3) | 20.8 (52 x 0.4) |
| Plan price per 1,000 records delivered | 1.39 | 0.10 | 1.25 |
| Plan price per 1,000 usable records | 1.46 | 1.90 | 1.39 |
| All-in price per 1,000 usable records | 1.55 | 2.19 | 1.61 |
When paying more for refresh frequency is the right call
Pay for frequency when your task's data goes stale faster than the cheaper cadence refreshes it, or when an evaluation set has to stay unseen by the models it tests. Otherwise a slower, validated refresh usually costs less per usable record.
Staleness is common and measurable. Vela and colleagues, writing in Scientific Reports in 2022, ran four standard model types on 32 datasets from healthcare operations, transportation, finance and weather and found temporal quality degradation in 91 percent of the 128 model and dataset pairs. The pace differs by task, so time your own model's decline on recent held-out records before paying for weekly delivery.
Evaluation buyers have a second reason. GSM1k shows the risk: a set of new grade school math problems, built to match the public GSM8k benchmark in difficulty, lowered leading models' accuracy by as much as 8 percent, and several model families appeared to have overfit the public set (arXiv 2405.00332). For evaluation data, the value of a refresh comes from novelty the tested models have not seen, not from volume.
A workable rule: estimate how many usable records each cadence delivers inside the window in which your task still benefits from them, and compare prices on that count. Records that arrive after the window has closed are volume, not value.
Terms the formula cannot price: revocation, expiry and change notice
Some terms change what you keep rather than what you pay, so score them beside the price instead of folding them into it. Three matter most in refresh subscriptions.
Revocation. AWS Data Exchange lets a provider revoke subscriber access to a specific revision at any time, an action it says is typically taken for compliance reasons, after which subscribers cannot export or query that revision's data. Ask any vendor what happens to records already in your training sets, evaluation sets or retrieval indexes if a batch is withdrawn, and require notice that lists record identifiers.
Expiry. On the same marketplace, once a subscription expires the subscriber can no longer view or export the data sets, and an offer can grant anywhere from no historical revisions to all of them. A license that vests each batch once it is delivered and paid behaves very differently at the end of the term from one that ends every right at once.
Change notice. Datasheets for Datasets asks dataset creators whether a dataset will be updated, how often, by whom and how updates will reach users, and it recommends issuing an updated datasheet with each updated version while noting that its workflow can strain with dynamic datasets. A vendor that cannot answer those questions for each refresh cannot tell you what changed between the batch you tested and the batch you trained on.
Can catalog size or a security badge stand in for refresh evidence?
No. Catalog size measures what a vendor lists, a badge measures how it secures its systems, and a case study shows its best customer; none measures how many records in your next refresh will be new, valid and licensed for your use. This is the strongest alternative to the method above, and it fails on its own terms.
The test is to ask which decision each artifact would change. A larger catalog matters only if your task needs categories the smaller vendor lacks. A security attestation belongs in the security review, where it is necessary but not sufficient. A case study counts only if it reports acceptance results on a task like yours, with its denominator. If none of those holds, the artifact is marketing, and the comparison should rest on two pilot refreshes run through your own acceptance test.
Pilots expose the normalizers directly. Two consecutive refreshes show the real cadence, the share of new records, the pass rate on your test and whether the license text matches the sales description, which is enough to fill every row of the usable-refresh table with observed values instead of claimed ones.
A refresh vendor scorecard that keeps unknowns visible
Score each vendor on observed evidence, with pass, partial, fail or unknown in every cell, so that no single total hides a failed term. The rows are the four normalizers plus the terms outside the formula, and the columns are the illustrative vendors from the worked example.
Read across each row before reading the price. Vendor Z's missing revocation notice is a fail that a lower price cannot offset if your models are slow to retrain; Vendor X's partial on post-term rights matters only if you plan to retrain on old batches after the term ends. The decision follows your intended use, not the lowest number.
| Criterion (Illustrative) | Vendor X | Vendor Y | Vendor Z |
|---|---|---|---|
| Cadence committed in the contract | Pass: monthly, in the schedule | Partial: quarterly stated, not committed | Pass: weekly, in the schedule |
| Manifest of new, changed and deleted records | Pass | Fail: full snapshot only | Pass |
| Batch validation and replacement | Pass: evidence and a replacement path | Fail: buyer tests alone | Partial: automated checks, no replacement path |
| Permitted use covers training and evaluation | Pass | Fail: training only | Pass, plus retrieval |
| Rights in each batch after the term | Partial: models survive, raw records deleted | Unknown: agreement silent | Pass: each batch vests on payment |
| Revocation notice with record identifiers | Pass | Unknown | Fail: no notice period |
| All-in price per 1,000 usable records | 1.55 | 2.19 | 1.61 |
What goes in the refresh purchase decision memo
The memo records the decision, the evidence behind it and what would change it, on one page filed with the pilot results. It lets a later reviewer, or the team handling the renewal, see why a vendor won without rerunning the comparison.
Fill these fields, each with a date and a pointer to the document behind it:
- Decision: buy, buy a narrower scope, extend the pilot or decline, with the named owner.
- Intended uses: training, evaluation, retrieval, and whether each must survive the term.
- Normalized terms per vendor: committed cadence, new records a year, validation pass rate and permitted-use share.
- Usable-refresh price per vendor, plan-only and all-in, with the checking-cost assumption stated.
- Scorecard results, with every unknown listed beside the question sent to the vendor.
- Pilot evidence: the two refreshes tested, their manifests and your acceptance results.
- Limits: what the pilot could not show, such as behavior through a schema change.
- Reopen triggers: a missed cadence, a revoked batch, a failed refresh or a change in how fast your task drifts.
Writing refresh terms into a request to SourceX
Put the refresh terms in the request rather than leaving them for the negotiation. SourceX sources datasets for AI buyers on request from U.S. businesses instead of holding stock, and its request form asks for scope, volume, language, format, timing and licensing needs; a recurring need, its cadence, the new-record unit and the acceptance test belong in that same request.
Whether a supplying business can support recurring refreshes is assessed deal by deal. Three terms frame every deal: the supplying company signs, with scope and terms approved, before anything is delivered; SourceX or its processing partner carries out de-identification; and each deal's signed agreement states exclusivity and term, so cadence and post-term rights are settled in writing rather than assumed. A buyer can run the scorecard above against that agreement as it would against any vendor's.
Frequently asked questions
What is a data refresh subscription?
It is a license that delivers new versions or batches of a dataset on a schedule for a recurring price, rather than one delivery for a one-time fee. Terms vary on how often refreshes arrive, what each contains, who validates it and what you may do with each batch during and after the term.
How do you compare data subscription prices fairly?
Convert each vendor to a price per usable refreshed record: new or changed records a year that pass your validation test and fall within your permitted use, with your own checking cost added to the plan price. Then score revocation, expiry and change-notice terms separately, because a price cannot capture them.
Is a full snapshot refresh better than a delta?
Not by itself. A snapshot repeats records you already hold, so measure the share that is new or changed; a delta needs a reliable record of deletions and corrections. Choose the unit that lets you count new records and carry withdrawals through to your own systems.
What happens to refreshed data when a subscription ends?
It depends on the agreement. On AWS Data Exchange, for example, subscribers can no longer view or export data sets once a subscription expires, while some licenses vest each batch permanently once it is delivered and paid. Check whether trained models, evaluation sets and retrieval indexes are treated differently.
How many pilot refreshes should a buyer test before signing?
Test at least two consecutive refreshes, so you can compare them and see the real cadence, the share of new records and whether validation holds from one batch to the next. One pilot batch shows the data; two show the refresh.
Sources
- AWS Data Exchange products are subscription-based, with offer terms covering price, duration, payment schedule, a data subscription agreement and a refund policy; a revision may contain all of a data set's data or only what is new, what each revision contains is up to the provider, offers grant zero to all historical revisions, and once a subscription expires subscribers can no longer view or export the data sets. Source
- AWS Data Exchange lets a provider revoke subscriber access to a specific revision at any time, typically for compliance reasons; subscribers then cannot export or query the data within the revoked revision. Source
- The Open Data Contract Standard's service-level section lists frequency (frequency of update) and latency, described as preferred to freshness, among its recommended SLA properties. Source
- The Open Data Contract Standard's pricing block states a subscription price per unit of measure, with megabyte and gigabyte as example units. Source
- The Data Provenance Initiative's audit of more than 1,800 text datasets found license omission of 70 percent or more and error rates of 50 percent or more on widely used dataset hosting sites. Source
- Vela and colleagues (Scientific Reports, 2022) tested four standard model types on 32 datasets from healthcare operations, transportation, finance and weather and found temporal quality degradation in 91 percent of the 128 model and dataset pairs. Source
- The GSM1k paper (arXiv 2405.00332, current version) reports accuracy drops of up to 8 percent for leading models on new problems built to match GSM8k, with several model families showing systematic overfitting. Source
- Datasheets for Datasets asks whether a dataset will be updated, how often, by whom and how updates will be communicated, and recommends updated datasheets with updated versions while noting its workflow may pose problems for dynamic datasets. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.