AI data market
Operational data valuation evidence study: a buyer interest protocol
By SourceX Editorial · Updated
10 sections
Short answer
To study which dataset features go with buyer interest, take every operational dataset offered in a fixed window as the denominator, code features such as history length, volume, outcome labels and documentation before any buyer sees them, define interest as pre-set observable steps, adjust for industry, timing and sales attention, and test how much hidden confounding would erase each association. What follows is a study design only, with no findings reported.
Value drivers
Operational data valuation
Key takeaways
- The denominator is every dataset offered in the window, not only those that buyers saw or liked.
- Code dataset features at intake, before any buyer contact, or buyer interest can cause the features you measure.
- Define buyer interest as observable steps fixed in advance, not as impressions from a sales team.
- Associations are reported with intervals and a sensitivity test; a causal claim needs a randomized sub-study.
- This is a protocol only: it reports no results and invents no figures.
How could a study find which dataset features go with buyer interest?
It would follow a defined set of offered datasets over a fixed period, record their features before any buyer is involved, record which ones attract defined signs of interest, and compare the two with adjustment for the obvious confounders. That is an observational cohort design. It can show which features travel with interest and how strongly; on its own it cannot show that changing a feature would change interest.
Most claims about what makes business data valuable for AI come from the other direction: someone remembers the deals that happened and describes what those datasets had in common. That skips the datasets that had the same features and attracted nothing. The protocol below is built to count those too.
This page publishes the method only. No study has been run under it here, no results are reported, and none of the examples carry measured values.
Why an association with buyer interest is not a price driver
An association means datasets with a feature were more often followed by interest. A driver means adding the feature to a given dataset would raise interest. The gap between the two is the subject of causal inference, which Hernán and Robins set out in Causal Inference: What If, a text the authors make freely available. Three mechanisms explain most of the gap here.
At a glance
Why an association with buyer interest is not a price driver
Confounding: a third factor causes both. Companies in industries where AI developers are active may also keep longer histories, so history length and interest move together because of industry. Reverse causation: once a buyer shows interest, the supplier often improves documentation, so documentation quality measured later reflects interest instead of predicting it. Selection: if the study only includes datasets that a sales team chose to promote, any feature the team used to choose them will look predictive.
Interest is also not value. A related research line measures value directly from model performance; Data Shapley, for example, scores individual training examples by their contribution to a model. A study of buyer behavior answers a market question, which features draw attention and follow-up, not a modeling question about how much a dataset improves a model.
The bias map below lists the ways a feature could look important without being so, and the part of the design that blocks or measures each one.
- Industry confounding: sectors with active AI demand may also keep longer, richer records. Blocked by adjusting for industry and repeating the comparison within industries.
- Timing confounding: interest rises and falls with the market, and offers that arrive in a busy quarter look better. Blocked by adjusting for offer date and for the buyer requests open at the time.
- Exposure bias: offers matched against more buyer requests have more chances to draw interest. Measured by recording matches and sales hours per offer and adjusting for both.
- Reverse causation: interest prompts suppliers to improve documentation or add history. Blocked by coding every feature at intake and freezing those codes.
- Selection on the outcome: studying only the deals that closed makes every common feature look decisive. Blocked by the full intake denominator.
- Coder knowledge: a coder who knows which offers succeeded may rate their documentation higher. Blocked by coding features before outcomes exist and keeping coders blind to them.
- Multiple testing: with many features, some will look predictive by chance. Limited by naming primary features in advance and labeling the rest exploratory.
The research question, unit and population
State the question in one sentence that names the unit, the features, the outcome and the period. For example: among operational datasets offered for AI licensing through one intermediary between two fixed dates, which features recorded at intake are associated with reaching a defined level of buyer interest within a fixed follow-up window?
The unit is the dataset offer, not the company, because one company can offer several datasets with different features. Record the company as a cluster so the analysis does not treat related offers as independent.
The population is every offer that entered intake in the window, including those later declined on eligibility, rights or quality grounds. Report how many were excluded at each stage and why, so a reader can see who the results speak for. A study drawn from one intermediary's pipeline speaks for that pipeline; it does not describe the whole market for business data, and the write-up should say so in its first paragraph.
Fix the follow-up window per offer, counted from the date the offer cleared intake, and close intake early enough that every offer gets its full window before analysis. Offers withdrawn by the supplier during follow-up are reported separately, with the step they had reached, rather than counted as no interest.
The observational study plan, row by row
This is the asset. Each row fixes one part of the design, then says what the study can conclude from it as a correlation and what further evidence a causal reading would need. Fill the second column before any outcome data is looked at. Feature definitions borrow their structure from Datasheets for Datasets, which proposes documenting a dataset's motivation, composition, collection process and intended uses.
| Plan element | What the plan fixes in advance | Correlational reading allowed | Needed for a causal reading |
|---|---|---|---|
| Question | One sentence naming unit, features, outcome and period | Which features co-occur with interest in this pipeline | A manipulable feature and a comparison that varies only it |
| Unit and clusters | Dataset offer as unit; company as cluster | Associations per offer with cluster-aware intervals | Within-company comparisons of offers that differ in one feature |
| Denominator | Every offer entering intake in the window, with exclusions counted by stage | Rates of interest among all offers with and without a feature | No selection on the outcome or on sales choice |
| Features | History length, record volume, outcome labels present, systems linked, documentation completeness, rights clarity, language share, each coded at intake | Feature levels that travel with interest | Features measured before any buyer contact and not changed by it |
| Outcome | Interest ladder: request for details, request for a sample, request for terms; follow-up window fixed | Share of offers reaching each step, by feature | The same ladder applied blind to feature coding |
| Confounders | Industry, company size band, offer date, open buyer requests at the time, sales attention hours | Adjusted associations under the stated model | All major common causes measured; remaining ones bounded by sensitivity analysis |
| Analysis | Primary features named in advance; models, interval method and multiple-comparison rule written down | Estimates with intervals for primary features; others labeled exploratory | Agreement with a randomized sub-study on at least one feature |
| Sensitivity | E-value for each primary association; re-analysis without the largest industry | How strong hidden confounding would need to be to erase each result | Large E-values and stable estimates across the re-analyses |
| Reporting | STROBE-style write-up with flow of offers, missing data and limits | A reproducible description of what was seen | Independent replication in another pipeline or period |
Measuring interest without letting the sales team define it
Interest has to be an observable event with a timestamp, not a judgment. The ladder in the plan uses three steps that leave a record: a buyer asks for details of a specific offer, asks for a sample, or asks for terms. Each offer's outcome is the highest step reached within the follow-up window. Decide in advance whether an offer shown to no buyer counts as no interest or as missing; the honest answer is usually missing, with a separate analysis of what predicted being shown at all.
Code features and outcomes separately. The person who codes documentation completeness at intake should not see the outcome, and the outcome should be pulled from system timestamps rather than recalled. Write a short coding guide for every feature with an example for each level, test it on a handful of offers with two coders, and resolve disagreements before the window opens.
Exposure differs between offers, and that is the most likely hidden bias. An offer that a busy team showed to many buyers had more chances to attract interest than an equally good one that sat unpromoted. Record how many buyer requests the offer was matched against and how much time was spent on it, and adjust for both.
The analysis plan and the sensitivity test for hidden confounding
Name a small number of primary features before the data is seen and treat everything else as exploratory. For each primary feature, report the share of offers reaching each step of the ladder with and without the feature, then an adjusted estimate with an interval that accounts for clustering by company. Write down the multiple-comparison rule you will apply, and keep it even if a favorite feature falls just short.
Then ask how fragile each association is. VanderWeele and Ding's E-value expresses the minimum strength of association, on the risk ratio scale, that an unmeasured confounder would need with both the feature and the outcome to explain the result away. Report it for every primary association. A small E-value tells a reader that a modest hidden factor, such as which industries happened to be in demand that quarter, could account for the finding.
Finally, plan the comparisons that would embarrass the hypothesis. If history length matters, it should matter within industries as well as across them, and it should matter for offers matched against the same buyer requests. Write those checks down in advance so a reader can see they were not added after a good result.
Turning one association into a causal test
Only a feature the study can change can be tested causally, and documentation is the obvious candidate. Among offers that pass intake, randomly assign half to receive a structured documentation upgrade, such as a filled data dictionary and an event description, before being shown to buyers, and leave the rest as they arrived. Compare the interest ladder between the two arms over the same window.
Randomization removes the confounding and selection problems that the observational part can only bound. It also has limits: suppliers must agree to take part, the upgrade has to be one the study can deliver consistently, and withholding help from the control arm must be acceptable to everyone involved. Features such as history length or record volume cannot be randomized at all, so their associations remain correlational however carefully they are adjusted.
Pre-register the sub-study with the main study, analyze it on its own, and only then compare it with the observational estimate for documentation. If the two disagree, the randomized result governs, and the gap is itself a finding about how much bias the observational design carried.
Pre-registration checklist before the first record is coded
The Center for Open Science describes preregistration as posting a time-stamped, read-only version of the study plan to a public repository before data collection or analysis begins. Use the checklist below as the minimum; anything decided later is labeled exploratory in the report.
At a glance
Pre-registration checklist before the first record is coded
Checklist · 10 items
- Research question in one sentence, with unit, features, outcome and period.
- Intake window dates, follow-up window length and the rule for offers still open when the study ends.
- Inclusion and exclusion rules, and the stage at which each exclusion is recorded.
- Feature coding guide with levels, examples and the inter-coder test planned.
- Outcome ladder definitions and the system fields each step is read from.
- Confounders measured and the adjustment model, including clustering by company.
- Primary features, the interval method and the multiple-comparison rule.
- Sensitivity analyses: E-values, the leave-one-industry-out re-analysis and the exposure adjustment.
- Missing data rules for features and outcomes.
- What result would count against each hypothesis, written before any outcome is seen.
0 of 10 ticked. Tick each item as you go.
How to report it so a reader can check the work
Follow the structure of STROBE, the reporting guideline for observational research, whose checklist runs from title and abstract through methods, results and discussion. In particular, show the flow of offers from intake to analysis with counts at each stage, report missing data per variable, and state the population the findings apply to and the ones they do not.
Publish the coding guide and the aggregate tables. Do not publish anything that identifies a supplier, a buyer or a specific offer, and do not report cells so small that a participant could be recognized. A useful report says plainly which associations survived the sensitivity tests, which did not, and which questions only the randomized arm can answer.
What a business owner can take from the protocol today
Before any study is run, the protocol is a way to read claims about what makes data valuable. Ask whether the claim counted the datasets that had the feature and drew no interest, whether the feature was measured before buyers were involved, and whether industry and timing were held constant. A claim that fails those questions is an anecdote, however confident it sounds.
SourceX publishes a working rule of the same kind: as a general guide, buyers look for years of consistent history, typically two or more, and tens of thousands of records or more in one system, while smaller sets can qualify when the work is rare, and each dataset is assessed on its own. That is exactly the sort of statement this protocol is built to test rather than repeat. For a company wondering where its own records stand, the first step is a short company assessment that needs no data upload.
Frequently asked questions
What is an operational data valuation evidence study?
It is a study that collects evidence about which properties of operational business datasets go with interest from AI data buyers, using a defined population, fixed definitions and disclosed methods. The version on this page is an observational design with a randomized sub-study for one feature. It is published as a protocol, with no results.
Why not just survey buyers about what they value?
A survey records what buyers say they value, which is useful but different from what their behavior shows. Stated preferences tend to name features that are easy to describe, and a survey cannot count the offers buyers ignored. The two methods work best together, with the survey used to choose which features the observational study codes.
What population does this kind of study cover?
Only the offers that entered the chosen intake pipeline during the study window. Its findings describe that pipeline, its industries and its period, and they do not describe the wider market for business data. A second pipeline or a later period is needed before anyone generalizes.
What sample size does the study need?
That depends on how many offers the pipeline receives, how often the outcome occurs and how small an association is worth detecting, so it should be set with a statistician before the window opens. With few offers, report estimates with wide intervals and treat the work as exploratory rather than lowering the bar after the fact.
Can the study say what a dataset is worth?
No. It studies signs of buyer interest, not prices or model performance, and it reports associations, not amounts. Price depends on terms, exclusivity and the buyer's use, none of which an interest ladder measures.
Sources
- STROBE publishes reporting checklists for observational studies covering title, abstract, introduction, methods, results and discussion. Source
- VanderWeele and Ding define the E-value as the minimum strength of association an unmeasured confounder would need with both exposure and outcome to explain away an observed association. Source
- Hernán and Robins make Causal Inference: What If, a text on causal inference from observational data, freely available. Source
- The Center for Open Science describes preregistration as posting a time-stamped, read-only study plan before data collection or analysis. Source
- Data Shapley values individual training examples by their contribution to a model's performance. Source
- Datasheets for Datasets proposes documenting each dataset's motivation, composition, collection process and recommended uses. Source
See if your company qualifies
A short company assessment. No data uploads are needed.