Skip to content

AI data market

How to measure AI data supplier concentration: a study protocol

By SourceX Editorial · Updated

9 sections

Short answer

This is a study protocol, not a result. To measure how concentrated the supply of real workflow data is, define who counts as a supplier, pick one share measure with a stated denominator, code disclosed licensing activity from public sources under rules fixed in advance, and report top-four share and HHI with bounds for undisclosed supply. No findings are reported here; the page publishes the method only.

Value drivers

AI data supplier

This page covers how to measure AI data supplier concentration: a study protocol.Illustrative

Key takeaways

  • This page publishes a method for measuring supplier concentration and reports no results.
  • Count originating suppliers and intermediaries separately; mixing layers distorts concentration either way.
  • Choose one share measure and state its denominator before collecting anything.
  • Report top-four share and HHI with bounds for undisclosed supply, and do not treat merger thresholds as verdicts.
  • Code every observation against the limits of its public source, and pre-register the plan.

How could a study measure concentration in the supply of real workflow data?

A defensible study answers one narrow question: among organizations that license real business workflow records to AI developers, what share of disclosed supply comes from the largest few? It needs a written definition of a supplier, one share measure with a stated denominator, a fixed set of public sources searched with fixed queries, two independent coders, and concentration statistics reported with explicit bounds for the supply nobody discloses.

This page is that protocol. It describes the design, the public source research and limitation framework, the uncertainty treatment and a pre-registration checklist. It reports no measurements, no market shares and no ranking of companies, because none have been collected under this method yet. Anyone quoting a concentration figure for this market should be able to show the equivalent of each step below.

Real workflow data here means records produced by operating businesses doing their work: tickets, case files, approvals, quotes, schedules and the event logs around them. It excludes synthetic data, crowd-sourced annotation produced for a buyer and scraped public web text, which are separate supply chains with different suppliers.

Who counts as a supplier, and at which layer

Count suppliers at the layer where records originate, and measure intermediaries separately. An operating company that licenses its own records is an originating supplier. A broker, marketplace or data company that aggregates records from many companies is an intermediary, and concentration among intermediaries can be high while origination stays widely spread, or the reverse.

Mixing the layers is the most common way a concentration claim goes wrong. If one intermediary represents dozens of originating companies, counting it once understates originating suppliers; counting it alongside them double counts the same records. The study codes every observation with its layer and reports each layer on its own.

Who counts as a supplier, and at which layer
Organization typeLayerIncluded?Coding rule
Operating company licensing its own recordsOriginationYesOne supplier per legal entity; affiliates under common control combined
Broker or marketplace aggregating many companies' recordsIntermediaryYes, reported separatelyRecord the originating companies when disclosed; otherwise flag as undisclosed origin
Annotation or expert-labeling vendorServiceNoExclude unless it licenses pre-existing operational records
Synthetic data producerServiceNoExclude; note in limitations if a deal mixes real and synthetic records
Publisher or platform licensing user-generated contentOrigination, separate stratumSensitivity analysis onlyReport with and without, since it is not business workflow data
AI developer collecting from its own operationsIn-houseNoExclude from supply; mention as a substitute in limitations

Choosing a share measure and stating its denominator

Pick one primary share measure before collecting anything, because different measures can rank the same suppliers differently. Disclosed deal counts favor companies that announce deals; record volume favors a few very large datasets; buyer counts favor suppliers that sell non-exclusively. None is wrong, but a result is meaningless without saying which one was used and what the denominator was.

The denominator should be the total for the same measure across all included suppliers in the period, with undisclosed observations counted in a separate residual. A share computed only over suppliers with complete data will overstate how much the largest ones hold whenever smaller suppliers disclose less.

Choosing a share measure and stating its denominator
Share measureWhat it capturesWhere it can be observedMain bias
Disclosed licensing agreementsHow many deals each supplier is party toFilings, announcements, court recordsOver-represents public companies and large deals
Distinct datasets offeredBreadth of each supplier's catalogDataset listings and documentationCounts listings, not delivered records; catalogs may overstate availability
Record or case volumeSize of supply in units of workDataset documentation and deal disclosuresRarely disclosed; one bulk dataset can dominate
Distinct buyers servedReach of each supplierAnnouncements and buyer disclosuresExclusive deals undercount reach by design
Contract valueEconomic weight of supplyMaterial agreement disclosures onlyAlmost never public; not recommended as primary

Which concentration statistics to report

Report the combined share of the four largest suppliers and the Herfindahl-Hirschman Index, and show the full share distribution beside them. The U.S. Justice Department describes the HHI as the sum of each firm's squared market share, approaching zero when many firms of similar size compete and reaching 10,000 when one firm holds the whole market; its worked example of four firms with shares of 30, 30, 20 and 20 percent gives 2,600.

Concentration ratios are the simpler companion. The Census Bureau's 2017 Economic Census reports the share of the 4, 8, 20 and 50 largest firms and calculates the HHI from the top 50. Copying that convention makes the study's output familiar to economists, as long as the study states that its shares cover disclosed supply only.

Do not borrow merger thresholds as verdicts. The 2023 Merger Guidelines from the Justice Department and FTC treat a market with an HHI above 1,800 as highly concentrated and an increase of more than 100 as significant, but those are presumptions applied to antitrust markets defined under the guidelines' own tests. A public-source count of data licensing deals is not such a market, and the study should say so wherever an HHI appears.

The public source research and limitation framework

Use only sources a reader can check, record every search exactly as run, and code each observation against the limitation that applies to its source. The framework below lists the source types, what each can establish, what it cannot, and the coding rule that keeps its weakness visible in the results.

  • SEC states that EDGAR full-text search covers the full text of filings submitted electronically since 2001, which fixes the earliest period a filing-based measure can cover.
  • The Data Provenance Initiative's audit of more than 1,800 text datasets found license omission rates above 70 percent and error rates above 50 percent on widely used hosting sites, so host-displayed licenses cannot be coded as fact.
  • Datasheets for Datasets proposes that every dataset document its motivation, composition and collection process, and those fields are the right place to look for origin, when they exist.
  • The Consent in Crisis audit of about 14,000 web domains documented rapidly rising restrictions on AI crawlers between 2023 and 2024, which is context for why licensed private data matters, not a measure of its supply.
The public source research and limitation framework
Public sourceWhat it can establishWhat it cannotCoding rule
SEC EDGAR full-text searchLicensing agreements, revenue lines or risks disclosed by U.S. public filersAnything private companies do; deals below materialityStore accession number and quoted passage; flag public-filer bias
Company announcements and press releasesThat a named deal or partnership existsVolume, terms or whether delivery happenedCode as claimed by the company; never as verified volume
Dataset documentation and datasheetsOrigin, composition and collection process of a listed datasetWhether the dataset was licensed, or to whomCode catalog presence separately from deal evidence
Dataset hosting license metadataThe license a host displaysThe license the original author intendedCheck against the original source; flag mismatches
Court filings and regulatory recordsDisputed or investigated data arrangementsThe typical, undisputed dealUse as existence evidence only
Web crawl and access-restriction auditsChanges in the public web as a substitute sourceLicensed private workflow dataUse as comparison context, never as supply share
Interviews and surveysUndisclosed activity and contextRepresentative shares unless sampled properlyReport separately with recruitment method and response rate

Sampling, coding and how to show uncertainty

Fix the search strings, sources and period in advance, run every search on a recorded date, and keep the raw result lists. Two coders should code each observation independently for supplier, layer, measure and evidence strength, then resolve disagreements by a written rule; report their agreement before resolution.

Uncertainty in this design comes less from sampling noise than from missing disclosure, so show it as bounds. Treat the undisclosed residual two ways: as if it were spread across many small suppliers, which gives a lower bound on concentration, and as if it belonged to a single additional supplier, which gives an upper bound. Report the disclosed-only figures beside both bounds. If the bounds are wide, that width is the finding.

Add resampling intervals only for the parts of the data that are a sample, such as survey responses, and say which figures they apply to. Run the analysis again excluding the supplier with the largest share, and excluding each source type in turn, to show how much any single input drives the result.

A pre-registration checklist for a supplier concentration study

Register the plan before collecting data. The Center for Open Science describes preregistration as posting a time-stamped, read-only version of the study plan to a public repository before data collection or analysis begins, and a concentration study benefits from it because the choice of measure and supplier definition can change the answer.

At a glance

A pre-registration checklist for a supplier concentration study

Checklist · 10 items

  1. Research question in one sentence, naming the layer and the share measure.
  2. Population: supplier definition, inclusion and exclusion rules, and the treatment of affiliates.
  3. Period: start and end dates, and how deals that span the boundary are counted.
  4. Sources: every source type, with the exact search strings and the date each search will be run.
  5. Denominator: how the total is built and how the undisclosed residual is estimated or left open.
  6. Coding: the codebook, the two-coder process, the agreement statistic and the tie-break rule.
  7. Statistics: top-four share, HHI, the full distribution, the bounds method and any resampling intervals.
  8. Sensitivity analyses: which suppliers, layers or source types will be dropped and re-run.
  9. Stopping rule: what result, such as bounds too wide to interpret, will be reported as inconclusive.
  10. Disclosure: who funded the study and any interest the authors have in the market.

0 of 10 ticked. Tick each item as you go.

What a report from this protocol should publish

Publish the evidence table, not only the statistics. A reader should be able to recompute every share and every concentration figure from the released tables, and to see which observations rest on a single company announcement rather than a filing or documented dataset.

Report time comparisons only when the same sources, queries and coding rules were applied to each period. A rise in measured concentration between two periods can come from a change in who discloses deals, so any trend statement needs the disclosure counts by source type beside it. When the method changes between waves, say so and rerun the earlier period under the new rules before comparing.

  • The registered plan and a list of every deviation from it, with the reason and date.
  • The raw search log: source, query, run date and number of results returned and screened.
  • The coded observation table with supplier, layer, measure, evidence strength and source link for each row.
  • Share tables for each layer, with the disclosed-only figures and the lower and upper bounds side by side.
  • Coder agreement before resolution, and the sensitivity runs with each one's effect on top-four share and HHI.
  • A plain-language limitations paragraph placed before the results, naming the disclosure bias.

Conclusions the design cannot support

A public-source concentration study can describe disclosed supply in a stated period. It cannot define an antitrust market, show that any supplier has market power, estimate prices or buyer demand, rank data quality, or say why supply is concentrated. Any of those claims needs different evidence.

The population also limits the reading. Disclosure favors public companies, large deals and suppliers that market themselves, so a high measured concentration can reflect who announces deals as much as who supplies data. Name that bias in the abstract of any report, not only in a limitations section.

For business owners reading concentration claims, the useful question is whether the study shows its supplier definition, denominator and bounds. If it does not, treat the figure as an opinion. An owner who suspects the company could itself be one of the suppliers such a study would count can find out through SourceX's short eligibility assessment, which involves no upload; if the company goes ahead, SourceX runs the rights review, contracting, preparation, delivery and payment for the records it chooses to license.

Frequently asked questions

Does this page report how concentrated AI data supply is?

No. It publishes the method for measuring concentration and reports no results, shares or rankings. Results would require collecting and coding public evidence under the registered plan, then reporting them with their bounds.

What is the Herfindahl-Hirschman Index?

It is a concentration measure calculated by squaring each firm's market share and adding the squares. It approaches zero when many similar-sized firms compete and reaches 10,000 when one firm holds the whole market, according to the U.S. Justice Department.

Can merger guideline HHI thresholds be applied to data supply?

Not as verdicts. The 1,800 and 100 point thresholds in the 2023 Merger Guidelines are presumptions for antitrust markets defined under the guidelines' tests. A count of publicly disclosed data licensing deals is not such a market, so the thresholds can be mentioned for context only.

Why not use dataset hosting sites to count suppliers?

Hosting sites show what is listed, not what was licensed, and their license metadata is often missing or wrong according to the Data Provenance Initiative's audit. They are useful for finding datasets and their documented origin, but each listing needs checking against its original source.

What conclusion would this study design not support?

It would not show market power, prices, demand, data quality or the causes of concentration. It describes disclosed supply in a stated period and population, and its results should be read with the disclosure bias toward public companies and large deals.

Sources

  • The U.S. Justice Department's Antitrust Division explains how the HHI is calculated from squared market shares, its range up to 10,000, and a four-firm worked example totaling 2,600. Source
  • The 2023 Merger Guidelines treat markets with an HHI above 1,800 as highly concentrated and a change of more than 100 as a significant increase, as part of a rebuttable structural presumption. Source
  • The Census Bureau's notes on the 2017 Economic Census say firm concentration ratios are given for the 4, 8, 20 and 50 largest firms and the HHI is calculated using the top 50 firms. Source
  • The SEC's EDGAR full-text search FAQ states that the search covers the full text of filings submitted electronically since 2001. Source
  • The Data Provenance Initiative audited more than 1,800 text datasets and found license omission rates above 70 percent and error rates above 50 percent on widely used dataset hosting sites. Source
  • Consent in Crisis audited about 14,000 web domains behind major training corpora and documented a rapid rise in restrictions on AI crawlers between 2023 and 2024. Source
  • Datasheets for Datasets proposes that every dataset be accompanied by a datasheet documenting its motivation, composition, collection process and recommended uses. Source
  • The Center for Open Science describes preregistration as posting a time-stamped, read-only version of a study plan to a public repository before data collection or analysis. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies