SourceX Research · public sources
Why datasets get rejected: a review of public research on licensing and data quality problems
Quick answer
This page is a review of published research, not a SourceX study. Academic audits of widely used AI training datasets have found that license information is very often missing or wrong, and that consent signals on the web are inconsistent, which points to unclear rights and poor documentation as leading reasons datasets fail scrutiny.
By Noah Loul · Updated
This report summarizes published public sources, listed at the end. It does not contain SourceX survey or deal data.
Checklist
Why datasets get rejected
Why this is a literature review, not a SourceX study
SourceX has no proprietary data on dataset rejections. The findings below come from peer-reviewed and preprint research by the Data Provenance Initiative, a multi-university research collaboration, each cited at the source.
Missing or wrong license information
In a large-scale audit of more than 1,800 text datasets published in Nature Machine Intelligence, researchers found license omission rates of more than 70% and license error rates of more than 50% on popular dataset hosting sites, meaning that most of the time, the license shown (or missing) did not reliably describe the terms under which the data was actually released [1].
Inconsistent consent signals
In a separate audit of 14,000 web domains, titled "Consent in Crisis," the same research group found a rapid increase between 2023 and 2024 in AI-specific restrictions on web content, along with general inconsistencies between what a website's terms of service say and what its machine-readable robots.txt file actually permits [2]. That mismatch between stated intent and technical signal is a documented source of confusion for anyone trying to determine if content can be used.
What this suggests about rejection, generally
Based on this research, two broad categories explain why a dataset might fail a licensing or quality check: the rights to use the data are not clear or are actively restricted, and the documentation describing what is actually in the dataset is incomplete or incorrect. Both are fixable with better record-keeping, but both require work before data is handed to an outside party.
What this means for a business
A U.S. company considering licensing its own records should treat this research as a checklist: confirm and document exactly what rights the company holds over each dataset, write down where the data came from and when, and do not assume that a dataset with no stated license is automatically free to license as-is. Clear, accurate documentation is itself a major part of being ready to license data.
Frequently asked questions
Is this based on SourceX's own rejected submissions?
No. SourceX has no deal or rejection data of its own. This page summarizes independent academic research on dataset licensing problems.
What research is this based on?
Primarily the Data Provenance Initiative's published audits: a Nature Machine Intelligence paper auditing dataset licensing and attribution, and a separate "Consent in Crisis" audit of web domain consent signals.
Does missing a license mean data is unusable?
Not necessarily, but the research found that missing or incorrect license labels are extremely common, which creates legal uncertainty that an Acquisition Partner may treat as a reason to pass on a dataset.
Sources
Public sources, checked 2026-10-09. SourceX is not affiliated with the organizations named.
- Audit of over 1,800 text datasets found license omission rates over 70% and error rates over 50% on popular hosting sites nature.com
- Audit of 14,000 web domains found a rapid rise in AI-specific restrictions between 2023 and 2024 and inconsistencies between terms of service and robots.txt signals arxiv.org
See if your company qualifies
A short company assessment. No data uploads are needed.