Skip to content

Definitions and comparisons

SourceX dataset acceptance stages: what each status really confirms

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

Each SourceX dataset acceptance stage records that one check happened, not that the dataset is accepted. Qualification needs no data uploads, rights review examines documents, signing fixes scope before any data moves, and only review of the delivered, de-identified data against written quality thresholds and the buyer's task test can support acceptance. The signed agreement, not a status label, defines each stage, so read the terms before the status.

Checklist

SourceX dataset acceptance

This page covers sourceX dataset acceptance stages: what each status really confirms.Illustrative

Key takeaways

  • A status is a summary of one completed check; the signed agreement for the deal defines what each stage means and what follows from it.
  • The early stages examine paperwork: a supplier qualifies without uploading data, and rights review works from contracts and notices, not the records.
  • De-identification changes the data, so test results from before preparation do not carry over to the delivered dataset.
  • Measured quality and buyer fit are separate tests; a dataset can meet every written threshold and still fail the task it was bought for.
  • Before fixing a dataset that was not accepted, find the failing gate in order: scope, rights and privacy, measurement, fit, then commercial terms.

What does each SourceX dataset acceptance stage actually mean?

Each stage records that one specific check was completed; none of them, short of a decision recorded under the signed terms, means the dataset has been accepted. Reading a stage correctly means knowing what it confirms, what it leaves open and which document defines it.

SourceX's public description lists its work in this order: sourcing and qualification, rights review, contracting, data preparation, delivery and payment. It also states that every dataset goes through rights, privacy, quality and buyer-fit review. SourceX has not published a glossary of status labels, so the stage names in this guide are descriptive rather than system labels, and the signed agreement for each deal is the authority on what each stage means.

The practical decision for a delivery or acceptance owner is simple to state: which evidence to ask for at each stage, and which question to answer first when a delivery is not accepted. The sections below give a reading guide, a five-gate decision tree and a rejection log that traces each failure to the stage that should have caught it.

Which document decides what a status means?

The signed agreement decides; a public explanation, a portal label or an email only summarizes it. When a summary and the signed terms disagree, the signed terms win and the mismatch should be logged so the summary gets corrected.

SourceX's public facts point the same way. The order runs approval, signature, then delivery: SourceX receives no data until the supplying company has accepted the scope and terms and signed, and from then on SourceX's right to use the de-identified dataset, sublicense it and distribute it reaches only as far as the agreement allows. Each deal's signed agreement also fixes exclusivity and term. Scope, permitted use and duration all live in the signed document, so a status cannot widen them.

Log each mismatch with four fields: the summary's wording and where it appeared, the clause of the signed agreement it conflicts with, who noticed it, and the date the summary was corrected. A short log like this is how public explanations stay aligned with signed terms as templates and pages change.

Federal contracting gives a precise reference definition. Under FAR 46.501, acceptance acknowledges that delivered supplies or services conform with the contract's quality and quantity requirements, it can happen before, at or after delivery depending on the contract, and it is ordinarily documented on an inspection or receiving report. A data license is not a federal supply contract, but the three ideas transfer: acceptance is defined by the agreement, tied to stated requirements and recorded in writing.

The stage reading guide: what each status confirms and leaves open

Read every stage as a narrow statement. The guide below covers the eight stages a buyer-side or supplier-side owner will meet, from a buyer's request through a recorded decision; payment, the last item in SourceX's public description, is governed by the signed terms.

Because SourceX finds datasets for buyers on request, from U.S. businesses, instead of keeping stock, the first stage is a written request, not a catalog pick. Use the guide to decide what to ask for before treating a stage as settled:

The stage reading guide: what each status confirms and leaves open
StageWhat it confirmsWhat it leaves openEvidence to ask for
Request receivedA buyer stated scope, volume, language, format, timing and licensing needsWhether matching records exist anywhereThe written request, intended use and acceptance test
Supplier qualifiedA company passed a short assessment without uploading dataAnything about the records' quality or fitAssessment result and system inventory
Rights reviewedAuthority to license the selected records was examinedPrivacy treatment, quality and buyer fitFindings listing the contracts and notices checked
Scope and terms signedThe supplier approved exact scope and terms before any data movedWhether the delivered data matches that scopeSigned agreement and its scope definition
Prepared and de-identifiedRecords went through de-identification by SourceX or its processing partnerWhether the data still passes the buyer's testMethod description and re-run test results
DeliveredData reached the agreed route, such as the seller's storage or encrypted drivesCompleteness, usability and acceptanceManifest with counts, checksums and validator output
Reviewed for quality and fitData was measured against written thresholds and the task testPayment, or any change to the signed scopeMeasurement report with sample sizes and results
Decision recordedThe party with authority under the signed terms decidedWhether a model trained on it will improveDated written decision citing the agreement

Which stages examine the data, and which only examine paperwork?

Only the stages after preparation examine the dataset a buyer will actually receive. Everything earlier examines descriptions, documents or a different version of the data, so it cannot by itself support acceptance.

Qualification is paperwork by design: SourceX's first step for a supplying company is a short assessment, and no data uploads are needed to see whether the company qualifies. Rights review works from contracts, notices and ownership, and it cannot lean on license labels. When MIT Media Lab researchers audited more than 1,800 text datasets, popular hosting sites had left licenses off at rates above 68 percent and labeled them incorrectly at rates above 50 percent.

Preparation changes the data. De-identification, performed by SourceX directly or through its processing partner, removes or transforms fields, so a sample that passed before preparation says little about the prepared version. Delivery is a logistics stage: a large dataset either remains in storage the seller controls or travels on encrypted drives, and arrival proves only that the data moved.

Format validation is the first real data check, and it is narrow. OpenAI's fine-tuning data-preparation notebook counts structural errors such as missing message lists, unrecognized roles, missing content and conversations without an assistant turn, and warns that examples over 16,385 tokens are truncated. Passing those checks shows the file is usable, not that it fits the task.

Measured quality versus buyer-specific acceptance

Quality and fit are separate tests and deserve separate statuses. Quality asks whether defects stay under written thresholds; fit asks whether the data does the job the buyer bought it for, and a dataset can pass the first while failing the second.

Quality measurement borrows from acceptance sampling. The NIST/SEMATECH handbook defines the acceptable quality level as the producer's baseline defect rate and the lot tolerance percent defective as a defect level the consumer would find unacceptable, and it frames the plan around producer's risk and consumer's risk. Writing both numbers down, with the sample size, turns a quality review into a check either side can rerun.

Fit needs a task test. Anthropic's January 2026 evaluation guide sets the bar for a good task at agreement: two domain experts, working separately, should arrive at the same pass-or-fail call. It also recommends a reference solution that passes every grader to prove the task is solvable. A buyer-specific acceptance test built that way stops an acceptance decision from resting on one reviewer's taste.

Documentation keeps both tests honest. Datasheets for Datasets asks creators to record motivation, composition, collection process, preprocessing, uses, distribution and maintenance, and MLCommons' Croissant RAI vocabulary gives machine-readable fields for collection, limitations, intended uses and personal or sensitive information. A delivery that records these lets a reviewer tell a quality miss from a fit miss.

The not-accepted decision tree: find the failing gate first

When a delivery is not accepted, identify the gate that failed before changing any data. The gates run in a fixed order because an earlier failure makes later work wasted: there is no point repairing records the signed scope excludes or the rights do not cover.

At a glance

The not-accepted decision tree: find the failing gate first

Work through the gates in order and stop at the first one that fails:

  • Gate 1, scope: are the failing records inside the signed scope? If not, the cure is a new signed approval or removal, not cleanup.
  • Gate 2, rights and privacy: would the fix need records, fields or uses that the signed terms or the privacy treatment do not permit? If so, stop and return to rights review.
  • Gate 3, measurement: does the failure reproduce when the agreed measurement is rerun on the agreed sample? If not, correct the test or the sample before touching the data.
  • Gate 4, fit: does data that meets every written threshold still fail the buyer's task test? If so, the problem is task mismatch, and the options are to re-scope or walk away.
  • Gate 5, terms: is the disagreement really about price, exclusivity, term or volume? If so, it is a commercial question, and no technical fix will settle it.
A five-gate decision tree for a dataset that was not accepted, checking scope, rights and privacy, measurement, fit and commercial terms in order, with the outcome for each failed gate.
The not-accepted decision tree: stop at the first gate that fails. Editorial framework; it reports no SourceX data.

When a technical fix cannot change the outcome

A technical fix only cures a technical defect. Three kinds of rejection survive any amount of cleaning, and spotting them early saves a repair cycle that cannot succeed.

A rights gap is the first. Cleaning, relabeling or de-identifying records does not create authority to license them; if the supplier lacked authority for some records, the only cures are removing them or obtaining the authority, and either may change the signed scope.

Commercial terms are the second. Exclusivity and term are set in each deal's signed agreement, so a disagreement about them is a negotiation, not a data defect, and it belongs with the people who can change the agreement.

Genuine task mismatch is the third. If the records lack the decision, outcome or context the task needs, no format change adds it. De-identification can sharpen this problem: removing identifiers can also remove signal, and adding fields back reopens the privacy review rather than closing the acceptance question.

The stage-of-origin rejection log

Log every rejection with the stage that should have caught it, not only the stage where it surfaced. A rejection found later than its stage of origin is a stage leak, and counting leaks by stage shows which earlier check needs tightening.

Each row below pairs a common rejection with its stage of origin, the kind of cure it allows and the evidence that closes it:

The stage-of-origin rejection log
Rejection reasonStage that should have caught itCure classEvidence that closes it
Records outside the signed scopeScope and terms signedNew signed approval or removal, not cleanupDiff of delivered records against the signed scope
Supplier lacks authority for some recordsRights reviewedRemove records or obtain authorityUpdated rights findings
Residual identifiers in prepared dataPrepared and de-identifiedRerun de-identification, then retestPrivacy check rerun on a stated sample
Wrong format or truncated examplesDeliveredTechnical repairValidator output before and after the fix
Defect rate above the written thresholdReviewed for quality and fitRepair, then re-measure on a fresh sampleMeasurement report with sample size and limits
Meets thresholds, fails the task testRequest received, where the task was underspecifiedRe-scope or walk awayRevised acceptance test agreed in writing
Dispute over price, exclusivity or termScope and terms signedCommercial negotiationSigned change to the agreement, or a recorded no-deal

What SourceX has not published, and how to measure stage leaks

SourceX has not published a status glossary, counts of deals at each stage, rejection rates or review timing, and this article reports none. The current supplier process is described on SourceX's how-it-works page, and the signed agreement for each deal governs its stages and any timing.

The stage-leak measure can be run by any team that keeps a rejection log. Its design is set out below; results are not yet published.

Checklist · 4 items

  1. Unit: one rejection finding, tagged with the stage where it surfaced and the stage of origin from the log above.
  2. Leak rate per stage: findings that originated at that stage but surfaced later, divided by all findings that originated there.
  3. Cohorts and denominators: deliveries grouped by the quarter the scope was signed, with the number of deliveries in each cohort reported beside every rate.
  4. Limits: small cohorts reported as counts, not percentages; findings re-coded by a second reviewer on a sample to check agreement; no comparison across buyers whose acceptance tests differ.

0 of 4 ticked. Tick each item as you go.

Writing the acceptance test into a SourceX data request

The cheapest stage leak to prevent is an underspecified request. Buyers who use the SourceX request form already share scope, volume, language, format, timing and licensing needs; adding the intended use and an acceptance test at that point lets every later stage check against the same target. Datasets are sourced to that request, so no stage before review of the delivered data can promise acceptance, and a precise test is what lets the final review reach a clear answer.

A useful acceptance test answers five questions before any data moves:

  • Task: what the model or evaluation must do with the data, written as one sentence a supplier can test against.
  • Reference: the outcome field, grader or expert rubric that decides a pass, with one worked example that passes it.
  • Thresholds: the defect types that count, the acceptable and unacceptable defect levels, and the sample size used to measure them.
  • Privacy constraints: the fields the de-identified data must still carry for the task, so preparation and fit are checked against one list.
  • Decision owner: who records the decision under the signed terms, and which evidence that person needs to see.

Frequently asked questions

Does a signed agreement mean the dataset has been accepted?

No. At SourceX the agreement is signed, with its scope and terms approved by the supplying company, before any data is delivered, so the signature precedes the data in delivered form. Acceptance is a later decision about the delivered, prepared data, made under the signed terms.

Why can a dataset pass quality checks and still not be accepted?

Because quality and fit are different tests. Quality checks count defects against written thresholds, while fit asks whether the data does the buyer's task, for example whether records carry the outcome a model must learn. A dataset can be clean and complete and still lack what the task needs.

Can de-identification cause a dataset to fail acceptance?

Yes. De-identification removes or transforms fields, and some of them may carry signal the task needs, so results from before preparation do not carry over. Rerun the acceptance test on the prepared data, and treat any request to restore fields as a new privacy review.

What should a buyer put in a request so acceptance can be decided cleanly?

State the intended use, the task test or grader, the written quality thresholds with a sample size, the privacy constraints and who records the decision. A test that two experts would grade the same way keeps the decision from turning on one reviewer's judgment.

Where are SourceX's acceptance terms actually defined?

In the signed agreement for each deal. Public pages, including SourceX's how-it-works page, summarize the current process, and this guide states no review timing; where a summary and the signed terms differ, the signed terms govern.

Sources

  • Under FAR 46.501, acceptance acknowledges conformance with contract quality and quantity requirements, can occur before, at or after delivery, and is ordinarily documented on an inspection or receiving report. Source
  • An audit of more than 1,800 text datasets found license omission above 68 percent and error rates above 50 percent on popular hosting sites. Source
  • OpenAI's fine-tuning data-preparation notebook checks structural errors and warns that examples over 16,385 tokens are truncated. Source
  • The NIST/SEMATECH handbook defines AQL and LTPD and the producer's and consumer's risk in acceptance sampling. Source
  • Anthropic's January 2026 guide calls a task good when two experts would reach the same verdict and recommends a reference solution that passes all graders. Source
  • Datasheets for Datasets proposes documenting each dataset's motivation, composition, collection process, preprocessing, uses, distribution and maintenance. Source
  • The Croissant RAI vocabulary provides fields for data collection, limitations, intended uses and personal or sensitive information. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies