Dataset Acceptance Testing: Inspection Windows, Rejection Notices and Cure Periods

This page covers procedure: who tests, when, and what paper changes hands. The thresholds themselves belong in your acceptance criteria for licensed training data, and the money side in milestone payments tied to acceptance. Both sit under the AI training data procurement hub.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Why data deliveries need a defined acceptance procedure

Data needs an explicit procedure because its defects surface late and quietly, often weeks after the files land. A schema mismatch fails on the first load, but label drift, duplicated conversations, leaked identifiers or a missing month of tickets may only appear during deduplication, a training run or an eval regression. Software contracts handle this with acceptance clauses that test against agreed criteria, require written acceptance or stated grounds for failure, and cap the fix window [2]. Public procurement templates go further, such as the Council of Europe's act of acceptance, which records a formal, signed acceptance of what was delivered [1].

Without that structure, three things go wrong. Payment milestones trigger on "delivery" rather than on a tested result. Rejection arrives as a Slack message nobody can find during a dispute. And the license's remedy clauses have no clean starting date.

Who tests and who signs

Acceptance works when one named owner signs and each test has a named executor. A practical split for a licensed operational dataset (support tickets, engineering records, finance workflows) looks like this:

• Data engineering runs structural checks: file inventory, checksums, schema, row counts, encodings, partitioning.
• ML or applied research runs fitness checks: label agreement on a sample, class balance, deduplication, leakage against your eval sets.
• Privacy runs a residual-PII audit on a sample, using the method in auditing residual PII in a delivered dataset.
• Legal or procurement confirms the delivery matches the license schedule (record types, fields, date range, permitted uses) and owns the notice.

The procurement manager compiles results and issues the certificate or rejection. Name these people in the license schedule or the delivery plan, with a backup for each, because a window that runs while the privacy reviewer is on leave is a window that lapses.

Setting the inspection window

The inspection window should be long enough to run every agreed test on the full delivery, not just to open the files. Size it from the work: a structural pass on Parquet can be automated in hours, since each file should begin and end with the PAR1 magic number and carry its metadata in a footer that a reader can validate [5]. A human label audit or PII review of a few hundred records takes days of reviewer time.

Practical rules for negotiating the window:

1. Start the clock at complete delivery, defined as all files plus the manifest, data dictionary and preparation notes, not at the first upload.
2. Pause the clock while the supplier answers a written clarification request.
3. Run separate windows per batch for staged or ongoing deliveries, so a late batch does not compress testing time.
4. Keep latent-defect rights after acceptance for defects that a reasonable inspection would not catch, such as rights or consent problems or re-identifiable records.

Sampling plans for checks you cannot run on every record

Use a written sampling plan for any check that needs human review, and agree it before delivery. Industrial quality control has mature tools for this: a single sampling plan specifies a sample size and an acceptance number derived from acceptable and rejectable quality levels [3], and ANSI/ASQ Z1.4 publishes attribute tables that set sample sizes and accept/reject numbers by lot size [4]. Treating each batch as a lot and each mislabeled or PII-bearing record as a defective unit is a workable adaptation, though these tables were designed for manufactured lots, not records.

Two cautions apply. Stratify the sample by source system, month and record type, because defects in operational data cluster (one ticketing queue, one migration). And never treat a clean automated PII scan as proof; detection tools themselves state that they cannot guarantee finding all sensitive information [6]. For health data, confirm which HIPAA method was used, Safe Harbor or Expert Determination [7], before you start testing, because the evidence you review differs.

Written acceptance, rejection notices and certificates

Every outcome should end in a dated document, never in silence or chat. A data acceptance certificate mirrors the public-procurement act of acceptance [1]: what was delivered, against which criteria, by whom, with what result and which reservations.

Illustrative example: invented to show structure; it does not describe an available dataset.

A good rejection notice is specific enough that the supplier can fix the problem without a meeting: "Field resolution_code null in 14.2% of sampled rows; agreed maximum 2%; affected partitions 2025-03 and 2025-04." Vague notices ("quality is poor") invite disputes and can be argued to fail the notice requirement.

Cure periods, re-delivery and re-testing

A cure period gives the supplier a capped time to fix specified defects, after which the buyer re-tests only the corrected scope. Cap the fix window [2] and the number of cure cycles, and state what happens after the last one fails: partial acceptance, a price adjustment or termination of that batch. The remedies themselves, from replacement records to credits and refunds, are covered in remedies when a data delivery fails.

For re-delivery, require a new manifest with new checksums, a change log that names corrected partitions, and the same delivery format as the original, so your pipeline does not need rework. Re-test with a fresh sample rather than the records you already flagged; fixing only flagged rows is the classic way to pass re-testing while leaving the defect rate unchanged.

Partial acceptance by batch or subset

Partial acceptance lets you take and pay for what passes while the rest is cured. It suits staged deliveries (monthly extracts, one source system at a time) and deliveries where one record type fails, such as call recordings passing while their transcripts miss accuracy thresholds.

Write it in three parts. Define the acceptance unit in the license schedule (batch, partition, record type). Make clear that accepting one unit does not waive rights on others or on latent defects. And state whether a failed unit can be removed from scope with a price adjustment, since some subsets are worthless without their linked records, for example tickets without their conversation threads.

Avoiding silent deemed acceptance

Deemed acceptance is a clause stating that a delivery counts as accepted if the buyer does not reject it within the window; some templates include it by default [2]. For data, negotiate it out or neutralize it, because the defects that matter most surface after the window.

Illustrative example: invented to show structure; it does not describe an available dataset.

Deemed acceptance checklist

• Window starts at complete delivery, including manifest and documentation
• Window length based on the agreed test plan, with a pause for open clarification requests
• Deemed acceptance removed, or triggered only after a written reminder plus a further grace period
• Using data in production counts as acceptance only if stated; trial loading and testing never count
• Latent-defect and rights-defect claims survive acceptance
• Notice addresses and methods named for both sides
• Calendar reminders set for each window, owned by the procurement manager

Note that loading files into a feature store or starting a training run on them may be argued to be use that implies acceptance. Keep acceptance testing in an isolated environment until the certificate is signed.

How SourceX fits the acceptance step

SourceX sources operational datasets from US companies on request and manages the commercial process, including licensing agreements and ongoing purchases. Each dataset is rights-reviewed for ownership and consents and delivered under a license that defines the records, uses, term and delivery. Personal details are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. That gives your acceptance testing a documented baseline to test against, and you can describe the data you need on the buyers page. For how supplier-side commitments interact with acceptance, see data supplier SLAs and buyer acceptance and getting paid. Delivery mechanics are covered in dataset delivery formats, schemas and transfer.

Start a dataset request with acceptance in mind

SourceX looks for US businesses that hold the data you describe, and every release is approved by the supplying company; nothing is contracted until a supplier agrees. Diligence materials on source, rights, preparation and allowed use are prepared per dataset, and delivery runs through private, access-controlled workflows only after an executed agreement. Describe the dataset you need to license.