Regulatory Requirements for AI Validation and Test Data

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Which rules reach validation and test sets directly

The EU AI Act is the main horizontal law that names testing data sets as a regulated object; sector regulators add independence expectations on top. Article 10(1) says high-risk systems that train models must be developed on training, validation and testing data sets meeting the quality criteria in paragraphs 2 to 5 [1]. Those criteria cover governance practices (design choices, collection processes and origin, preparation such as labelling and cleaning, assumptions, bias examination and gap analysis), plus the requirement that sets be relevant, sufficiently representative, and to the best extent possible free of errors and complete [1].

As of October 2026, the AI Act has been amended by Regulation (EU) 2026/1744, published in the Official Journal on 24 July 2026 [2]. Secondary commentary reports that high-risk obligations now apply from 2 December 2027 for Annex III systems and 2 August 2028 for Annex I products; the same regulation also amends Article 10. Check the consolidated text before relying on these points [2]. The deeper Article 10 walkthrough for training data lives on our Article 10 data governance guide; this page focuses on the held-out side.

Article 10(6): when only the test set is regulated

If a high-risk system does not use techniques involving the training of AI models, Article 10(6) applies paragraphs 2 to 5 only to the testing data sets [1]. That covers rules-based scoring engines, retrieval pipelines on a frozen third-party model, or a prompted foundation model deployed in an Annex III use case where the provider does no fine-tuning. In those builds the test set is the only data artifact that carries the full governance burden.

The practical consequence is that a licensed evaluation set for such a system needs the same paperwork you would demand for training data. You need documented origin, the original purpose of collection, labelling, cleaning and enrichment steps, and an examination of possible biases affecting health, safety or fundamental rights [1]. A benchmark scraped from a public leaderboard rarely ships with that record, which is one reason teams move to private, held-out evaluation sets.

Independence and separation: the regulatory expectation

Regulators expect test data to be independent of training data, and the expectation goes beyond avoiding duplicate rows. The Good Machine Learning Practice guiding principles published jointly by the FDA, Health Canada and the MHRA in 2021 ask that training and test data sets be independent, with sources of dependence such as patient, data acquisition and site factors considered. They also ask that data represent the intended patient population; verify the current FDA texts before citing them in a submission. Our FDA device data page covers the device-specific submissions.

The AI Act does not use the word "independent," but Article 10's representativeness and error criteria are hard to meet with a leaked test set, and practitioner guidance recommends strict separation of training, validation and test sets plus a documented split method [10]. Common failure modes a reviewer will probe include:

• Entity leakage: the same customer, patient, device serial or code repository appears in both training and test splits.
• Temporal leakage: a random split over a ticket or claims history lets the model see the future; reviewers often expect a cutoff-date split.
• Template leakage: near-duplicate documents (same contract template, same form layout) sit on both sides even when exact hashes differ.
• Pretraining contamination: a public benchmark, or text derived from it, already sits in the foundation model's corpus.
• Supplier overlap: the same source company supplies both your training corpus and your "independent" test set, sharing labeling conventions and data-entry habits.

For the mechanics of detecting overlap in licensed sets (n-gram and embedding near-duplicate checks, canary strings, cutoff dating), see contamination checks for licensed evaluation data.

Test-set quality is a compliance question, not only a metrics question

Article 10(3) requires data sets, test sets included, to be free of errors to the best extent possible [1], and published evidence shows how far common test sets fall short. Northcutt, Athalye and Mueller estimated an average label error rate of at least 3.3% across test sets of 10 widely used vision, language and audio datasets, including at least 6% of the ImageNet validation set [8]. Errors at that rate can reorder model rankings, which matters if you declare accuracy figures in technical documentation.

ISO/IEC 5259-1 gives a shared vocabulary for data quality characteristics across the data life cycle [9]. Using its terms in your test-set datasheet makes it easier for a notified body to map your evidence onto Article 10. If you build the set yourself from operational records, our guide to a golden evaluation dataset from business records covers adjudication and reviewer agreement.

What to document for each licensed validation or test set

Annex IV requires technical documentation describing the validation and testing procedures used, including information about the validation and testing data and their main characteristics, the metrics used, and dated and signed test logs and reports [3]. For training data, point 2(d) asks for datasheets covering general description, provenance, scope and main characteristics, how the data was obtained and selected, and labelling and cleaning methods [3]; using the same headings for each validation and test set keeps the record consistent. The Annex IV dataset documentation guide shows the training-side equivalent.

Illustrative example: invented to show structure; it does not describe an available dataset.

test_set_record:
  set_id: TS-CLAIMS-TRIAGE-v3
  role: test            # training | validation | test
  frozen_on: 2026-08-31
  sha256_manifest: manifests/ts-claims-triage-v3.sha256
  source:
    supplier_type: US property insurer (operational claims notes)
    collection_purpose: claims handling, not ML
    license_ref: LIC-2026-014, schedule B
    permitted_uses: [evaluation, regulatory_review]
    regulator_access_permitted: true
  split_method:
    rule: temporal cutoff (claims opened after 2026-03-01) + policyholder-ID disjoint
    entity_overlap_check: 0 shared policyholder IDs with training manifest
    near_dup_check: MinHash Jaccard >= 0.8 flagged and removed
    pretraining_contamination_check: 13-gram overlap scan vs base-model probe set
  composition:
    records: 4,200
    strata: [state, line_of_business, claim_severity_band]
    representativeness_note: compared against 2025 book mix by state
  labels:
    guideline_version: triage-guide-2.1
    annotators: 3 licensed adjusters, double-labeled
    agreement: Cohen kappa reported per stratum
    adjudication: senior adjuster resolves disagreements
  bias_examination: error rates reported by state and claimant age band
  personal_data: names, policy numbers, phones replaced with typed placeholders; method recorded
  retention: keep frozen copy for life of system plus Article 18 period

Licensing terms that make a test set usable for regulatory review

A test set licensed only for "internal evaluation" can block you at audit time, so the license has to anticipate regulators. Under Annex VII, a notified body may be granted access to training, validation and testing data sets, and Article 74 gives market surveillance authorities related access powers [4][5]. Article 78 binds those bodies to protect trade secrets and confidential business information, which is the point to raise with a supplier worried about disclosure [7].

Retention is the second gap. Article 18 requires providers of high-risk systems to keep technical documentation for 10 years after the system is placed on the market or put into service [6]. If your evaluation license forces deletion of the test set at term end, you may keep the report but lose the ability to re-run it for a new model version or a regulator's query. Our retention versus deletion duties guide covers how to reconcile the two, and whether data can be licensed for evaluation only explains how evaluation-scoped licenses differ.

Illustrative example: invented to show structure; it does not describe an available dataset.

A pre-release checklist for regulated evaluation leads

Before you report a metric from a licensed test set in technical documentation, confirm the set can withstand a notified-body review.

Illustrative example: invented to show structure; it does not describe an available dataset.

1. Classify the system: high-risk under Annex III or Annex I, and whether training occurs (decides if Article 10(6) applies) [1].
2. Freeze the set with a hashed manifest and a version ID referenced in test reports.
3. Record the split rule and run entity, temporal, near-duplicate and pretraining contamination checks.
4. Measure label quality: double-label a sample, report agreement, log adjudication.
5. Document representativeness against the intended population, by stratum.
6. Run and record a bias examination on test-set error rates.
7. Confirm license terms cover regulator access, retention and re-testing.
8. Sign and date test logs and reports as Annex IV requires [3].

For the broader evidence pack auditors ask for, see AI training data audit readiness and the compliance hub. The distinction between the two data roles is covered in training data vs evaluation data.

How SourceX approaches licensed validation and test data

SourceX sources operational datasets from US companies on request and manages the licensing process; nothing is held in stock and a request does not guarantee a match. Every dataset is rights-reviewed for ownership and consents and delivered under a license that defines the records, uses, term and delivery. Diligence materials covering source, rights, preparation and allowed use are prepared per dataset, which buyers can draw on when compiling their own Article 10 and Annex IV records. Evaluation teams can describe the held-out data they need by role, strata and permitted uses.

Request held-out data for regulated evaluation

SourceX finds US businesses that hold the operational data you describe, assesses data and licensing permissions, and agrees allowed uses in a license before any transaction. Personal details are removed or replaced before delivery, and delivery runs through private, access-controlled workflows only after an executed agreement and supplier approval. Describe your validation or test data requirement.