FDA Expectations for AI Device Training and Test Data: What Licensed Datasets Must Show

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Which FDA documents set the data expectations

Three FDA documents shape what a data package for an AI-enabled device has to show, and only one of them is final. The draft guidance "Artificial Intelligence-Enabled Device Software Functions: Lifecycle Management and Marketing Submission Recommendations" (docket FDA-2024-D-4488) covers what to put in 510(k), De Novo and PMA submissions and proposes lifecycle recommendations. FDA's guidance list still marks it as a draft containing non-binding recommendations [1][2][3]. Treat it as FDA's stated current thinking, not a rule, and check for a final version before you lock a submission plan.

The Good Machine Learning Practice (GMLP) guiding principles, published jointly by FDA, Health Canada and the UK MHRA in October 2021, are ten short principles [4]. Several of them are about data. They call for clinical study participants and data sets that represent the intended patient population, training data sets independent of test sets, and reference data sets built on the best available methods.

The final guidance "Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions" (December 2024) rests on section 515C of the FD&C Act and covers PCCPs included in marketing submissions such as 510(k) and PMA [6]. It sits alongside the five PCCP guiding principles from FDA, Health Canada and MHRA [5]. For data buyers, the PCCP matters because any retraining data you plan to add after clearance has to fit a protocol you describe up front.

For the EU equivalent on training, validation and test data, see EU AI Act Article 10 data governance. For a cross-regime view of test sets, see regulatory requirements for AI validation and test data.

What the data management section of a submission covers

The draft guidance asks sponsors to describe the data lifecycle end to end, not just to report dataset sizes [2]. In practice that means a reviewer can trace every record in the training, tuning and test sets back to a source, a collection protocol and a labeling method. Licensed data that arrives without those records leaves a hole in the submission that you cannot fill later.

The topics a regulatory team should expect to document for each dataset are:

• Collection. Clinical sites or source organizations, collection period, acquisition devices and protocols (for imaging, scanner manufacturer, model, and acquisition parameters), and inclusion and exclusion criteria.
• Sources and independence. Which sources feed training versus test data, and whether any patient, site or device appears in both.
• Reference standard. How ground truth was established: pathology, follow-up outcome, consensus read or single reader. Also record reader qualifications and how disagreements were adjudicated.
• Annotation. Labeling instructions, tools, annotator training, inter-annotator agreement and quality control.
• Cleaning and processing. Exclusions, missing-data handling, normalization, and any de-identification transform applied before the data reached you.
• Representativeness. Distribution across age, sex, race and ethnicity, disease severity and subtypes, clinical sites and acquisition equipment, compared with the intended-use population.

These are the same dimensions a structured data quality management process covers. ISO/IEC 5259-3 gives requirements and guidance for running that process, though it does not prescribe metrics [10]. Asking a supplier to map its documentation to this list is a fast way to find gaps before contracting.

Why test data independence is the most fragile requirement

Independence fails most often through overlap that nobody recorded, not through deliberate leakage. GMLP lists independence of training and test data as its own principle [4], and the draft guidance asks sponsors to explain how test data was kept separate [2]. A licensed dataset can break that promise in ways that are invisible in the delivered files.

Common failure modes:

1. Same patient, different studies. A patient's 2022 chest CT lands in training and their 2023 CT lands in test. Without a stable, supplier-held pseudonymous patient key, you cannot detect it.
2. Same site in both sets. A model learns a site's scanner signature or a burned-in annotation style. Site-held-out testing is the stronger design, and that requires site identifiers that survive de-identification as coded values.
3. Supplier reuse. The same source data was licensed to a vendor whose pretrained model you fine-tune, so your "independent" test set overlaps with that model's training data.
4. Tuning on the test set. Thresholds or operating points chosen on the data later reported as the test set. This is a process failure, but the data records need to show which partition was used when.

Label quality in the test set matters as much as separation. An audit of ten widely used benchmark test sets estimated an average label error rate of at least 3.3%, and showed that such errors can change which model looks best [9]. For a device, a noisy reference standard in the test set directly weakens the performance claim.

What representative data means in practice

Representativeness is judged against the device's intended use and indications, not against the general population. The draft guidance asks sponsors to describe test data demographics and clinical characteristics and to report performance across relevant subgroups [2]. GMLP frames the same idea as data that reflects the intended patient population so that results generalize [4].

For a licensed dataset, that turns into concrete fields you need per record or per study: age band, sex, race and ethnicity where collected, the clinical site as a coded identifier, the acquisition device, disease status and severity, and relevant comorbidities. Many operational sources record race and ethnicity inconsistently or not at all. Ask suppliers for field-level completeness rates, not a yes or no.

Subgroup analysis also needs enough records per subgroup to estimate performance with useful confidence intervals. When a supplier's data skews toward one region or one device family, it may still serve as training data while failing as a test set. Decide which role each source plays before you license it.

How a PCCP changes what you license

A PCCP lets you make specified modifications after authorization without a new submission, provided they follow the protocol FDA reviewed [5][6]. The guiding principles describe a PCCP as planned modifications, a modification protocol and an impact assessment, and they require changes to stay focused and bounded within the device's intended use [5]. Retraining on new data is one of the most common planned modifications.

That makes data supply a regulatory dependency. If the modification protocol says new training data will come from sources meeting stated collection, labeling and representativeness criteria, each future delivery has to meet those criteria and be documented the same way. Check before signing that the license term and delivery terms support the retraining cadence your PCCP describes, and that the supplier will provide the same record set for each tranche. For retention questions, see training data retention requirements.

Where de-identified operational data fits, and where it does not

De-identified health data can support some device development work, but its fit depends on the role it plays. HIPAA de-identification uses either Safe Harbor, which removes 18 listed identifiers, or Expert Determination, a statistical assessment of re-identification risk [7][8]. A limited data set under 164.514(e) keeps some elements, such as dates, and requires a data use agreement [8]. Safe Harbor removes most dates and fine geography, which can break longitudinal follow-up, site identification and outcome linkage. Expert Determination can sometimes keep coded versions of these fields.

Fits well:

• Pretraining or representation learning on large volumes of imaging, waveforms or notes, where reference-standard rigor matters less.
• Workflow and triage models built on scheduling, claims or administrative events. See healthcare LLM fine-tuning data and SourceX's healthcare administration AI data page.
• Supplementary training data to widen device or site diversity.

Fits poorly:

• The pivotal test set, where FDA expects a documented reference standard, a defined collection protocol and subgroup characteristics that operational data rarely records [2].
• Any use that needs patient-level linkage across time when the de-identification removed the key that made linkage possible.

For what a HIPAA claim on a dataset should actually mean, see HIPAA-compliant AI training data.

Data documentation checklist for licensed device datasets

The checklist below turns the FDA expectations into requests you can send a supplier before you license data.

Illustrative example: invented to show structure; it does not describe an available dataset.

A worked record header for an imaging study might carry: patient_key (pseudonymous, stable), site_code, scanner_vendor, scanner_model, acq_date_offset_days, age_band, sex, race_ethnicity (nullable, with a reason code), reference_standard (e.g., biopsy), reader_ids, adjudicated (true or false), partition_lock (training, tuning or test, set by the buyer), and deid_method.

Records to keep after the data arrives

The submission is only as good as the records you keep from the day data arrives. Keep the license and its permitted uses, the supplier's provenance and de-identification documentation, a manifest with checksums for each delivery, and a frozen partition assignment showing which records went to training, tuning and test, with the date each was locked.

Record which model versions saw which partitions, so you can show the test set stayed sequestered. FDA's lifecycle approach also expects postmarket performance monitoring [1]. Plan how you will document real-world data used for monitoring with the same discipline. For the wider evidence pack, see AI training data audit readiness and the AI training data compliance hub.

How SourceX fits a device data plan

SourceX sources operational datasets from US companies on request and manages licensing; it does not hold inventory, and a request does not guarantee a match. Every dataset is rights-reviewed for ownership and consents and delivered under a license that defines records, uses, term and delivery. Health records require HIPAA de-identification by Safe Harbor or Expert Determination. Diligence materials covering source, rights, preparation and allowed use are prepared for each dataset, which you can map to the checklist above. Pivotal test data built to a reference-standard protocol is usually collected under its own clinical protocol, so describe precisely which role each dataset plays when you submit a buyer request.

Source data for an AI-enabled device submission

SourceX looks for US businesses that hold the data you describe, and every release is approved by the supplying company. Personal details are removed or replaced before delivery, the method is recorded, and a sample is checked, though no method is perfect. To start, describe the data your device submission needs.