Short answer
Where a plant inspects by sampling, only some units in each lot are examined, so an inspection archive is a biased slice of production, not a census. A lot-level accept or reject decision is also not a per-image defect label. Buyers should ask how samples were drawn, what was recorded per unit versus per lot, and which units were ever imaged.
The question behind this page
A buyer looking at a manufacturer's inspection archive sees images, defect codes and pass/fail outcomes, and tends to read them as a description of production. Often they are not. Some plants decide whether to accept a lot by inspecting a sample of it, using a sampling plan, rather than examining every unit. In that case the archive reflects which units were drawn, how they were examined and what was written down, not the full set of units that were made.
This matters for defect-image buying because it explains why the data looks the way it does: why defects appear rarer or more common than expected, why some lots have many images and others none, and why a "reject" can sit beside images that all look fine. For sourcing and class balance in general see sourcing defect image datasets and rare defects and class imbalance.
What the standard describes
ISO 2859-1 is one reference a buyer may hear named. It describes an acceptance sampling system of single, double and multiple sampling plans for inspection by attributes, indexed by acceptance quality limit (AQL) for lot-by-lot inspection. It applies to finished products, components, materials, records and procedures. National standards-body listings give ISO 2859-1:2026 as the current edition, published in January 2026, following the 1999 edition and its 2011 amendment (ISO, 2026); the linked ISO page has not been confirmed to show that edition, so confirm which edition a supplier's plans refer to.
This page does not reproduce plan tables or tell you which plan a plant uses, and nothing here is a statement about what the standard requires of a particular plant. A manufacturer may use ISO 2859-1, a customer-specified plan, an internal procedure or no sampling at all. The standard's text is a published document with its own terms, so a supplier handing it over is also a license question. This is not legal advice.
How sampling shapes the archive
| Sampling feature | Likely effect on inspection data | What the buyer should ask |
|---|---|---|
| Only a sample of each lot is examined | Most units have no inspection record or image | Which units were sampled, and were non-sampled units ever imaged elsewhere? |
| Lot accepted or rejected as a whole | Decision applies to a lot, not each unit | Is a per-unit result stored, or only the lot disposition? |
| Sample size may vary with lot size and plan | Image counts may vary by lot, not by risk | Were lot sizes and plan identifiers kept? |
| Plan sets an acceptance criterion for the sample | Counts may be recorded, images may not | Were defective units photographed or only tallied? |
| Double or multiple sampling | Additional units may be drawn in later stages | Are second-stage units flagged, and is that bias recorded? |
| 100% screening on some characteristics | Different record type for those checks | Which characteristics were screened, and by what method? |
A buyer who ignores these effects may mistake the sampled set for the population, and train or evaluate a model on a defect rate that does not describe production.
Lot disposition is not an image label
Suppose a lot is rejected because the sample contained more nonconforming units than the plan allows. The record says "reject." It does not say which images show the nonconformity, how severe it was, or whether the other sampled units were clean. If only the lot disposition is stored, assigning that label to every image from the lot would mislabel the clean ones.
Link outcomes to units wherever you can. The pattern is discussed in inspection images linked to outcomes, the label schema in defect taxonomy and label schema, and the definitions that make a "defect" mean something are in acceptance standards as label definitions. Written reports of nonconformity may add per-unit detail; see nonconformance reports.
Where sampling creates leakage and bias
- Same-lot leakage. Images from one lot share lighting, material batch and operator. If lots are split across training and test sets, the model may learn the lot rather than the defect. See leakage in defect datasets.
- Selection bias. Inspectors who choose samples by eye, rather than by a random draw, may pick suspicious units. The archive then over-represents defects.
- Survivorship. Rejected lots may be reworked, scrapped or returned, and their records stored elsewhere.
- Silent gaps. Units never sampled have no label. Absence of a defect record can mean "not examined" rather than "examined and clean."
- Rate drift. If a plant changes its plan or sample sizes over time, sampling intensity shifts without any change in the product.
Held-out evaluation sets should be separated by lot, line, shift or time as your claim requires, not by random image; the same principle applies to every inspection set.
Illustrative record map
This Illustrative record map uses synthetic identifiers. It shows what a buyer might ask a plant to supply so that sampling can be understood.
# Illustrative only - not an available dataset
inspection_record_map_v0:
lot:
lot_id: lot_00871
lot_size: recorded
plan_id: plan_ref_A # internal or standard-based; supplier names it
plan_type: single | double | multiple
disposition: accept | reject | rework
sample_drawn_by: random_procedure | inspector_choice | unknown
unit_in_sample:
unit_id: u_0004412
image_ids: [img_221, img_222]
characteristics_checked: [surface, dimension_A]
result: conforming | nonconforming | not_recorded
defect_codes: from supplier taxonomy
unsampled_units: count only, no records
linked_documents: [nonconformance_report_id or null]
If the supplier cannot fill the sample_drawn_by and plan_id fields, assume the sample is not statistically understood and treat any rate estimates with caution.
What this context cannot support
Sampling context does not turn an archive into a representative dataset. It also does not create training data on its own: the standard describes sampling schemes, and a plan document contains no images, labels or defect examples. At best it supplies vocabulary for acceptance definitions and for describing how the data was selected. A buyer still needs images with defect labels, and for a humanoid that inspects or handles parts, robot or teleoperation data for the handling; inspection images and records do not train control.
Records from sampling inspection cannot estimate how often defects occur in unsampled units, and they should not be used to claim a defect rate for production as a whole without the plan, lot sizes and selection method. Treat any such claim from a supplier as something to verify.
Questions to ask suppliers
- Which characteristics are inspected by sampling and which by full screening?
- What sampling plans or procedures do you use, and are plan identifiers stored per lot?
- How are sample units chosen, by random draw or by an inspector's selection?
- Are results stored per unit, per lot, or both?
- Were defective units photographed, or only counted?
- Are images taken for every sampled unit or only for nonconforming ones?
- What happens to units from rejected lots, and are their records kept?
- Has the plan changed over time, and are changes dated?
- Can you share lot sizes and sampling parameters without disclosing customer or product details?
- Who owns the plan documents, and are any based on customer specifications you cannot license?
Tell SourceX what your evaluation set needs
If you need inspection images with their sampling context, describe the products, defect types and permitted uses to SourceX. SourceX looks for US businesses that may hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match; SourceX reviews the requirement and follows up about potential sources. Start at the humanoid training data guide or the inspection, defect and failure data hub. The catalog pages on inspection reports and manufacturing quality records describe the related categories.
Tell SourceX what your evaluation set needs to cover
Sources
- ISO 2859-1 Sampling procedures for inspection by attributes - Part 1: Sampling schemes indexed by acceptance quality limit (AQL) for lot-by-lot inspection — description of the sampling system; edition details to be confirmed.