Sourcing Defect Image Datasets for Visual Inspection Models

Why humanoid programs buy defect images

A humanoid at a workstation may need to inspect parts, check its own assembly, set aside damaged items before handling them, and say what is wrong. Each of those needs a model that recognizes defects on the specific parts and surfaces it will see. Defect images are perception data: they teach the robot to judge quality, not to handle the part.

This page covers where to find such images and how to specify them. The rest of the inspection, defect and failure cluster goes deeper on individual defect types and labels.

What public benchmarks reveal about the data you need

Public industrial benchmarks are built for comparing methods, and they show the shape of real defect data.

• Normal images dominate. MVTec AD contains 5,354 high-resolution images across 15 categories; its training set is defect-free only, and its test set covers more than 70 defect types with pixel-precise ground truth (Bergmann et al., 2021). VisA has 10,821 images, of which 1,200 are anomalous (Zou et al., 2022).
• Viewpoint and lighting matter. Real-IAD captures 150,000 high-resolution images of 30 objects from multiple views and allows defective images in training (Wang et al., 2024). MVTec AD 2 adds small defects in large images, defects at image borders, transparent and reflective parts, and a test split under varied lighting (Heckler-Kram et al., 2025).
• Few defects can go a long way, with the right labels. One surface-defect method learned from few defective samples (Tabernik et al., 2019), and a follow-up found that a few pixel-labeled samples, mixed with weaker labels, performed comparably to full supervision at much lower annotation cost (Božič et al., 2021).

Several of these benchmarks are licensed for non-commercial use only; MVTec AD, for example, uses CC BY-NC-SA 4.0 (Bergmann et al., 2021). A production inspection model usually needs proprietary images licensed for that purpose.

Supervised labels or anomaly detection

Supervised defect models need labeled examples of each defect class; anomaly detectors can be fit on defect-free images alone, as PatchCore does (Roth et al., 2021). The choice changes what you buy, so settle it using anomaly detection vs supervised defect data before writing the request.

Supplier types

The largest single source in many plants is the machine-vision archive; in-line machine vision archives covers how to evaluate one. When the supplier makes parts for others, read contract manufacturers as suppliers.

What to specify

• Part family and material, including surface finish, color and transparency.
• Imaging setup: camera, lens, resolution, lighting geometry and viewing angle, recorded per image.
• Taxonomy: defect classes with written definitions and example images, plus a "good" class.
• Label type: image-level verdict, bounding box or pixel mask, and which source the label comes from (machine, inspector, final disposition).
• Severity and disposition: scrap, rework, use-as-is, and the acceptance standard used.
• Normal images: enough defect-free images across lots and lines to model normal variation.
• Borderline cases: samples that inspectors disagree on, flagged rather than discarded.
• Split rules: separation by lot, line, camera or date, so the same part never appears in both training and test.

Illustrative package contents

# Illustrative only — synthetic values, not an available dataset
package: defect_images_supplier_023_v0
parts: stamped steel brackets, 3 part numbers (PN-0000-A/B/C), painted and bare
imaging: {camera: area-scan mono 12 MP, lighting: low-angle ring + dome, views: 2 per part}
images: {normal: 40,000, defective: 1,900, borderline: 260}
labels:
  verdict: final disposition from MRB records (not the in-line machine verdict)
  classes: scratch, dent, burr, missing_hole, paint_run, contamination, other
  masks: pixel masks on 600 defective images; boxes on the rest
  severity: minor / major / critical per plant acceptance standard
metadata: line_id, camera_id, lot_id, capture_date, inspector_id (pseudonymous)
splits: train lines L1-L2 (2024); test line L3 and all of 2025 Q1
excluded: customer drawings; images showing customer logos or part markings
rights: model training and internal evaluation; no redistribution; deletion at term end

Quality and acceptance criteria

1. Label audit. An independent inspector re-labels a random sample; agreement meets a threshold you set, and disagreements are reviewed. See inter-inspector agreement.
2. Verdict source. Each label states whether it came from the machine, an inspector or the final disposition.
3. Leakage. No part, lot or near-duplicate frame appears in more than one split.
4. Class counts. Counts per class per split are delivered, with rare classes identified; see rare defects and class imbalance.
5. Image integrity. Files are original resolution and bit depth, not screenshots or recompressed exports.
6. Evaluation set. A held-out set is defined by line, lot or time, as described in defect detection evaluation sets.

Limitations

• Rarity. The most important defects are often the rarest, and a single archive may hold only a handful.
• Line specificity. Models trained on one line's camera and lighting often degrade on another.
• Inherited errors. Machine verdicts encode the old system's misses and false alarms.
• Wrong viewpoint for a humanoid. Fixed inspection cameras do not see parts the way a robot's head or wrist camera will, so plan robot-view images for deployment.
• No handling data. Defect images contain no grasps, motions or contact outcomes.
• Short retention. Many systems keep images for weeks or months, which limits how far back an archive reaches.

Questions to ask suppliers

1. How long are inspection images retained, and how many are on hand today by part number?
2. Are verdicts from the machine, from inspectors, or from the final disposition after review?
3. Which defect classes exist, and are definitions and example images written down?
4. Do you keep defect-free images, or only rejects?
5. Are camera, lighting and line recorded per image?
6. Have cameras, lighting or software changed during the archive period?
7. Who designed the parts: your own engineers or the companies you make them for?
8. Can any part markings, logos or drawings be excluded before delivery?

Permissions and confidentiality

Defect images can reveal product designs, process weaknesses and customer relationships. Where parts are made to a customer's design, the customer may need to agree before images are licensed. Treat this as a diligence item from the start; confidentiality in defect data lists the questions.

Discuss your requirements

If you need defect images for a specific part family, describe the parts, defect classes and label type to SourceX. SourceX looks for US businesses that hold suitable inspection data, checks the data and the supplier's licensing permissions, and coordinates pricing, allowed uses and delivery in a license. It reviews the request and follows up about potential sources; a request does not guarantee a match.

For related records, see the catalog entry for manufacturing quality and inspection records and the page on licensing images and inspection photos. The humanoid training data guide covers the other data types.

Explore sourcing and licensing options with SourceX

Sources

• The MVTec Anomaly Detection Dataset: A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection — structure, labels and license of a standard defect benchmark.
• SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation — VisA normal-to-anomalous balance.
• Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection — multi-view capture with defects in training.
• The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection — small, border, reflective and lighting-varied cases.
• Segmentation-Based Deep-Learning Approach for Surface-Defect Detection — learning from few defective samples.
• Mixed supervision for surface-defect detection: from weakly to fully supervised learning — mixing a few pixel labels with weaker labels.
• Towards Total Recall in Industrial Anomaly Detection — anomaly detection from defect-free images.