Short answer
Synthetic defects are a reasonable way to pretrain, augment or stress-test a visual inspection model, especially when only defect-free images are plentiful. They are a weak basis for evaluation, severity grading or rare real-world failure modes, because they reflect the generator's assumptions. Buy real defect examples for evaluation sets and for the classes where the real appearance matters.
The decision in practice
Defects are rare by design: a line that works produces few of them. That makes real examples expensive, so many teams ask whether they can generate them instead. The answer depends on what the model is for. Training a detector to notice "something is off" from clean images needs very different data from scoring whether a model catches a hairline crack on a new lot.
This page separates those uses so the buy-versus-generate choice is made per purpose. For the wider contrast between training on normal images only and training with labeled defects, see anomaly detection vs supervised defect data.
What published work reports about synthetic anomalies
Two well-known results show how far defect-free data and synthetic corruption can go on one benchmark. PatchCore fits only nominal images, storing a memory bank of patch features, and reports image-level AUROC up to 99.6% on MVTec AD (Roth et al., 2021). CutPaste trains a representation by distinguishing normal images from ones with a patch cut and pasted at a random location, then fits a one-class classifier; on MVTec AD it reports 96.6 image-level AUC with ImageNet-pretrained transfer and 96.0 pixel-level localization AUC with a patch-based extension (Li et al., 2021).
Read these results narrowly. They describe one benchmark of controlled, curated images, and the pasted patches are a pretext task, not a model of real defects. They suggest nominal-only and synthetic-corruption approaches can be strong baselines on such data, and nothing more. Harder settings exist: MVTec AD 2 adds small defects in large images, transparent or reflective objects, and a private test set captured under varied lighting and served via an evaluation server (Heckler-Kram et al., 2025).
The same pattern appears in robotics. AHA uses FailGen to perturb successful simulated demonstrations into failure trajectories, and the resulting model generalizes to real-world failure datasets (Duan et al., 2024). Synthetic failures extended training; real failure data was still how generalization was checked.
Decision table by purpose
| Purpose | Synthetic defects | Real defects | Practical guidance |
|---|---|---|---|
| Pretraining a feature extractor | Often fine | Optional | Generate or use clean images |
| Training a "normal vs not normal" detector | Often sufficient as a pretext | Helpful for tuning | Start with clean images; add real samples to calibrate |
| Training a supervised defect classifier | Augmentation only | Required for each class | Use synthetic to balance, not to define classes |
| Pixel-level segmentation | Possible with generated masks | Needed to validate masks | Real-image check on edges and faint defects |
| Severity grading and disposition | Weak | Required | Grades depend on real appearance and standards |
| Evaluation and acceptance testing | Not suitable | Required | Held out, real, captured on the target line |
| Rare, safety-critical defects | Stress tests only | Required where obtainable | Consider physical samples; see below |
| Manipulation failure detection | Useful for coverage | Required for validation | Check on robot-recorded failures |
Where synthetic data goes wrong
- The generator's world is the only world. A model trained on cut-and-paste or rendered cracks learns what those look like, not the mix of porosity, scratches and discoloration real lots produce.
- Artifacts become shortcuts. Seams, color shifts and texture mismatch around pasted regions can be easier to detect than the defect itself. A model may learn the artifact and then miss real defects.
- Realism is not verified. Without real examples to compare against, nothing shows whether synthetic defects look like production defects.
- Severity is invented. Real severity depends on size, location, standard and customer, which a generator can only guess.
When real defects are worth buying
- The evaluation set. Even if training is largely synthetic, the test set should be real, held out, and sampled to reflect the failure modes you care about; see defect detection evaluation sets.
- Rare classes where an error is costly. Rare defects and class imbalance covers how to find and weight them.
- Borderline cases near the acceptance line, where labeling judgment matters.
- Domain-specific materials and lighting, including transparent and reflective parts.
- Real samples for rendering: physical defect samples can seed both imaging and simulation.
Real defect images can be sourced from existing quality archives; see sourcing defect image datasets.
Illustrative hybrid plan
# Illustrative only - proportions and sources are examples a buyer would set
training:
nominal_images: from line captures, many lots
synthetic_defects: generator v2, flagged synthetic=true in metadata
real_defects: all available, labeled, never regenerated from synthetic
evaluation:
real_only: true
split_by: lot, line, capture_week
includes: borderline_samples, rare_classes, varied_lighting
audit:
synthetic_realism_check: expert review of 200 random synthetic patches
artifact_check: classifier trained to separate synthetic from real must stay near chance
Limitations: what neither option can show
Synthetic and real images alike show appearance only. Neither says whether a defect matters to the customer, how a robot should handle a rejected part, or what a manipulator should do once a problem is found. Real defect images from quality archives also reflect past inspection decisions and past camera setups, so they inherit the labeling errors and blind spots of the process that created them.
Also remember that benchmark results do not describe your line. The curated images behind published numbers are can be cleaner than production captures, and performance on them does not by itself predict yours.
Quality gates for any synthetic content you accept
Require a flag on every synthetic image, a record of the generator version and settings, and the source images each defect was placed on. Ask for a sample you can inspect pixel by pixel at the defect boundary, since blending seams are the commonest give-away. Run a quick classifier that tries to tell synthetic from real; if it succeeds easily, a detector may do the same and learn the seam rather than the defect. Keep synthetic images out of the evaluation split, and report results separately for real-only and mixed training, so the contribution of generated data is visible rather than assumed.
Questions to ask suppliers
- Are any images synthetic, augmented or inpainted, and are they flagged per image?
- If synthetic defects are included, what generator, parameters and source images were used?
- Which defect examples are real production defects, and from how many lots and lines?
- Is the real/synthetic ratio per class documented?
- Can you show an expert review of synthetic realism and a check for generation artifacts?
- Are evaluation images strictly real and held out from the generator's source images?
- How were defect masks produced, and which were manually checked?
- Are borderline and rare defect examples present, and how were they selected?
- What license applies to generated content, and does it depend on a third-party model or dataset?
- Can I obtain physical defective samples to test the imaging setup?
Discuss your requirements
If you have a defect list and know which parts must be real, tell SourceX what your evaluation set needs to cover. SourceX looks for US businesses that hold suitable real defect material, checks the data and licensing permissions, and coordinates the license and delivery. A request does not guarantee a match. For the licensing side of generated and collected data, see licensed vs synthetic vs scraped data.
Start from the humanoid training data guide, the inspection, defect and failure data hub, and the synthetic renders vs real images comparison on the object side. The robot manipulation failure datasets page covers the robotics analogue.
Tell SourceX what your evaluation set needs to cover
Sources
- CutPaste: Self-Supervised Learning for Anomaly Detection and Localization — synthetic cut-and-paste anomalies as a pretext task.
- Towards Total Recall in Industrial Anomaly Detection — nominal-only anomaly detection results.
- AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation — procedurally generated failure trajectories.
- The MVTec AD 2 Dataset: Advanced Scenarios for Unsupervised Anomaly Detection — harder scenarios and a private test set.