Short answer
Assembly verification data shows assemblies labeled against their intended configuration: which components should be present, in which position, orientation and variant, and whether each one is. Manufacturers hold it at end-of-line vision stations, error-proofing checks and audit photos. It can train a humanoid to check its own or a person's assembly; it cannot show whether a hidden connector is seated or how to correct the error.
Why configuration errors need their own data
A scratch is a local change in appearance. A missing clip is not: the surface where the clip should be may look perfectly normal, and the error only exists relative to what the assembly was supposed to contain. MVTec LOCO makes this split explicit, separating structural anomalies such as scratches, dents and contamination from logical anomalies, which are violations of constraints such as objects that are missing or in invalid locations (Bergmann et al., 2022). Benchmarks built around local defect types, such as MVTec AD (Bergmann et al., 2021), are not designed to exercise this kind of reasoning.
For a humanoid that assembles, kits or packs, a configuration check is what it runs after every step and before handing work on. That makes assembly verification data both a training asset and a natural test set. Surface defects are handled in the guide to sourcing defect image datasets.
Error types to cover
| Error type | Example | What reveals it in an image | Label it needs |
|---|---|---|---|
| Missing component | Clip, screw, gasket, label absent | Empty slot at a known location | Expected vs observed per slot |
| Wrong variant | Wrong-color cap, wrong-length screw, wrong resistor value | Fine detail, color or markings | Variant ID from the build record |
| Wrong position | Part fitted in the adjacent slot | Location relative to a datum | Slot ID and observed position |
| Wrong orientation | Connector reversed, bracket flipped | Asymmetric features | Expected and observed orientation |
| Wrong count | Extra washer, doubled clip | Counting | Expected and observed count |
| Incomplete action | Connector not fully seated, latch open, screw proud | Small gaps or height differences, often barely visible | Visual state plus the result of any functional test |
| Foreign object | Loose screw or offcut left inside a housing | Anything not in the reference | Box or mask, flagged as foreign |
Ask suppliers which of these their checks actually cover. Most vision stations inspect a fixed list of regions, so errors outside those regions are absent from the archive no matter how large it is.
The reference configuration is half the dataset
A label of "fail" is not useful unless you know what the assembly should have been. Ask for the expected configuration to travel with every image: bill-of-materials revision, option or variant code for that serial, and the slot list the station checks. Without it, a correct assembly of a rare variant looks like an error.
Template comparison is a common pattern. DeepPCB pairs each of 1,500 test images with a defect-free template and annotates six defect types (Tang et al., 2019); assembly stations may store a golden image per variant in the same way. Golden samples and reference images covers how to source and version them.
Acceptance criteria also define what counts as an error. In electronics, IPC-A-610 sets visual acceptance criteria for assemblies, including component placement, with three product classes of increasing strictness (IPC, 2024). A slightly offset part can pass in one class and fail in another, so record which standard and class applied.
Video adds the time dimension. IndustReal benchmarks assembly state detection alongside procedure step recognition, with omissions and execution errors included (Schoonbeek et al., 2023). If you need to know which step caused the error, see human procedural mistake videos.
Supplier types
| Supplier | Typical asset | Strength | Watch for |
|---|---|---|---|
| Manufacturers with end-of-line vision | Images with pass/fail per inspection region | Large volumes, consistent framing | Only checked regions covered; verdicts tied to a recipe |
| Electronics assemblers with automated optical inspection | Component presence, polarity and offset calls | Per-component labels | Boards designed by customers |
| Lines with error-proofing | Pick-to-light confirmations, fastener counts, sensor checks | Ground truth for what was fitted | Often no images; see error-proofing records |
| Quality audit teams | Photos of finished units against an audit checklist | Careful, adjudicated labels | Small volumes |
| Rework stations | Images of nonconforming assemblies and the fix applied | Real errors with causes | Biased toward errors that reached rework |
Many plants combine these: the vision station flags, an operator confirms, rework records the cause. Joined records are worth more than any one source; inspection images linked to outcomes explains the join.
What to specify
- Products and variants in scope, and how the variant is identified for each image.
- Expected configuration source per image: BOM revision, option code, golden image version.
- Slot list with stable IDs, so labels refer to slots rather than free text.
- Label granularity: per-slot status from a closed set (present, absent, wrong variant, misoriented, not seated, foreign object).
- Views: number of cameras or angles, and which slots each view can see.
- Error provenance: production error, operator-induced test, or staged error, flagged per image.
- Split key: serial, lot and date; hold out a whole variant if you need to test generalization to new builds.
Illustrative verification record
# Illustrative only — synthetic identifiers and values
image_id: eol_cam2_000731
station: eol_vision_02, recipe R-VAR-B3, view: top
serial: SN-0000-55120
variant: B3 (option code OPT-17)
bom_revision: rev F
reference_image: golden/B3_top_v4.png
slots:
- {slot: S01_clip_left, expected: clip_A, observed: clip_A, status: present}
- {slot: S02_clip_right, expected: clip_A, observed: none, status: absent}
- {slot: S07_connector, expected: conn_6pin, observed: conn_6pin, status: not_seated_suspected}
- {slot: S09_screw_M4x12, expected: M4x12, observed: M4x16, status: wrong_variant}
station_verdict: fail
operator_confirmation: fail (S02, S09); S07 passed on re-check
functional_test: pass
error_provenance: production
split_key: lot_2207
The record shows why station verdicts and confirmed labels must both be kept: the suspected unseated connector in this illustration turned out to be acceptable.
What this data can and cannot support
Can support: training and testing presence, variant, orientation and count checks; logical anomaly detection (see anomaly detection vs supervised defect data); step-completion checks a humanoid runs on its own work; error examples for models that explain what is wrong.
Cannot support: states that are hidden or barely visible, such as whether a connector latched or a screw reached torque, unless functional test or tool data is joined; the actions needed to correct the error; the robot's own viewpoint, since station cameras are fixed and lit for the line. Seating and fastening are contact problems; when video is not enough covers the extra data they need. Station images also only cover the variants built during the archive period.
Assembly order itself, as opposed to the finished state, is covered in assembly sequence data.
Quality and acceptance criteria
- Every image carries its variant, BOM revision and reference image version.
- Per-slot labels use the agreed closed set; free-text reasons are mapped or flagged.
- A sample of station fails has been confirmed by a person, with the confirmation rate reported.
- Staged errors are flagged and kept out of any production-error test set.
- Counts per error type and slot are reported per split, with rare combinations identified.
Questions to ask suppliers
- Which slots or regions does each station check, and which components are never inspected?
- Is the expected configuration stored per serial, or only implied by the station recipe?
- Are operator confirmations or overrides recorded against station verdicts?
- How are wrong-variant errors detected: by appearance, markings or a scan?
- Do you keep passing images, or only fails?
- Are functional test results linkable to the same serial?
- Have recipes, cameras or variants changed during the archive period?
- Who owns the product design shown: your engineering team or a customer?
Permissions and confidentiality
Assembly images reveal product structure, variants and sometimes unreleased designs. Where you build to a customer's design, the customer may need to approve licensing. Treat this as a diligence item from the start; confidentiality in defect data lists the questions. Public logical-anomaly benchmarks can carry non-commercial terms; MVTec LOCO, for example, uses CC BY-NC-SA 4.0 (Bergmann et al., 2022).
Discuss your requirements
If your humanoid must verify assemblies, describe the products, error types and labels you need to SourceX. SourceX looks for US businesses that hold suitable inspection records, checks the data and the supplier's licensing permissions, and coordinates pricing, allowed uses and delivery in a license. It reviews each request and follows up about potential sources; a request does not guarantee a match. See the inspection, defect and failure hub, the humanoid training data guide, and the catalog entries for manufacturing quality and inspection records and images and inspection photos.
Explore sourcing and licensing options with SourceX
Sources
- Beyond Dents and Scratches: Logical Constraints in Unsupervised Anomaly Detection and Localization (MVTec LOCO AD) — structural vs logical anomalies and license terms.
- The MVTec Anomaly Detection Dataset — a benchmark built around local defect types.
- Online PCB Defect Detector On A New PCB Defect Dataset — template and test image pairs.
- IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos — assembly state detection with procedural errors.
- IPC-A-610 Acceptability of Electronic Assemblies — visual acceptance criteria and product classes for electronic assemblies.