Multimodal and embodied data
Synchronized Manufacturing Data: Line Video, Machine Signals and Quality Outcomes
Quick answer
A multimodal manufacturing dataset is useful only when every stream from one cell lands on a shared clock and joins to a per-part outcome. That means camera frames with exposure metadata, PLC tags at known sampling rates, vibration or acoustic channels, and the MES pass/fail or measurement result, all keyed by part serial, station ID and synchronized timestamps. Public benchmarks rarely provide this because most use simulated or induced faults, so production records from operating plants are the practical source.
By SourceX Editorial · Updated
Why public industrial datasets fall short for process models
Public industrial datasets mostly isolate one modality and stage the failures, which leaves no link between process conditions and real quality outcomes. MIMII recorded valves, pumps, fans and slide rails with anomalies such as leakage and rotating unbalance created as simulated faults [1], and the DCASE 2020 anomalous-sound task included toy machines with intentionally induced damage [2]. The Tennessee Eastman fault benchmark is a simulated chemical plant [6], and DAGM 2007, a standard defect-image set, is synthetic with at most one defect per image [5].
Image benchmarks also saturate. The Real-IAD authors note that leading methods already exceed 99% AUROC on MVTec AD, so the benchmark no longer separates them [4]. None of these sets tells you which spindle load, mold temperature or torque curve preceded a scrapped part, and that join is what process anomaly detection, root-cause analysis and physical-AI world models need.
Distribution shift is the other gap. MIMII DUE shows that sound-based detectors can fail when operating speed or background noise changes between training and deployment [3]. Production data from several shifts, product variants and maintenance states is the most direct way to measure that shift before a model reaches the line.
The streams to specify, and the metadata that makes them usable
Name each stream with its capture system, rate and metadata, because a stream without its metadata cannot be aligned or interpreted. Most buyers need four layers from one cell, and the request should say which are mandatory and which are nice to have.
- Vision: machine-vision or line-camera frames, with frame timestamp, exposure time, gain, lighting program and camera pose. For inspection stations, include the vision system's own verdict and the region of interest it used.
- Control and process signals: PLC tags and historian points, each with tag name, engineering unit, sampling or change-based logging rule, and deadband. CNC cells may expose data items through MTConnect's information model [8]; OPC UA servers add alarm and condition states [7] that matter for labeling events.
- Acoustic and vibration: accelerometer or microphone channels with sample rate, mounting location and sensor model. These overlap with the audio cluster, so ask only for channels tied to the same cell timeline.
- Outcomes: the MES or QMS record per part: pass/fail, defect code, measured dimensions, rework and scrap disposition, and the inspection timestamp. ISA-95 (IEC 62264-1) gives a shared vocabulary for how these operations records relate to control-level data [9].
For the defect-image side alone, see inspection photo licensing; for standalone telemetry, see sensor and IoT data. This page covers the case where they must arrive joined.
Join keys and clock alignment: where these datasets break
The join is the product, and it most often fails on clocks and part identity rather than on any single stream. Cameras, PLCs, data loggers and the MES frequently run on separate clocks; a historian may log on change while the camera fires on a trigger. Ask the supplier how clocks are disciplined, what offset was measured, and whether timestamps are capture time or ingest time. The time synchronization guide covers verification methods in depth.
Part identity breaks next. Serial numbers may be assigned after the station you care about, parts can be reworked and re-enter the line, and batch processes may only carry a lot ID. A usable dataset states the join key at each station and documents how rework loops are represented.
Illustrative example: invented to show structure; it does not describe an available dataset.
| Field | Example value | Why it matters |
|---|---|---|
| part_serial | P-0412-88213 | Primary join key to the MES outcome |
| station_id | WELD-03 | Separates cells running the same recipe |
| cycle_start_utc / cycle_end_utc | 2026-03-02T14:11:07.120Z / 14:11:52.480Z | Windows every stream for one part |
| frames_ref | s3 prefix, 30 fps, exposure_us per frame | Links video without embedding it |
| plc_tags_ref | Parquet, 12 tags, 100 Hz or on-change | States the logging rule explicitly |
| vibration_ref | WAV, 25.6 kHz, sensor on spindle housing | Ties acoustic data to the cycle |
| clock_offset_ms | camera +4, PLC 0, MES +210 | Measured offsets, not assumed |
| quality_result | FAIL, defect_code POROSITY, rework=1 | Outcome label with disposition |
| recipe_id | hashed REC-7f3a | Trade-secret recipe kept as a token |
Store bulk streams as references (object-store prefixes) and keep a cycle-level index in Parquet or similar; a format and transfer plan is in the delivery guide.
Label quality: outcomes are not ground truth by default
Quality outcomes are operational decisions, not curated labels, so treat them as noisy until checked. Inspectors disagree, end-of-line tests have false accepts, and a part scrapped for a handling scratch says nothing about the weld process. Even curated benchmark test sets carry an estimated average label error rate of at least 3.3% [10].
Ask for the defect-code taxonomy and its revision history, the measurement system analysis (gauge R&R) where it exists, and how rework and customer returns feed back. ISO/IEC 5259-4 frames this as a data quality process covering labeling for supervised ML [12]. For evaluation, hold out whole production weeks or product variants rather than random cycles, which leak recipe and tool-wear state across splits; the private evaluation sets guide explains held-out design.
Rights, trade secrets and people in frame
Manufacturing cell data mixes three sensitive layers: the plant's process know-how, customers' part designs, and workers on camera. Process recipes, setpoints and cycle-time data can be trade secrets, and part geometry may belong to the plant's customer under a supply agreement, so field-of-use limits or tokenized recipe IDs are common asks. Licensing across several rightsholders is covered in the multimodal licensing guide.
Operators often appear in line video. Texas defines a record of face geometry as a biometric identifier and requires notice and consent before commercial capture [11], and Illinois BIPA applies similar rules; see the biometric data guide. Ask whether workers received notice, whether faces are blurred or cropped, and whether badge numbers or operator IDs in PLC or MES logs are replaced with tokens. The multimodal de-identification guide covers faces, voices and metadata together.
This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.
Acceptance checks before you commit to a supply
Run acceptance checks on a defined slice before scaling, because alignment and coverage problems are cheap to find early and expensive later.
- Re-derive clock offsets on 50 or more cycles by matching a visible event (clamp close, weld arc) across video and PLC signals.
- Confirm every cycle in the index joins to exactly one MES outcome, and count orphans on both sides.
- Check class balance: how many FAIL cycles exist per defect code, and from how many distinct root causes.
- Verify coverage across shifts, variants, tool changes and maintenance events.
- Inspect a sample of frames and logs for faces, names and operator IDs.
- Confirm the license states records, allowed uses, term and delivery.
Related checklists for physical data appear in robot dataset acceptance checks, and the multimodal cluster hub lists adjacent topics such as assembly video datasets.
How SourceX handles manufacturing data requests
SourceX sources operational datasets from US companies on request and manages the commercial process, including licensing and ongoing purchases; nothing is held in stock and a request does not guarantee a match. You describe the data, such as the cell streams and outcomes above, and SourceX looks for US businesses that hold it; every release is approved by the supplying company. Each dataset is rights-reviewed for ownership and consents, personal details are removed or replaced before delivery with the method recorded, and delivery runs through private, access-controlled workflows after an executed agreement. Start a request at the SourceX buyer page, or see manufacturing quality records and manufacturing buyers.
Request synchronized manufacturing cell data
Describe the cell, the streams, the outcome labels and the intended use, and SourceX will look for US businesses that hold it and assess data and licensing permissions. Pricing and allowed uses are agreed per deal in a license, and nothing is contracted until a supplier agrees. Submit your manufacturing data request.
Sources
- arXiv (Purohit et al.), "MIMII Dataset: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection" (2019). https://arxiv.org/pdf/1909.09347
- DCASE Community, "DCASE 2020 Challenge Task 2: Unsupervised Detection of Anomalous Sounds for Machine Condition Monitoring" (2020). https://dcase.community/challenge2020/task-unsupervised-detection-of-anomalous-sounds
- arXiv (Tanabe et al.), "MIMII DUE: Sound Dataset for Malfunctioning Industrial Machine Investigation and Inspection with Domain Shifts" (2021). https://arxiv.org/pdf/2105.02702
- arXiv (CVPR 2024), "Real-IAD: A Real-World Multi-View Dataset for Benchmarking Versatile Industrial Anomaly Detection" (2024). https://arxiv.org/pdf/2403.12580
- arXiv, "A Review of Benchmarks for Visual Defect Detection in the Manufacturing Industry" (2023). https://arxiv.org/pdf/2305.13261
- IEEE DataPort, "Tennessee Eastman Process Dataset". https://ieee-dataport.org/documents/tennessee-eastman-process-dataset
- OPC Foundation, "OPC 10000-9 Alarms and Conditions, 4.8 Alarms". https://reference.opcfoundation.org/specs/OPC-10000-9/4.8
- MTConnect Institute, "MTConnect Standard Part 2.0". https://docs.mtconnect.org/MBSD_MTConnect_Part_2_2_0_0.pdf
- IEC, "IEC 62264-1 Enterprise-control system integration - Part 1: Models and terminology". https://webstore.iec.ch/en/publication/6675
- arXiv (Northcutt, Athalye, Mueller), "Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks" (2021). https://arxiv.org/abs/2103.14749
- Texas Legislature, "Texas Business and Commerce Code Section 503.001 - Capture or Use of Biometric Identifier". https://statutes.capitol.texas.gov/Docs/BC/htm/BC.503.htm
- ISO/IEC, "ISO/IEC 5259-4:2024 Data quality for analytics and machine learning - Part 4: Data quality process framework" (2024). https://www.iso.org/standard/81093.html
Tell us what your models need
Share scope, volume, language, format, timing and licensing requirements.