Assembly Demonstrations with Tool Signals, Work Instructions and CAD for Robot Learning

This page sits beside our guide to human demonstration video for robot learning, which covers what video alone can teach. Here the focus is the multimodal pairing that makes assembly data trainable and evaluable. For the wider cluster, start at the multimodal and embodied data hub.

Why assembly video alone underperforms for policy learning

Video without contact signals and outcomes cannot tell a model whether an insertion seated, a bolt reached spec or a connector latched. The hard part of assembly is contact: peg-in-hole insertion, nut threading, connector mating and the handling of flexible parts such as wire harnesses and cables. Those phases last a fraction of a second, happen behind the operator's hands and look nearly identical on camera whether they succeed or fail.

Research datasets show both the value and the ceiling of vision-only capture. Assembly101 records procedural assembly and disassembly from multiple synchronized views [1], which is useful for step recognition and task planning but carries no tool or force data. Demonstrations that capture how a skilled operator detects a cross-thread, backs off and re-seats are more valuable for policy learning than clean runs, and those recovery moments are only legible when force or torque traces sit next to the video. See tactile and force-torque data for contact-rich learning for sensor-level specification.

Which modalities to pair with each assembly recording

The minimum trainable unit is a segment of video aligned to one work-instruction step, its tool or force signal, the parts involved and an outcome. Academic assembly datasets show what annotation layers are feasible: IKEA ASM pairs multi-view video with depth, atomic action labels, object segmentation and human pose [2], and ATTACH annotates two-handed assembly actions [3]. Neither includes production torque traces, part CAD or quality outcomes, which is the gap industrial buyers usually need closed.

In a real plant the useful signals typically come from systems already in place:

• DC electric nutrunner and smart-tool controllers: final torque, final angle, the torque-angle curve, the program or parameter set (Pset) number and an OK/NOK result per rundown.
• Work-instruction and MES records: operation and step IDs, revision level, station ID, serial or lot number and timestamps.
• PLM or engineering vault: part numbers, revisions and native or neutral CAD (STEP, Parasolid, JT) for the components touched. Our owner page on CAD and PCB engineering files covers the file side.
• Quality systems: end-of-line test results, defect codes and rework tickets, described further under manufacturing quality records.
• Video: fixed overhead or station cameras, or head-mounted first-person capture as covered in egocentric video of skilled manual work.

If you need line-level telemetry rather than operator demonstrations, synchronized manufacturing process data is the closer fit.

An illustrative per-step record for assembly demonstrations

A per-step record should let you join video frames, tool data, geometry and outcome without manual lookup. The structure below shows one fastening step; field names will vary by supplier and should be fixed in the license schedule before delivery.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "episode_id": "ep_000412",
  "station_id": "ST-07",
  "work_instruction": {"doc_id": "WI-3381", "revision": "C", "step": 6,
                       "text": "Torque M6 bolts 1-4 in cross pattern"},
  "parts": [{"part_number": "BRK-2210", "revision": "B",
             "cad_ref": "cad/BRK-2210_B.step", "customer_owned": true}],
  "video": {"file": "video/ep_000412_cam2.mp4", "fps": 30,
            "segment_start_s": 41.20, "segment_end_s": 58.95},
  "tool": {"type": "dc_nutrunner", "pset": 12, "target_nm": 9.5,
           "final_nm": 9.6, "final_angle_deg": 212, "result": "OK",
           "trace_file": "signals/ep_000412_tool.parquet"},
  "sync": {"clock": "PTP", "max_offset_ms": 5},
  "outcome": {"eol_test": "PASS", "rework": false, "defect_code": null},
  "privacy": {"faces_blurred": true, "operator_id": "pseudonymized"}
}

The sync block matters more than it looks. If video and controller clocks drift, the torque spike will not line up with the visible seating motion, and the pairing loses its value. Ask how clocks were synchronized and what offset was measured, then verify it yourself on a sample.

Rights, export control and worker consent checks

Assembly data carries three rights layers that pure video does not: the plant's ownership of its recordings and tool logs, the end customer's ownership of part designs, and the people on camera. A contract manufacturer may hold CAD and work instructions for parts it builds under a customer's design, and releasing that geometry for AI training usually needs the customer's authorization. Our guide on who owns robot data walks through operator, OEM and facility claims.

Part CAD, drawings and work instructions can also be export-controlled. Under the Export Administration Regulations, "technology" covers information needed for the development, production or use of an item, including technical data such as blueprints, drawings, models, engineering designs and manuals [4]. The EAR carve-out for published information applies only to material already made public without dissemination restrictions [6], which proprietary production drawings rarely are. Data for defense articles can instead be ITAR technical data [5]. Screen each part family before a cross-border transfer.

For people on camera, confirm written notice and consent from recorded operators, blurring of co-workers and badges, and pseudonymized operator IDs in tool and MES logs. Public research datasets are not a shortcut here: a 2023 audit of popular dataset hosting sites found licenses omitted more than 70% of the time and mislabeled in more than 50% of cases [7]. See open robot datasets and commercial licenses before training on them.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Buyer checklist for an assembly demonstration request

Write the request around tasks, signals and outcomes rather than around a company or a plant. A specification that names the operations, tolerances and labels lets a supplier tell quickly whether its records match.

Illustrative example: invented to show structure; it does not describe an available dataset.

Pair the checklist with our robot dataset acceptance checks and, if you are weighing a new capture program, commissioning collection versus licensing recordings.

How SourceX handles assembly demonstration requests

SourceX sources operational datasets from US companies on request, including new recordings of hands-on work, and manages the commercial process through licensing and ongoing purchases. Nothing is held in stock, and a request does not guarantee a match. You describe the data you need, not the businesses that might hold it, and SourceX looks for US businesses that do; every release is approved by the supplying company.

Each dataset is rights-reviewed for ownership and consents and delivered under a license that defines the records, allowed uses, term and delivery. Personal details such as names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. Delivery runs through private, access-controlled workflows only after an executed agreement and supplier approval. You can start a request on the SourceX buyers page, and the robotics training data use case shows related categories.

Source assembly demonstration data for your robot program

If your team needs assembly recordings paired with tool signals, work instructions, part CAD and quality outcomes, describe the tasks and fields you require. SourceX runs Find, Assess, Agree, Transact and Manage, and nothing is contracted until a supplier agrees. Describe the assembly demonstrations you need.