Short answer
Illustrated assembly manuals pair step-by-step diagrams with parts lists, which makes them useful for learning assembly plans: which part is added, where, and in what order. Published work converts manual images into machine-readable plans. The manuals contain no motion, force or grasp data, and the main sourcing questions are rights: who holds copyright in the manuals and what dealer agreements allow.
Why diagram manuals differ from other instructions
A typical SOP is prose written for a trained worker. An assembly manual is the opposite: mostly pictures, few words, written for a customer with no training. Each page shows a sub-assembly, arrows for insertion direction, callouts for part IDs and fastener counts, and sometimes a warning icon. The structure is already close to a plan: a sequence of steps, each adding named parts in a stated relationship.
That structure is what research has exploited. It is also why this page is separate from the broader guidance on visual work instructions and product user guides as instruction data: OEM-published manuals are public-facing documents with a different rights position from the internal instructions a business writes for its own staff.
What published work does with assembly manuals
IKEA-Manual pairs 102 IKEA objects with their assembly manuals and annotates decomposed assembly parts, assembly plans, manual segmentation and 2D-3D correspondence between manual images and the object (Wang et al., 2022). The point of the annotation set is to let a model relate what a diagram shows to the 3D parts and the order they are joined.
A second study, MEPNet, takes a sequence of LEGO manual images and outputs the assembly steps, meaning which parts to add and their 3D poses at each step. It combines 2D keypoint detection with 2D-3D projection, and was evaluated on three newly collected LEGO manual datasets and a Minecraft house dataset (Wang et al., 2022).
These are results about reading manuals and producing plans, not about a robot physically assembling anything, and a buyer should not read them as the latter. What they establish is that manual pages can be turned into structured step-and-pose sequences, which is a planning-level input. How a plan becomes grasps and insertions is a separate problem; see task graphs from procedures.
Where this data fits and where it does not
| Use | Manuals can contribute | Still needed |
|---|---|---|
| Assembly sequence learning | Step order, part additions, sub-assembly structure | Alternative valid orders, which manuals rarely show |
| Part and step grounding | Links between diagram regions and 3D parts | Real-world appearance of parts, not line art |
| Language grounding | Part names, fastener types, warnings | Colloquial phrasing used by workers |
| Plan checking | A reference sequence to compare against | Physical feasibility for a given robot |
| Physical control | Nothing directly | Robot or teleoperated demonstrations, force data, calibrated sensors |
What a buyer can build from a manual set
A parsed manual set can serve three distinct jobs, and the preparation differs for each. For plan learning, the target is a graph: nodes are steps, edges are ordering constraints, and each node lists parts added and a pose hint. For grounding, the target is a correspondence between regions of a diagram and named parts, which the IKEA-Manual annotations of 2D-3D correspondence illustrate. For evaluation, the target is a held-out set of products whose manuals and parts were never seen in training, so that a plan-reading claim is not a memorization claim.
Hold out by product family, not by page. Manuals for variants of one product share diagrams almost verbatim, and a random page split would leak. Also decide early whether you need multiple languages, and ask whether the OEM publishes one diagram set with text in several languages and whether the translated text is a separate rights item from the art.
Limitations and additional preparation
Manuals show the intended end state of each step, not how a person gets there. A diagram of a dowel entering a hole says nothing about alignment tolerance, the force needed, or what to do when it jams. They also omit tacit knowledge: holding a panel steady, rotating a half-built frame, using a second person.
Rendering style creates a domain gap. Line drawings and exploded views look unlike camera images, so models trained only on them need paired real images or video of the same assembly. A manual also covers the happy path: no failures, no wrong-part errors, no recovery.
Before use, manuals need parsing. Scanned PDFs need page and step segmentation, part-callout detection and OCR of multilingual text; see parsing scanned manuals. Parts lists should be linked to CAD or catalog identifiers where the OEM can supply them, which connects this data to manufacturer 3D models and CAD. If the manual references a procedure owned by someone other than the OEM, see who owns a procedure.
Who holds these manuals
| Supplier type | Typical holdings | Rights question |
|---|---|---|
| OEMs and product manufacturers | Current and legacy manuals, source illustrations, translations | Copyright in text and drawings; whether illustrators or agencies hold any rights |
| Authorized dealers and service networks | Copies distributed under dealer agreements, repair bulletins | Whether the dealer agreement allows any use beyond servicing |
| Technical-documentation vendors | Structured source files (XML, vector art) | Customer ownership of the content they authored |
| Public archives and community sites | Scanned or user-uploaded manuals | Provenance; uploader may not own the content |
Source files can be more useful than PDFs. Vector art with layer names and part IDs lets you extract structure that OCR would have to guess. An OEM may be willing to supply structure without all of its prior content, or may only license a defined product range.
Rights and provenance diligence
Ask whether the manuals are protected by copyright and whether the OEM licensed any images from outside illustrators. A dealer holding a manual in a service portal is not necessarily able to license it for AI training. These are diligence questions rather than conclusions, and this is not legal advice. Ask who owns the text, drawings and part numbers, whether translation vendors hold any rights, whether any use of the material for training is restricted by contract, and whether trademarks or safety warnings are included. The licensing terms guide for robotics training data lists clause topics.
Illustrative package
# Illustrative only - fields and values are examples, not an available dataset
package: oem_assembly_manuals_v0
scope: 40 flat-pack furniture products, current model year
assets:
manual_pdf: vector PDF, original page order
source_art: SVG per step where the OEM holds it
parts_list: CSV (part_id, name, quantity, fastener_type)
cad_link: product_id -> CAD file reference (optional)
annotations:
step_segmentation: page -> step boundaries
step_graph: step_id -> parts_added, pose_hint, prerequisites
language: step text in source language plus one translation
metadata: product_id, revision_date, language, illustrator_credit
excluded: motion, force, grasp data, video of real assembly
rights: training and evaluation; no redistribution; revision list on request
Questions to ask suppliers
- Who owns the text, the illustrations and the part numbering, and are any elements licensed from third parties?
- Do you hold source files (vector art, structured XML) or only final PDFs?
- Which product models and revision dates does the set cover, and which are retired?
- Are step order and part IDs consistent across the manuals, or does each product line use its own conventions?
- Can parts lists be linked to CAD or catalog identifiers?
- Are translations included, and who holds the rights to them?
- Does any dealer or distributor agreement restrict how manuals may be used or sublicensed?
- Are there companion materials such as assembly videos or service bulletins, and are they covered by the same terms?
- How will revisions be handled if a manual is corrected or recalled?
Explore sourcing and licensing options
If you need diagram-based instruction data for assembly planning, describe the product types, formats and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match; SourceX follows up about potential sources. Start from the humanoid training data guide or the procedures hub, and see the site's page on internal documentation.
Explore sourcing and licensing options with SourceX
Sources
- IKEA-Manual: Seeing Shape Assembly Step by Step — manuals paired with parts, plans and 2D-3D correspondence for 102 objects.
- Translating a Visual LEGO Manual to a Machine-Executable Plan — converting manual images into steps and 3D poses.