Short answer
Visual work instructions are station-level procedures in which each step pairs short text with a photo or diagram, often marked with arrows, callouts and good-versus-bad examples. Manufacturers and service operators author them for training, so they are ready-made image-text pairs for grounding objects, parts, states and step order. They are still images taken for people, so they carry no motion, force or robot viewpoint.
Why they are a distinct asset
Text-only procedures name objects without showing them. A visual work instruction shows the exact part, location and finished state for each step, photographed or drawn by an engineer who knew which detail mattered. The arrow on the photo points at the connector to seat; the circle surrounds the clip the harness must pass behind. That is a weak region label that someone has already paid for.
Many visual instructions also include "correct" and "incorrect" photos side by side: a pinched cable next to a properly routed one, a label applied crooked next to one applied level. Labeled examples of wrong states are scarce in many corpora, and they are exactly what success detection needs. How visual instructions differ from SOPs and checklists as document types is covered on SOPs vs work instructions vs checklists.
What a step usually contains
| Element | Illustrative example | Grounding use |
|---|---|---|
| Step text | "Route the harness behind clip C3." | Step and instruction language |
| Photo or diagram | Close-up of the clip and harness at the station | Image-text pair |
| Callouts | Arrow to C3; circle around the harness | Point or region grounding |
| Part and tool IDs | Clip C3, part 000-1234; T20 driver | Object identity linked to the part master |
| Good and bad examples | Harness routed vs harness pinched | State recognition; defect examples |
| Quality and safety marks | Critical-to-quality symbol; glove icon | Constraint flags |
| Parameters | Torque value; connector orientation | Constraints, often confidential |
What published work shows
Research on instruction manuals shows what image-plus-step data can support once it is structured. IKEA-Manual pairs 102 objects with their assembly manuals and annotates decomposed parts, assembly plans, manual segmentation and 2D-3D correspondence (Wang et al., 2022). MEPNet translates a sequence of manual images into machine-executable steps, predicting which parts to add and their 3D poses at each step (Wang et al., 2022).
Those are consumer manuals with consistent drawings; the asset is covered on illustrated assembly manuals. Workplace visual instructions use photos of real stations, which are closer to deployment conditions but follow no shared drawing convention. The pixels also often show a hand on the part at the moment of action. Affordance grounding has been learned by transferring from exocentric images of people interacting with objects to images of the objects alone (Luo et al., 2022), so these hand-on-part photos are more than decoration.
Parsing is the main cost. Steps sit in tables, slide layouts and multi-column PDFs. Human-annotated layout data such as DocLayNet, with 80,863 pages in 11 layout classes across six document categories including manuals (Pfitzmann et al., 2022), helps train the parsers that separate image, caption, callout and step text.
Who holds them
- Discrete manufacturers. Assembly, kitting, machine setup and changeover instructions at each station.
- Contract manufacturers. The same, but often built from customer-owned designs, which limits who can license them.
- Field service and equipment operators. Photo-based service steps such as filter changes and part swaps.
- Food-service operators. Build cards with a photo of each component stage.
- Retailers and merchandisers. Fixture and display setup guides with photos.
- Logistics operators. Customer-specific pack-out instructions with photos of each packing stage.
They live in slide decks, spreadsheets, PDFs, manufacturing execution systems and connected-worker platforms. Platform exports often keep steps, images and completion data together; see connected-worker platform exports. The catalog pages on licensing SOPs and playbooks and internal documentation describe the broader asset class.
Illustrative package contents
# Illustrative only — one synthetic step record from a visual work instruction package
doc: {doc_id: VWI-HAR-044, revision: G, station: harness_cell_2, site: site_052}
step: 5
text: "Route the harness behind clip C3. Do not pinch the harness under the bracket."
image: {file: VWI-HAR-044_s05.png, source: original_native_resolution, px: 3024x4032}
callouts:
- {type: arrow, tip_xy: [1812, 1440], target: clip_C3, author_drawn: true}
- {type: circle, bbox: [1220, 1300, 1950, 1710], target: harness_main}
parts: [{label: clip_C3, part_no: "000-1234"}]
tools: []
examples:
good: VWI-HAR-044_s05_ok.png
bad: [{file: VWI-HAR-044_s05_ng1.png, defect: pinched_under_bracket}]
marks: [critical_to_quality]
parameters: []
redactions: [customer_logo_on_bracket]
linked_video: null
Ask for original image files, not screenshots of PDF pages. Resolution lost in export cannot be recovered, and small callout targets such as clips and pins can shrink to a few dozen pixels in exported pages.
Preparation a package needs
- Extract originals from the authoring files and record their resolution.
- Parse layout so each image, callout and caption is attached to the right step; the general technique is covered on image-text pairs from procedures.
- Convert callouts into point and region annotations, flagged as author-drawn rather than annotator-drawn.
- Read embedded text in images, and keep burned-in labels separate from step text.
- Link part numbers to a part master so the same object has one identity across documents.
- Pair good and bad examples with defect names, which feed object state recognition.
- Redact customer logos, confidential parameters and any faces in photos.
- Match revisions so images reflect the current part and station.
What visual instructions can and cannot support
Can support: image-text grounding of parts, tools and locations; recognition of correct and incorrect states; step-order understanding with visual context; evaluation items such as "which photo shows step 5 complete?"; object identity linked to part numbers.
Cannot support: motion, force or contact; the robot's viewpoint, since photos are close-ups framed by an engineer, often on a tidied station; how a state changes over time; proof that the station still looks like the photo. Completion photos taken during real work are a different asset, covered on photo evidence of step completion.
Quality checks
- Image coverage: share of steps with at least one image, reported per document.
- Resolution: original images meet a minimum you set for your smallest target parts.
- Linkage accuracy: on a sample, callouts and images are attached to the correct step.
- Currency: revision dates and whether superseded images are removed.
- Defect examples: count of good-bad pairs and the defect types they cover.
- Duplicates: the same photo reused across documents is detected and linked, not counted twice.
Questions to ask suppliers
- In which tools are visual instructions authored, and can original image files be exported?
- What share of steps carry photos, and at what resolution?
- Do instructions include good-versus-bad example photos, and for which defects?
- Are part numbers on each step linked to a part master or bill of materials?
- How current are the instructions, and are superseded revisions kept?
- Are any instructions built from customer-owned designs or equipment makers' manuals?
- Do photos show people, customer logos or confidential settings that need redaction?
- Is there video of the same stations that could be linked to the steps?
Discuss your requirements
If you need image-annotated procedure steps for grounding, describe the stations, step types and image detail you need to SourceX. SourceX looks for US businesses that hold suitable instructions, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. It reviews each requirement and follows up about potential sources; a request does not guarantee a match. The procedures hub and the humanoid training data guide cover the related assets.
Explore sourcing and licensing options with SourceX
Sources
- IKEA-Manual: Seeing Shape Assembly Step by Step — manuals annotated with parts, plans and 2D-3D correspondence.
- Translating a Visual LEGO Manual to a Machine-Executable Plan — manual images converted into executable assembly steps.
- DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis — layout annotation that includes manuals.
- Learning Affordance Grounding from Exocentric Images — affordances learned from images of human-object interaction.