Short answer
Written procedures record what a task requires, not how experienced workers actually do it. Grip choices, timing cues, sensory checks, workarounds and recovery moves are tacit and usually absent from SOPs. Buyers can recover part of this with video, narration and expert review, but no document or recording alone supplies the motion, force and contact data a humanoid needs for control.
Why the gap exists
An SOP is written for a person who already has a body of practice. "Seat the gasket" assumes the reader knows how much pressure, from which side, and what a correctly seated gasket looks like. Authors leave these details out because they are obvious to the author, because they vary by person, or because a regulator or auditor only needs the controlled steps.
The gap matters more for robots than for new hires. A new hire fills it by watching a colleague and by trial and error. A model trained on procedural text inherits only the written layer, and a model trained on video inherits the visible layer. The sensory and judgment layers reach neither unless someone records them on purpose.
This page is a companion to what SOPs can and cannot teach a humanoid. That page covers the whole inventory of what text provides; this one focuses on what is missing and what can partly replace it.
A working taxonomy of what is left out
| Layer | Example the SOP omits | Can video show it? | Needs something else |
|---|---|---|---|
| Motor technique | Which hand steadies, which acts; angle of approach | Often, with a clear view of the hands | Robot or instrumented data for control |
| Force and feel | "Tighten until snug," how hard to press a seal | No | Force-torque or instrumented-tool data |
| Perceptual cues | The sound of a latch, the color of a cured bead | Partly (audio, close-ups) | Narration, expert labels |
| Timing | Wait until the surface stops glistening | Partly | Timestamped narration |
| Decision rules | When to rework instead of continue | Only if a case occurs on camera | Branch documentation, expert review |
| Workarounds | Nudging a stuck part with a tool handle | If it happens while recording | Interviews, failure-and-recovery capture |
| Context | Why a step exists, what it protects | No | Hazard analyses, design records |
Two rows deserve emphasis. Force and feel do not appear in any video, however good, so tasks whose success depends on them need robot-side or instrumented data. Decision rules and workarounds appear only when the event occurs during recording, which is why coverage planning matters as much as camera quality.
Ways to recover tacit knowledge
Watch the task. Recording skilled workers captures technique that no author thought to write down. Ego-Exo4D recorded skilled activities, including bike repair, with synchronized first- and third-person cameras, and its benchmarks include proficiency estimation, which treats the quality of execution as something a model can be asked about (Grauman et al., 2023). That is a research dataset about human skill, not robot control data.
Ask the expert while they work. The same dataset includes expert commentary on the recorded activities. Think-aloud narration, or a later review session where the expert watches the clip and explains their choices, converts some tacit judgment into text. See narrated demonstrations for how to specify this.
Decompose the task formally. Hierarchical task analysis breaks a task into goals, sub-goals and plans, and was originally developed to identify training needs (Stanton, 2006). Applied to an SOP, it exposes steps that are really clusters of unwritten sub-steps. The hierarchical task analyses page covers how buyers can use the output.
Capture guided interaction. HoloAssist is an egocentric dataset built for interactive AI assistants (HoloAssist, 2023); it is one example of recording the instruction layer and the execution layer together. Treat it as a reference for that pairing, not as a source of task content for a workplace.
Record mistakes and corrections. Experts reveal tacit knowledge when something goes wrong. A recording protocol that deliberately includes slips, resets and rework captures recoveries that SOPs rarely describe.
What this means for what you buy
- If you buy only procedures, expect task structure, ordering and vocabulary. Plan separately for any motion or perception data.
- If you buy procedures plus video, align them step by step and measure the mismatch. Steps with no visible counterpart, and visible actions with no step, are your tacit-knowledge map. See SOP-to-video step alignment.
- If you commission recording, use the SOP as the script and ask for narration at the steps where the gap is largest. See SOPs as collection scripts.
- If your tasks are contact-rich, accept that the tacit layer lives in force and tactile signals, which come from robots, teleoperation or instrumented tools rather than from business archives.
Recovering tacit knowledge into labels and text improves a task representation. It does not convert a recording into actions, and it does not remove the embodiment gap between a human hand and a humanoid hand; see the human-to-humanoid embodiment gap.
Limitations
Even a thorough gap audit leaves limits a buyer should plan around:
- Narration is a reconstruction. Experts explaining their own work describe what they believe they do; think-aloud and retrospective commentary can omit or rationalise cues they use without noticing. Check narration against what the video shows.
- Coverage is narrow. One site, one shift pattern and a handful of experts capture one way of doing the task. Variants used elsewhere, and rare faults, may be absent.
- Labels are not signals. Annotating "apply firm pressure" names a tacit cue but does not measure it. Force, torque and tactile information still has to come from robots, teleoperation or instrumented tools.
- Procedures drift. SOPs and practice diverge over time; check the document revision and the recording date are close.
- Rights travel separately. A license to the SOP text does not cover the recordings, the narration or the people in them; each layer needs its own permissions.
None of this makes procedural data unhelpful. It sets realistic expectations: procedures plus recovered tacit knowledge improve task structure and language, not physical control on their own.
Illustrative gap audit for one procedure
# Illustrative only - synthetic values, not an available dataset
procedure_id: sop_0042_replace_filter_cartridge
written_steps: 9
audit:
steps_with_unwritten_substeps: [3, 5, 6, 8]
unwritten_substeps_identified: 14
tacit_layer_by_type:
motor_technique: 6
perceptual_cue: 4
decision_rule: 3
force_or_feel: 1
recoverable_from_video: 8
recoverable_from_narration: 4
needs_instrumented_or_robot_data: 2
evidence_sources: [clip_site_017_op_A12_003, expert_review_note_007]
reviewer_role: senior_technician
The counts are placeholders. The useful output is the last three lines: a per-procedure estimate of how much you can recover by annotation and how much needs different data.
Questions to ask suppliers
- Were the SOPs written from observation of the work, or from design intent and audit requirements?
- How current are they, and is there evidence that practice has drifted from the written version?
- Does the supplier hold video, photos or checklists that record how the steps are actually performed?
- Can an experienced worker annotate or narrate selected steps, and under what terms?
- Are exceptions, rework paths and escalation rules documented anywhere?
- Which steps depend on feel, sound or smell, and is any instrumented data available for them?
- Are different sites or shifts using different methods for the same step?
- Who owns the knowledge captured in narration, and can it be licensed with the SOP?
Discuss your requirements
If your project needs to pair procedures with recorded or annotated practice, describe the tasks and the layers you need to SourceX. SourceX reviews the requirement and follows up about potential sources among US businesses, checking the data and the supplier's licensing permissions; a request does not guarantee a match. For the wider map, see the humanoid training data guide and the procedural data hub, or the catalog entries for SOPs and playbooks and SOPs and playbooks to buy. Pairing with demonstration video is covered in what human demonstration video can and cannot do.
Discuss your humanoid training-data requirements with SourceX
Sources
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives — synchronized ego/exo skilled-activity recordings, expert commentary, proficiency benchmarks.
- HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real World — example of an egocentric dataset built for interactive assistants (existence and nature only).
- Hierarchical task analysis: Developments, applications, and extensions — HTA as a task-decomposition method developed for training needs.