Short answer
A procedural knowledge graph turns SOPs and work instructions into linked nodes for tasks, steps, objects, tools, conditions and outcomes, with edges for order, dependency and branching. It serves retrieval, planning and instruction grounding. It contains no motion, force or perception data and does not prove that the written procedure matches how work is actually done.
Why buyers ask for a graph and not a corpus
A folder of SOP PDFs is hard to use. Steps are buried in paragraphs, the same tool is named four ways, conditions ("if the torque reading is out of range, stop") hide inside notes, and one document refers to another by number. A model asked to plan from that text has to reconstruct structure on every query.
A graph front-loads that work. Each step becomes a node with typed fields; relationships such as "must follow", "requires", "uses" and "may branch to" become edges. Retrieval can then ask structural questions: which steps need a torque wrench, what must be true before step 7, which procedures share this inspection. Reasoning and evaluation benefit too, since you can test whether a planner proposes an ordering that violates a dependency.
This page is for the buyer deciding what a graph deliverable should contain and what it cannot do. For the document types that feed it, see what SOPs can and cannot teach a humanoid.
Graph elements a buyer can specify
| Element | Examples | Why it matters |
|---|---|---|
| Procedure node | Procedure id, title, version, owner, effective date | Provenance and revision control |
| Step node | Step number, action text, normalized verb, granularity level | The unit a planner reasons over |
| Object and tool nodes | Part, fixture, consumable, instrument, with canonical names and aliases | Links steps across documents |
| Condition nodes | Precondition, postcondition, acceptance test, hazard | Enables checks and safe stopping |
| Parameter slots | Setting, tolerance, quantity, unit, with a flag for redacted | Keeps structure while hiding values |
| Role and location nodes | Technician, inspector, station, zone | Connects tasks to people and space |
| Edge types | follows, requires, uses, branches_on, repeats, refers_to | The reasoning backbone |
| Evidence link | Source document id, page, passage | Auditability and de-duplication |
Each element should carry a pointer back to its source passage. A graph without evidence links can't be audited, and an error in an extracted step cannot be traced.
Public structures worth borrowing
Task-to-activity mapping already exists in public form. The O*NET database includes Task Statements and Detailed Work Activities, and a file that maps tasks to Detailed Work Activities (O*NET Resource Center). A procedural graph can use a similar two-level structure: workplace-specific steps linked to a shared activity vocabulary, so steps from different suppliers become comparable. See occupational task taxonomies.
Knowledge repositories in manipulation research show what linking objects to interactions looks like. OakInk pairs affordance annotations on 1,800 common household objects with recorded human interactions on 100 objects, transferred to virtual counterparts, for about 50,000 affordance-aware, intent-oriented hand-object interactions (Yang et al., 2022). It is a hand-object interaction dataset, not a procedural graph: it contains no steps, ordering or procedures. It is cited only to illustrate the principle that an object node becomes more useful when connected to evidence of how it is handled. A procedural graph can have the same link pointing from a "tighten fastener" step to a set of demonstration clips, when those exist.
The step-and-dependency view is developed in task graphs from procedures.
What the graph can support
- Instruction grounding. Mapping a spoken or typed request to the step or procedure it means. See procedural text for instruction grounding.
- Task decomposition and planning. A language-model planner conditioned on retrieved steps and dependencies. See grounding LLM planners with SOPs.
- Plan checking. Rejecting a proposed sequence that violates a precondition or skips a required inspection.
- Procedural evaluation sets. Held-out questions and orderings that test understanding of steps and constraints. See procedural reasoning evaluation sets.
- Alignment targets. A step list against which video segments can be matched, see SOP-to-video step alignment.
- Conversion to formal planning. Graph structure maps more cleanly onto planning languages and behavior trees; see procedures to PDDL and behavior trees.
What it cannot support
- Physical control. A step node says "seat the connector." It does not provide trajectories, grasp poses, forces or the visual cues for a connector that is properly seated. Control requires demonstrations or robot data.
- Tacit skill. Experienced workers do things documents omit. See tacit knowledge gaps in SOPs.
- Proof of practice. A procedure describes intended work. Whether people follow it is a different question, addressed by procedural drift and deviation records.
- Perception. Objects named in the graph still need images, 3D models or catalog data to be recognized.
- Completeness. A graph covers the documents supplied. Missing procedures, informal practices and exception handling are absent.
How graphs are built, and where errors enter
- Parse documents. PDFs, scans and slide decks become text with layout preserved. Tables and figures are the first casualties; see parsing scanned manuals.
- Segment into steps. Decide granularity; a coarse step ("replace filter") and a fine one ("loosen clamp") need different node levels. See step granularity.
- Extract entities and conditions. Tools, parts, quantities, warnings, branches.
- Normalize names. Merge aliases and part numbers to canonical entities.
- Link and verify. Add edges, then have a subject-matter reviewer check a sample.
- Redact where needed. Replace sensitive parameters and names with typed placeholders, and record that you did. Redaction affects value: a tolerance replaced by "[value]" preserves structure and loses the number. See redacting SOPs.
Automated extraction with language models is quick and produces plausible errors: swapped step order, dropped conditions, merged branches. Treat unreviewed extraction as a draft and read validating LLM-extracted steps.
Illustrative graph record
{
"_comment": "Illustrative only - synthetic values, not an available dataset",
"procedure": {"id": "SOP-0001", "version": "C", "owner_role": "quality_engineer", "source_doc": "doc_0042", "language": "en"},
"steps": [
{"id": "S3", "text": "Fit the cover plate and fasten.", "verb": "fasten",
"objects": ["cover_plate_A"], "tools": ["torque_driver_T1"],
"parameters": [{"name": "torque", "value": "[REDACTED]", "unit": "N*m"}],
"preconditions": ["S2.done", "surface.clean"],
"postconditions": ["plate.seated"],
"evidence": {"doc": "doc_0042", "page": 4, "span": "para 2"}}
],
"edges": [
{"from": "S2", "to": "S3", "type": "follows"},
{"from": "S3", "to": "S4", "type": "branches_on", "condition": "torque_check.fail -> S3b"}
],
"review": {"sample_checked": "20 percent", "reviewer_role": "subject_matter_expert", "error_notes": "stored"}
}
Use the procedure step schema as the starting point for field names, and agree preconditions language with the guidance on preconditions and postconditions.
Sourcing
The raw material sits in quality, operations and documentation teams of operating businesses. Some hold structured authoring systems that already contain step-level data; many hold PDFs. Conversion to a buyer-defined schema is annotation work, with the cost and quality risks described above. Rights questions are moderate: SOPs may be trade secrets, include third-party content such as manufacturer instructions, or carry safety obligations. Settle who owns each procedure before extraction; see who owns a procedure. This is not legal advice.
If you want graph-ready procedures in a specific domain, describe the schema, domain and permitted uses to SourceX and it will investigate whether US businesses hold suitable material. Annotation and graph construction are scoped case by case.
Questions to ask suppliers
- Which source documents were used, and can each node trace back to a page and passage?
- What schema and edge types does the graph use, and can it map to ours?
- Was extraction manual, automated or hybrid, and what share was reviewed by a subject-matter expert?
- How are aliases and part numbers merged, and who approved the mapping?
- Are branches, exceptions and warnings kept as separate nodes?
- How are redacted parameters represented, and is the redaction rate reported?
- Which document versions are included, and are effective dates recorded?
- Is any content from manufacturers, standards bodies or customers, and who can license it?
- Does the graph cover only normal steps, or exceptions and recovery too?
- Can a set of procedures be reserved as a held-out evaluation set?
Discuss your requirements
Discuss your humanoid training-data requirements with SourceX by describing the procedure domains, graph schema and permitted uses. SourceX reviews the requirement and follows up about potential sources; a request does not guarantee a match, and nothing is contracted until a supplier agrees. See the SOPs and procedural data hub, the humanoid training data guide, and the catalog page on SOPs, playbooks and knowledge bases.
Discuss your humanoid training-data requirements with SourceX
Sources
- OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interaction — example of a knowledge repository linking objects to interactions.
- ONET Database (ONET 29.1 per search snapshot): Task Statements and Detailed Work Activities — task statements mapped to Detailed Work Activities.