Short answer
A blank checklist is a business's own statement of what "done" means for a task: the items that must be true, in what order, and who confirms them. As data, it supplies success criteria that a planner, a success detector or an evaluation harness can check against. It contains no motion, force or perception data, so it specifies verification rather than control.
The buyer's problem
Most humanoid pipelines can generate attempts faster than they can judge them. Someone has to decide whether the tray was stocked, the room was reset or the fixture was clamped, and that decision needs a definition that a robot team did not invent after the fact. Operating businesses already hold such definitions in quality checklists, shift-start lists, room-reset cards and close-out forms.
The research motivation is straightforward. Closed-loop language-model planners replan using natural-language feedback, and success detection is one of the feedback channels in Inner Monologue (Huang et al., 2022). A detector needs something to detect against. REFLECT builds hierarchical summaries of multisensory robot data and asks an LLM to explain failures and propose corrections, evaluated on a dataset of failed executions (Liu et al., 2023). In both cases, the quality of the verdict depends on how well the criteria were stated.
This page covers blank checklists used as specifications. Filled-in forms are a different asset with different rights and value; see completed checklists and sign-off records. For how checklists differ from SOPs and work instructions, see SOPs vs work instructions vs checklists.
What checklist data can support, and what it cannot
| Use | What the checklist contributes | What else you need |
|---|---|---|
| Success-detector specification | Observable end-state conditions per item | A perception model that can see each condition |
| Evaluation rubrics | Pass/fail items scored per trial | Held-out trials, scorers, and a protocol |
| Task decomposition | Implicit step order and grouping | Step-level grounding in real execution |
| Precondition checks | "Before you start" items | Sensors or queries that expose that state |
| Error taxonomy seeds | The failure each item was written to prevent | Observed failures to confirm the taxonomy |
It cannot show how a step is performed, how hard to press, or whether workers actually follow the list. A checklist item such as "valve closed" says nothing about the grasp that closes it. Buyers also should not assume the items match practice; see procedural drift.
Illustrative item schema
Items are rarely machine-ready as supplied. This schema shows what a prepared checklist could look like after structuring. All values are synthetic.
# Illustrative only - synthetic example, not an available dataset
checklist_id: chk_0042
version: "r3"
task_ref: sop_0117 # link to parent procedure, if one exists
phase: close_out # pre_start | in_process | close_out
items:
- item_id: chk_0042_07
text: "Waste bin liner replaced and bin returned to marked floor position"
check_type: end_state # end_state | action_done | measurement | judgment
observable: visual # visual | sensor | measurement | human_only
evidence_expected: photo # photo | reading | none
tolerance: "bin within marked outline"
severity: minor # minor | major | critical
depends_on: [chk_0042_05]
exceptions: "skip if room status = out_of_service"
rights_note: "internal operational document; license review required"
The check_type and observable fields do most of the work for a robotics buyer. Items marked human_only, such as "area feels safe", cannot be scored by a vision-based verifier and should be excluded or paired with a proxy measurement.
Specifying the data you want
Buyers can adapt these parameters; none of them is a universal requirement.
- Granularity. Decide whether you want task-level lists (one per job) or step-level lists (one per sub-step). Mixed granularity inside one document is common and needs separating.
- Observability tagging. Ask for each item to be tagged by whether a camera, a sensor reading or a person is needed to verify it.
- Tolerances. Where items say "clean" or "aligned" without a measure, request the underlying standard if the supplier has one, or flag the item as ambiguous.
- Severity. Critical items often define stop conditions. They deserve separate handling, as described in safety-critical steps in procedures.
- Linkage. Each checklist should map to a parent procedure ID, a role and an equipment type so it can be joined to other assets.
- Revision history. Version identifiers and effective dates let you avoid scoring against superseded criteria.
Quality management frameworks treat checklists as one form of documented information that organizations maintain to run and evaluate their processes; ISO 9001's requirements cover documented information and performance evaluation (ISO, 2026). Holding an ISO 9001 certificate does not make a particular checklist well written, and a supplier citing it tells you little about item quality.
From checklist item to verifier input
A practical pipeline has three hops. First, each item is rewritten as an end-state statement a perception or state-reading module can test ("liner present, bin inside outline"). Second, the statement is tied to the camera view or sensor that can see it, which is a property of your robot rather than the business's document. Third, the item is scored per trial against a human judgment, so you learn how often the verifier agrees with a person.
Items about conditions before the task starts belong in a separate list, since they gate whether a trial should begin; see preconditions and postconditions. Keeping the two lists apart also stops a verifier from rewarding a robot for a room that was already clean.
Acceptance criteria you can set
- Parse rate. Every delivered checklist yields item text, order and phase without manual repair; set your own tolerance for exceptions.
- Observability coverage. Report the share of items tagged visual, sensor, measurement and human-only, and set the minimum visual share your verifier needs.
- Ambiguity audit. Have two reviewers independently mark ambiguous items on a sample; high disagreement means the list needs rewriting before use as a rubric.
- Join integrity. Checklist IDs resolve to procedure IDs in the same delivery.
- No leakage. If checklists feed an evaluation set, confirm that they were not used to build training prompts for the same tasks, and that the split follows your site, role or time design. See procedural reasoning evaluation sets.
Limitations and extra preparation
Checklists written for trained humans lean on shared context. "Check that the station is ready" assumes the worker knows what ready looks like. Converting such items into testable end states usually needs a domain reviewer, and the rewritten version is then your annotation rather than the supplier's original.
Some items verify records, not the physical world: "sign the log" or "notify supervisor" are real obligations but not observable manipulation outcomes. Separate them from end-state items.
Third-party content can sit inside a checklist, such as a manufacturer's torque value or a regulator's wording. Provenance for those lines may differ from the business's own text, and redaction decisions are covered in redacting SOPs. A photo requirement in the list also implies a photo archive; see photo evidence of step completion.
Questions to ask suppliers
- Are these the lists staff actually use on the floor, or audit-only documents?
- How often are they revised, and can you provide the revision log?
- Which items are verified by a person's judgment, and which by a measurement or photo?
- Do severity or "critical" flags exist in the source, or would they be added in preparation?
- Are any items copied from manufacturer manuals, customer specifications or regulations?
- Do checklists link to procedure, equipment or location IDs?
- What document formats exist (PDF forms, spreadsheets, an app export), and what is lost on export?
- Who owns the wording, and who can authorize a license?
Discuss your requirements
If you are building a verification or evaluation set, describe the tasks, document types and permitted uses to SourceX. SourceX looks for US businesses that hold suitable documents, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match; SourceX follows up about potential sources.
For the wider map, see the humanoid training data guide and the SOPs and procedural data hub. The catalog entry for SOPs and playbooks describes the general document category.
Tell SourceX what your evaluation set needs to cover
Sources
- Inner Monologue: Embodied Reasoning through Planning with Language Models — Success detection is one of the feedback sources language-model planners use to replan in closed loop.
- REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction — Hierarchical summaries of robot experience queried by an LLM for failure explanation, evaluated on a dataset of failed executions.
- ISO 9001:2026 Quality management systems - Requirements — Quality management requirements cover documented information and performance evaluation.