Trainer Feedback and Correction Transcripts as Language Supervision

Why corrections are their own signal

Instruction data says what to do. Correction data says what was wrong with what just happened, and how to fix it. A supervisor watching a new hire stack cartons does not repeat the procedure; they say "that one goes on the bottom" or "check the label first." The meaning of that sentence is only recoverable from the moment it follows.

Published work on language-model planners shows why this matters. Inner Monologue let a planner replan using natural-language feedback from success detection, scene description and human interaction, evaluated on simulated and real tabletop rearrangement and on long-horizon mobile manipulation in a real kitchen, without additional training of the model (Huang et al., 2022). The study took the feedback loop as given; a buyer building one needs realistic human feedback language to test it against. That is the gap this data fills.

Spoken coaching in training departments, apprenticeships and onboarding sessions is the natural source. It overlaps with, but differs from, the course content in LMS training courses: courses present a clean procedure, while coaching shows the errors and the vocabulary used to fix them.

What a correction record needs

A transcript alone is weak. The utterance becomes useful when it is attached to what it corrects.

• Anchor. The time, step or video segment the correction refers to.
• Error type. Wrong object, wrong order, wrong amount, wrong place, unsafe action, quality issue.
• Form. Imperative, negation, comparison ("a little higher"), deixis ("not that one"), question, praise.
• Resolution. What the learner did next, and whether the error was fixed.
• Speaker role. Trainer, peer, supervisor. Roles only; names are not needed.
• Context. The procedure or task and its step, so the correction can be checked against the text.

The deictic forms are the hardest and the most valuable. "Not that one" can only be resolved with a view of the scene, so a transcript without synchronized video can be used for linguistic analysis but not for grounding.

Illustrative record

The record is Illustrative: every value is synthetic.

{
  "session_id": "sess_site_022_015",
  "utterance_id": "u_0087",
  "speaker_role": "trainer",
  "text": "No, the gray one, then pull the tab.",
  "t_start_s": 412.6,
  "t_end_s": 415.0,
  "anchor": {"step_ref": "SOP-site_022-0031/step_5", "video_segment": "seg_0044"},
  "error_type": "wrong_object",
  "form": ["negation", "attribute_reference", "imperative"],
  "learner_response": "switched_object",
  "resolved": true,
  "audio_available": true,
  "consent_scope": "training_and_evaluation"
}

Existing recordings versus commissioned sessions

Existing recordings. Training departments sometimes record onboarding or coaching sessions for quality or audit purposes. Look for them in shadowing sessions, simulation-lab debriefs and recorded role-plays. They hold real corrections, but speakers did not consent to model training when recorded, audio and video capture settings were not designed for synchronization, and coverage is whatever tasks were taught that week.

Commissioned sessions. A trainer and learner perform a chosen task under a script that elicits specific error types, with consent covering training use from the start. This allows planned coverage of error classes, but staged errors can sound artificial. A mix of both is often better: use real recordings to learn natural phrasing and commissioned sessions to fill gaps. Whether a commissioned session is feasible for a particular task is a scoping conversation, not an assumption.

Matching corrections to demonstrations is described in instruction-paired demonstration datasets.

Quality and acceptance criteria

Decide in advance what a usable correction set looks like, then test a sample against it.

• At least a stated share of utterances carry an anchor to a step or segment, and unanchored ones are flagged and kept separate.
• Error types are labeled with a taxonomy the supplier documents, and two reviewers agree on a sampled subset.
• Corrections that depend on the scene (deictic references) are marked so they are only used where video is synchronized.
• The set includes sessions with no error, so a model can learn that silence is also a valid outcome.
• Evaluation sessions are separated from training by trainer, learner and site, because coaching style is highly personal.

How this differs from neighboring language data

Corrections are not the same as the clarifying questions covered in ambiguous instructions and clarification data: there the robot asks, here a human tells. They also differ from natural worker instructions, which are issued before the action rather than after it. Treat the three as separate label spaces even when one recording contains all of them. A single session can supply an instruction, an error, a correction and a clarification in a few seconds, and mixing the labels makes each weaker.

If the corrections are linked to proficiency judgments, see competency assessments and sign-offs for how those records can label the same sessions.

What this data cannot support

Correction transcripts cannot teach a robot to act on a correction. The robot needs a policy that maps "a little higher" to a joint or end-effector change, which comes from robot or teleoperation data. Human coaching recorded in a workplace shows people correcting people; it does not contain robot actions, and a language model that understands the sentence still needs low-level skills to execute it. Footage and transcripts alone cannot train a humanoid's physical control.

Other limits:

• Human-to-human register. Trainers calibrate to a person's body and skill. "Easy, you're hurting it" has no direct robot equivalent.
• Selection bias. Corrections occur when learners err, so the data under-represents correct performance and gives no base rate.
• Voice and affect. Audio carries identifiable voices and emotion; text-only releases lose prosody that often signals urgency.
• Safety. A shouted "stop" in an unsafe situation is not a template for safe-stop behavior, which is a robot-system requirement.

Permissions and provenance

Coaching recordings contain two identifiable people having a conversation about performance. Recorded voice is personal data in many settings, and learners may be students or new employees with limited ability to refuse. A transcript that blurs names but keeps the voice recording does not resolve the underlying questions. See worker consent and identifiable people for the diligence questions, and ask whether the supplier can license the audio, the transcript or both. This is not legal advice.

Questions to ask suppliers

• Were sessions recorded for training, quality assurance or audit, and what consent language applied?
• Is the audio, a transcript or a video available, and are they time-aligned?
• Can each correction be tied to a procedure step or task moment?
• How were error types labeled, and by whom?
• Do sessions include learner responses, or only the trainer's side?
• Are trainers and learners distinct people across sessions, so a held-out evaluation can split by speaker?
• Are there recordings where the learner is a minor or a student, and how are those handled?
• Can you provide a transcript sample with anchors before scoping a package?

Discuss your requirements

If correction language belongs in your plan, describe the tasks, error types, modalities and consent scope in a buyer request. SourceX reviews the requirement and follows up about potential sources; a request does not guarantee a match. If no existing recordings fit, SourceX can scope a commissioned collection of new recordings. Return to the procedures hub or the humanoid training-data section for the wider context.

Discuss your humanoid training-data requirements with SourceX.

Sources

• Inner Monologue: Embodied Reasoning through Planning with Language Models — replanning from natural-language feedback, including human interaction.