Short answer
Workplace task demonstration data is recorded human work, meaning video with optional hand pose, IMU and audio, plus labels and linked procedures, that shows how real tasks are done. It supports visual pretraining, task and step understanding, latent-action and reward learning, and co-training when hands are tracked in 3D. It contains no robot actions or forces, so humanoid control still needs robot data.
What this cluster covers
This is the largest cluster in the humanoid training data guide because demonstrations are where business supply and robot learning overlap most. It covers four things: deciding what demonstration data your model can use, finding existing footage, specifying new capture, and turning raw recordings into labeled, accepted deliveries. Task pages then apply those decisions to specific work such as garment folding, room turnover or shelf restocking.
The cluster treats demonstrations as human data. Teleoperated robot episodes and handheld-gripper data are robot-format data and sit in the procurement and specialist collection cluster.
What demonstration data looks like
At one end are archives recorded for other reasons: onboarding videos, practical-assessment recordings, technician body-camera sessions and fixed process cameras. At the other end are corpora designed for machine learning. Their design choices show what purpose-built capture can include:
- Scale and consent by design. Ego4D gathered 3,670 hours of daily-life video from 931 consenting camera wearers, under de-identification procedures (Grauman et al., 2021).
- Synchronized viewpoints and sensors. Ego-Exo4D recorded skilled activities, including bike repair, with an egocentric camera plus several exocentric cameras, eye gaze, IMU, 3D point clouds and expert commentary (Grauman et al., 2023).
- Tracked hands. EgoDex paired 829 hours of egocentric video with 3D hand and finger tracking across 194 tabletop tasks (Hoque et al., 2025).
- Language at capture time. EPIC-KITCHENS-100 used "pause-and-talk" narration as its first annotation stage, then transcribed, parsed and segmented it (Damen et al., 2020).
Business archives rarely have any of these properties out of the box. Most of the work in this cluster is about closing that gap, either by adding labels to existing footage or by commissioning recording that builds them in.
What it can support, and what it cannot
| Use in a humanoid stack | What the data must have | Usual route |
|---|---|---|
| Visual representation pretraining | Volume, diversity, hands and objects in view | Existing footage |
| Task and step understanding, evaluation | Task names, step boundaries, outcomes | Existing footage plus annotation |
| Video-language training | Narration, instructions or aligned SOP steps | Narrated footage, SOP pairing |
| Latent-action or world-model pretraining | Clean segments, constant frame rate, broad task coverage | Existing footage |
| Co-training with robot data | 3D hand pose, calibrated cameras, tasks matching the robot's | Commissioned capture |
| Progress and reward estimation | Complete episodes with outcome labels | Either, with annotation |
Representation learning from human video is well established: R3M pretrained on Ego4D and served as a frozen perception module for downstream robot learning (Nair et al., 2022). Foundation-model data mixes put human video at the base and label it with latent or pseudo-actions (NVIDIA, 2025). Co-training needs more: EgoMimic captured its human demonstrations with Project Aria glasses so hand and head trajectories could share a space with the robot's (Kareer et al., 2024).
What the data cannot supply is the robot side: joint states, commanded actions, contact forces, the robot's own viewpoint and its kinematics. The detailed account is in what human demonstration video can and cannot do and the human-to-humanoid embodiment gap. Tasks that depend on touch or force need the data described in when video is not enough.
Where demonstration data comes from
Existing footage. Corporate learning teams, trade schools and certification bodies, field-service organizations, franchise networks and facility contractors hold recordings of real work. The most plausible starting point is corporate training video archives. Fixed cameras are the most common and least useful source for manipulation; see what CCTV footage can and cannot support. Before buying, check that public instructional video would not do the job; public instructional video vs licensed footage compares the two.
Commissioned recording. New capture lets you choose camera placement, sensors, task scripts, operator mix and consent wording. It costs time and money and still needs willing businesses and site access. The site describes a separate physical-world path for new recordings of hands-on work coordinated with businesses, where each opportunity has its own requirements, and the catalog entry for first-person video of skilled manual work describes that collection program. The decision itself is covered in existing footage vs commissioned recording.
Annotation on either. Step segments, hand-object labels, language and outcome flags can be added after the fact, within limits set by what the camera saw. SourceX's role is sourcing, rights review, licensing and delivery coordination; it can also arrange annotation and engage specialist capture partners, scoped case by case.
How to evaluate demonstration supply
Ask for evidence on each of these before you license anything. The thresholds are yours to set; the examples are illustrative.
- Hands in view. Share of sampled frames with both hands and the manipulated object visible (for example, a buyer might require 70% for co-training use).
- Viewpoint. Camera type, mount and lens per recording, and whether it changes across the archive. Egocentric and fixed views serve different uses.
- Segmentability. Whether recordings map to job, work-order or procedure IDs that make step labeling tractable.
- Coverage. Counts by site, operator, object type and condition, including failures and recoveries.
- Capture quality. Frame rate, motion blur at production pace, lighting, gloves and PPE.
- Language. Narration, captions or linked SOPs. Automatic transcripts are cheap but loose: HowTo100M's captions are ASR narrations with no extra manual annotation (Miech et al., 2019), and narration often runs ahead of or behind the action.
- Permissions. Who appears, what they were told, and who can license the footage. See worker consent and identifiable people.
The delivery-level version of this list is acceptance criteria for demonstration video.
Illustrative triage of three candidate assets
| Candidate (synthetic) | What it offers | Main gap | Next step |
|---|---|---|---|
archive_LD_01: onboarding videos, tripod-mounted, narrated | Task and step understanding; video-language pairs | Third-person view; hands often occluded by the body | Sample 20 clips for hand visibility; confirm the course vendor does not own them |
bodycam_FS_02: technician body-camera sessions | First-person repair footage with expert talk | Customer premises; inconsistent framing | Check customer contract terms and premises permissions |
fixedcam_PK_03: ceiling cameras over pack stations | Coverage and task timing at volume | No hand detail | Use for activity timing and task-recognition evaluation only |
None of the three gives co-training data. If that is the goal, plan commissioned capture with tracked hands from the start.
Key pages in this cluster
| Group | Page | What it answers |
|---|---|---|
| Foundations and limits | Human demonstration video for robot learning | Which parts of the stack human video can feed |
| Foundations and limits | The human-to-humanoid embodiment gap | Why human motion does not transfer directly |
| Foundations and limits | Human demonstrations vs teleoperation | Where business data ends and robot data begins |
| Foundations and limits | When video is not enough | Tasks that need force or tactile data |
| Foundations and limits | Demonstrations for imitation vs pretraining | Different data requirements for the two uses |
| Foundations and limits | How many demonstrations do you need? | What published work reports, without universal minimums |
| Foundations and limits | Unlabeled video for latent-action pretraining | Sourcing volume and diversity for VLA pretraining |
| Existing footage | Existing footage vs commissioned recording | The core license-or-record decision |
| Existing footage | Corporate training video archives | Checklist for the most plausible existing asset |
| Existing footage | Instruction-paired demonstration datasets | Video that comes with matching written procedures |
| Existing footage | Public instructional video vs licensed footage | When web video suffices and when it does not |
| Existing footage | Fixed-camera and CCTV footage | What ceiling cameras can and cannot support |
| Existing footage | Annotating existing footage | Which labels can be added after the fact |
| Commissioned capture | Egocentric vs exocentric workplace video | Choosing the viewpoint for your use |
| Commissioned capture | Staged workcells vs live operations | Where to record demonstrations |
| Commissioned capture | Head-mounted camera capture specification | An adaptable spec for egocentric capture |
| Commissioned capture | Scoping a pilot collection | Testing assumptions before full capture |
| Commissioned capture | Demonstration coverage plan | Operators, sites, objects and conditions to cover |
| Commissioned capture | 3D hand pose data | Getting hand pose from workplace recordings |
| Annotation and acceptance | Step segmentation annotation | Task boundaries and step labels |
| Annotation and acceptance | Hand-object interaction annotation | Contacts, grasps and active objects |
| Annotation and acceptance | Language annotations for demonstrations | Instructions, captions and narrations |
| Annotation and acceptance | Clip metadata schema | Fields every clip should carry |
| Annotation and acceptance | Redaction without destroying training value | Face and bystander redaction trade-offs |
| Annotation and acceptance | Acceptance criteria for demonstration video | Pass/fail checks on a delivery |
| Task pages | Bimanual task demonstrations | Two-handed work for two-armed robots |
| Task pages | Garment folding and textile handling | Deformable-object demonstrations from laundries |
| Task pages | Hotel housekeeping room turnover | Long-horizon service work in guest rooms |
| Task pages | Commercial kitchen food prep | Cutting, portioning and plating |
| Task pages | Retail shelf restocking | Restocking and facing in stores |
Neighboring clusters
Demonstrations rarely stand alone. Written procedures supply the step names and success criteria that make video usable; see SOP-to-video step alignment and the procedures hub. Tool use and repair footage lives in maintenance and tool use; picking, packing and unloading in warehouse and fulfillment; assembly and machine tending in manufacturing and assembly.
Discuss your requirements
If you know the tasks, viewpoints and labels you need, describe them to SourceX, along with the uses you need licensed and how you will hold out evaluation data. SourceX investigates whether US businesses hold suitable footage, checks the supplier's licensing permissions and coordinates the license and delivery. It reviews each requirement and follows up about potential sources; a request does not guarantee a match.
Describe the workplace demonstration data your robotics project needs
Sources
- Ego4D: Around the World in 3,000 Hours of Egocentric Video — scale, consent and de-identification in a purpose-built egocentric corpus.
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives — synchronized egocentric and exocentric capture with extra sensors and commentary.
- EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video — egocentric video with paired 3D hand tracking.
- Rescaling Egocentric Vision — narration captured during recording as the first annotation stage.
- HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips — automatic transcripts as captions.
- R3M: A Universal Visual Representation for Robot Manipulation — human-video pretraining used as a frozen perception module.
- EgoMimic: Scaling Imitation Learning via Egocentric Video — tracked egocentric human data co-trained with robot data.
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots — human video at the base of a humanoid data mix.