Short answer
Human demonstration video can pretrain a robot model's vision, teach it what tasks and steps look like, supply latent actions or reward signals, and, when hands are tracked in 3D, be co-trained with robot data. On its own it contains no robot joint states, commanded actions or contact forces, so a humanoid still needs robot-specific data to learn control.
Why this question decides your data budget
Robot demonstrations are slow and expensive to collect: every episode needs a robot, an operator and a reset. Video of people doing the same work already exists inside operating businesses, in training libraries, process-monitoring cameras and technicians' recordings, and new footage can be recorded without a robot in the room. The question for a humanoid program is not whether human video is "useful" in general, but which part of the training stack it can feed and what you must still collect on hardware.
This page summarizes what published work reports, so you can decide where business video fits before you scope a purchase. For the physical reasons the transfer is hard, see the human-to-humanoid embodiment gap.
What published work shows human video can support
| Use in the stack | What the video contributes | What else the study still needed | Example |
|---|---|---|---|
| Visual representation pretraining | A frozen or fine-tuned image encoder that already "understands" hands, objects and scenes | Robot demonstrations to train the control policy | R3M, MVP |
| Reward and progress estimation | A value or reward signal that tracks task progress from images | Robot trajectories for offline or online RL | VIP |
| Latent-action pretraining | Discrete "latent actions" inferred between frames, used to pretrain a vision-language-action model | A smaller robot dataset to map latent actions to real actions | LAPA, GR00T N1 |
| Motion priors from tracked points | Predicted trajectories of points in the scene that a policy can follow | Robot data for the policy that executes them | ATM |
| Co-training with tracked hands | Human hand and head trajectories placed in a shared state-action space with the robot | Robot demonstrations in the same task family | EgoMimic, Humanoid Policy ~ Human Policy |
| Human-to-robot translation | Robot demonstrations synthesised from 3D hand and object poses | Simulation and a robot hand model | DexMV |
A few results make the pattern concrete:
- Vision pretraining. R3M pretrained an encoder on Ego4D human video and reported more than 20% higher success than training from scratch across 12 simulated tasks; a real Franka Panda then learned tasks in a cluttered apartment from 20 demonstrations (Nair et al., 2022). MVP trained a 307M-parameter vision transformer on 4.5 million images from the internet and egocentric video and used it, frozen, under a learned control module (Radosavovic et al., 2022).
- Rewards without action labels. VIP learned a dense visual reward from Ego4D using an action-free objective, then supported few-shot offline RL on a real robot with as few as 20 trajectories (Ma et al., 2022).
- Latent actions. LAPA pretrained a vision-language-action model with no robot action labels by learning discrete latent actions between frames, then fine-tuned on a small robot dataset; the authors report that pretraining only on human manipulation video outperformed pretraining on the Bridge V2 robot dataset in their evaluation (Ye et al., 2024).
- Point trajectories. ATM pretrained a model to predict the future trajectories of arbitrary points from video demonstrations and used it to guide policies, including transfer from human videos (Wen et al., 2023).
- Co-training. EgoMimic recorded human demonstrations with Project Aria glasses and co-trained one policy on human and robot data, reporting 34–228% improvement over robot-only data on its tasks (Kareer et al., 2024). Humanoid Policy ~ Human Policy treated egocentric human demonstrations as another embodiment, with roughly 27,000 human demonstrations against roughly 1,500 humanoid demonstrations, and reports better generalization and robustness from adding the human data (Qiu et al., 2025).
- Foundation-model data mixes. GR00T N1 describes a "data pyramid": web data and human egocentric video at the base, synthetic and simulated trajectories in the middle, real-robot teleoperation at the top, with latent actions and inverse-dynamics pseudo-actions used to label video that has no actions (NVIDIA, 2025).
Every one of these studies still uses robot data somewhere. Human video changes how much robot data you need and how well the model generalizes; it does not remove the need.
What human video cannot provide on its own
- Robot actions and states. There are no joint positions, end-effector commands or gripper states. Pseudo-actions and latent actions are inferred, and they must be grounded with real robot data. See human demonstrations vs teleoperated robot demonstrations.
- Forces and touch. Video does not show how hard a fastener was tightened or how much a part resisted. Contact-rich tasks need force or tactile data.
- The robot's viewpoint. Head-mounted cameras sit at human eye height with human head motion; fixed cameras look from the ceiling. Neither matches a humanoid's sensor rig without care.
- The robot's body. Human hands have more degrees of freedom, different strength and different reach. Retargeting loses information.
- Labels. Archive footage rarely has step boundaries, task names, success flags or language. Each must be added by annotation, and some cannot be added reliably when hands are occluded or out of frame.
- Coverage. Business footage mostly shows routine, successful work at one or a few sites. Failures, recoveries and unusual objects are rare unless collected deliberately.
What makes business footage more useful
| Property | Why it matters | How to check before licensing |
|---|---|---|
| Hands and objects visible for most of each task | Representation, hand tracking and co-training all depend on it | Sample 20 random clips; count frames with both hands visible |
| Consistent, known viewpoint | Lets you match or correct for the robot's cameras | Ask for camera type, mount and lens; check field of view |
| Task boundaries and names | Turns raw hours into segments a model can use | Ask whether recordings map to job, work-order or procedure IDs |
| Linked language | Enables vision-language training | Look for narration, captions or matching SOP steps |
| Time base and frame rate | Needed for any action or progress model | Ask for frame rate, variable-frame-rate use and timestamps |
| Diversity across sites, people and objects | Drives generalization | Ask for counts by site, operator and object category |
| Clear permissions | Determines whether the data can be used at all | Ask who recorded it, under what notice, and who can license it |
For a full delivery check, use the acceptance criteria for demonstration video.
Illustrative package: business video prepared for robot learning
# Illustrative only — fields and values are examples, not an available dataset
package: workplace_task_video_v0
source: existing training and process recordings from one operator (3 sites)
assets:
video: mp4 (H.264), 1080p, constant 30 fps, original audio removed
clips: segmented per task instance; median length under 3 minutes
annotations:
task_label: from the supplier's procedure IDs
step_segments: start/end time per procedure step
hand_visibility: per-second flag (left, right, both, none)
language: one instruction sentence per step, written by annotators
outcome: completed / abandoned / corrected
metadata: site_id, camera_type, mount (head/wrist/fixed), operator_id (pseudonymous)
excluded: robot actions, joint states, forces (not present in human video)
rights: training and internal evaluation; no redistribution; deletion on request
Use this as a starting point and change it to match your architecture. A team doing latent-action pretraining may care most about volume and diversity; a team co-training with tracked hands needs 3D hand pose and calibrated cameras, which usually means commissioned recording rather than existing footage.
Deciding where human video fits
- If your bottleneck is visual generalization, broad human video for pretraining is a reasonable first purchase; see demonstrations for imitation learning vs visual pretraining.
- If you pretrain a VLA from video, prioritize volume, task diversity and clean segmentation; see sourcing unlabeled task video for latent-action pretraining.
- If you co-train with human data, you need tracked hands, a known camera and tasks that match your robot's; plan commissioned capture.
- If your tasks are contact-rich, budget for robot, teleoperation or instrumented data from the start.
Questions to ask suppliers
- Who recorded the footage, for what purpose, and under what notice to the people in it?
- Which camera types and mounts were used, and does that vary across the archive?
- What share of clips shows the worker's hands and the objects being handled?
- Can recordings be linked to task, procedure or work-order IDs?
- Is there narration, captioning or an associated written procedure?
- How many sites, operators and object types are represented?
- Are failed attempts, interruptions and corrections included or edited out?
- What redaction has been applied, and does it obscure hands, tools or parts?
- Who can authorize licensing, and are any recordings vendor-produced or third-party content?
Discuss your requirements
If you are deciding how much human video to buy against robot data, describe the tasks, modalities and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match; SourceX follows up about potential sources. For the broader map of supply routes, start at the humanoid training data guide or the workplace task demonstration hub. The catalog entry for first-person video of skilled manual work describes the related collection program.
Discuss your humanoid training-data requirements with SourceX
Sources
- R3M: A Universal Visual Representation for Robot Manipulation — human-video pretraining improved downstream manipulation learning.
- Real-World Robot Learning with Masked Visual Pre-training — frozen encoder pretrained on internet and egocentric images.
- VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training — dense visual rewards from action-free human video.
- Latent Action Pretraining from Videos — VLA pretraining without robot action labels.
- Any-point Trajectory Modeling for Policy Learning — point-trajectory models learned from video.
- EgoMimic: Scaling Imitation Learning via Egocentric Video — co-training on tracked human and robot data.
- Humanoid Policy ~ Human Policy — egocentric human demonstrations as cross-embodiment data for humanoids.
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots — data pyramid and pseudo-actions for action-less video.
- DexMV: Imitation Learning for Dexterous Manipulation from Human Videos — translating human hand and object poses into robot demonstrations.