Short answer
The embodiment gap is the set of differences between a person and a humanoid that stop human demonstrations from transferring directly: body proportions, joint limits, hand dexterity, strength, sensor placement and the absence of robot actions in human data. Retargeting, shared state-action spaces, latent actions and co-training narrow it, but each published approach still relies on the target robot's own data, simulator or controller.
Why the gap sets the specification
When a humanoid team buys human demonstration data, the useful question is not whether people and robots are "similar enough". It is which differences the team's method absorbs, and which ones the data must be captured to work around. The answer sets the capture specification (camera placement, 3D hand tracking, task selection) and how much target-robot data stays in the budget.
This page breaks the gap into its parts and maps each part to a buying decision. For where human video fits in a training stack overall, see what human demonstration video can and cannot do for robot learning.
Six parts of the gap
| Dimension | Human demonstrator | Humanoid | Consequence for the data |
|---|---|---|---|
| Body and kinematics | Own limb lengths and joint ranges, flexible spine, shifting stance | Fixed link lengths, joint limits, balance constraints | Copied joint for joint, a human trajectory can be out of reach, self-colliding or unbalanced |
| Hands | Highly articulated fingers, skin friction, nails, fine independent control | Parallel grippers to multi-fingered hands with fewer actuated joints | Some grasps and in-hand moves have no feasible robot equivalent |
| Strength, speed and compliance | Adjusts force and stiffness without thinking; fast reflexive corrections | Payload, torque and speed limits; compliance set by the controller | The demonstrated pace and force may be unachievable or unsafe |
| Sensors and viewpoint | Eyes that move with head and gaze; touch across the whole hand | Fixed camera rig at a set height, field of view and baseline; touch only where sensors exist | The demonstrator's camera sees different pixels from the robot's cameras |
| Action record | No commanded actions; motion has to be estimated | Joint or end-effector commands logged at a control frequency | Actions must be inferred, retargeted or learned from robot data |
| Strategy | Bracing on a hip, pinning with a chin, feeling for an edge | Only strategies its body can execute | Some demonstrations teach a plan the robot cannot carry out |
Each row is a separate problem. A method that solves one, such as a shared representation for hand motion, does not solve another, such as missing forces.
How published work narrows each part
Retargeting body motion. Humanoid-X built more than 20 million humanoid poses with text descriptions by mining internet video, captioning it and retargeting the human motion to a humanoid (Mao et al., 2024). HumanPlus trained a low-level policy in simulation with reinforcement learning on 40 hours of human motion data, used it to shadow a person from a single RGB camera, and then trained skill policies that reached 60–100% success with up to 40 demonstrations per task on a customized 33-DoF humanoid (Fu et al., 2024). OmniH2O uses kinematic pose as a universal control interface, leaving balance and dynamics to the robot's own learned controller (He et al., 2024).
In HumanPlus and OmniH2O, a robot-specific controller trained in simulation absorbs the body and dynamics gap. The human data supplies motion targets, not torques.
Retargeting hands. DexMV extracts 3D hand and object poses from human video and translates them into demonstrations for a simulated multi-finger robot hand; the translated demonstrations improved learning over reinforcement learning alone (Qin et al., 2021). VideoDex draws visual, action and physical priors from internet human video for an arm with a dexterous hand (Shaw et al., 2022). DexCap records hands with a portable motion-capture rig and learns dexterous robot policies from it, but records no forces (Wang et al., 2024).
Hand models such as MANO, learned from about 1,000 high-resolution 3D hand scans, give these pipelines a common parameterization (Romero et al., 2017). The mapping from that model to a specific robot hand is where information is lost; retargeting human hand demonstrations covers those limits.
Shared state-action spaces. Humanoid Policy ~ Human Policy treats an egocentric human as one more embodiment. It captures hand and finger 3D poses with consumer VR devices on robot-aligned tasks and trains one model over a unified human/humanoid state-action space, with roughly 27,000 human demonstrations alongside roughly 1,500 humanoid ones in its reported setup (Qiu et al., 2025). EgoMimic co-trains a single policy on human data from Project Aria glasses and robot data, reporting 34–228% improvement over robot-only data and finding an extra hour of human hand data more valuable than an extra hour of robot data in its experiments (Kareer et al., 2024).
Embodiment-agnostic intermediate representations. GR00T N1 labels action-less human video with latent actions and inverse-dynamics pseudo-actions, then grounds the model with real-robot teleoperation at the top of its data pyramid (NVIDIA, 2025). Heterogeneous Pre-trained Transformers pretrain a shared trunk on real robots, simulation and human video, with an embodiment-specific stem for each body (Wang et al., 2024).
The gap also exists between robots. Open X-Embodiment pooled more than 1 million trajectories from 22 robot embodiments and reports positive transfer across platforms (Open X-Embodiment Collaboration, 2023). Humanoid Policy ~ Human Policy treats the human as a distant member of that family rather than a different category.
What the gap means for what you buy
| Your method | Gap you can tolerate | What the human data must include | Likely route |
|---|---|---|---|
| Visual representation pretraining | Most of it; actions and forces are unused | Broad, diverse video with hands and objects visible | Existing footage |
| Latent-action or world-model pretraining | Body and action gaps, if robot data grounds actions later | Volume, task diversity, stable frame rate, clean segments | Existing footage, curated |
| Co-training with tracked hands | Little; viewpoint and hand representation must line up | Calibrated head-mounted cameras, 3D hand and finger poses, head pose, robot-feasible tasks | Commissioned capture |
| Whole-body motion priors | Proportions, if a retargeter and simulated controller absorb them | Full-body pose of the target motions | Commissioned or specialist capture |
| Dexterous in-hand manipulation | Very little | High-quality finger tracking plus object pose, with occlusion handled | Specialist capture |
| Contact-rich work (insertion, fastening, pressing) | None on the force side | Force or tactile signals | Robot, teleoperation or instrumented rigs |
The last row is the hard boundary. No retargeting step recovers a force that was never measured; when video is not enough covers what to buy instead.
Illustrative embodiment-fit audit
Run a check like this on a small sample before licensing or commissioning. The thresholds are examples a buyer sets, not standards.
# Illustrative only — example checks and buyer-set thresholds
target_robot: humanoid_X # head stereo camera, two multi-finger hands, fixed payload limit
sample: 20 clips from supplier_017 (site_A, site_C)
checks:
hands_visible_both: share of frames with both hands in view # buyer threshold, e.g. 0.6
camera_known: intrinsics and mount documented per clip # yes / no
reach_envelope: share of grasps reachable from the robot's stance
payload: share of handled objects under the robot's payload limit
grasp_feasibility: share of grasps the robot hand can form (power, pinch, lateral)
bimanual_dependency: share of steps that need both hands at once
body_bracing: count of steps using hip, knee, chin or shoulder to hold an object
pace: median step duration vs the robot's speed on similar steps
gloves: share of frames with gloved hands (affects hand tracking)
result: pass / partial / fail per check, with failing clip IDs
A partial result is common. It usually means the material suits pretraining but not co-training, which changes both the price you should pay and the preparation you should budget.
Limits that preparation cannot remove
- Lost information stays lost. Annotation cannot recover occluded fingers, unmeasured forces or the hidden side of an object.
- Estimated motion carries error. Poses estimated from monocular video are noisier than motion capture, and retargeting them onto a different hand or body adds its own approximation.
- No one-to-one hand mapping exists. A five-finger grasp maps many-to-one onto a hand with fewer actuators. That mapping is a design choice, and it changes what the policy learns; see object data for humanoid hands vs grippers.
- Viewpoint shift persists. Images from human eye height, moving with the head, differ from a humanoid's fixed rig. Matching camera viewpoint to humanoid sensors explains how to plan capture around it.
- The robot is the judge. Gains reported from human data are specific to the tasks and robots studied. They are measured by running the robot, not by scoring the human data.
Questions to ask suppliers
- Which camera recorded each clip, where was it mounted, and are intrinsics available?
- Is 3D hand or body pose included, and in which representation (keypoints, MANO parameters, joint angles)?
- How was pose obtained: motion capture, headset tracking or estimation from video?
- Do the tasks stay within a humanoid's reach and payload, or rely on full human strength and posture?
- How often are hands gloved, occluded or out of frame?
- Were demonstrators asked to move in a particular way (slower, without bracing, one hand at a time), or is this natural work?
- Is head pose or gaze recorded alongside the video?
- What range of hand sizes, handedness and body sizes do the demonstrators cover?
Discuss your requirements
Once you know which parts of the gap your method absorbs, you can state exactly what the human data must contain. Describe the tasks, target embodiment and capture properties to SourceX and it will investigate whether US businesses hold suitable material. SourceX checks the data and the supplier's licensing permissions and coordinates the license and delivery. It reviews each request and follows up about potential sources; a request does not guarantee a match.
To compare data types first, read human demonstrations vs teleoperated robot demonstrations, browse the workplace task demonstration hub or start at the humanoid training data guide. The catalog's robotics and embodied AI use case lists related record types.
Discuss your humanoid training-data requirements with SourceX
Sources
- Learning from Massive Human Videos for Universal Humanoid Pose Control — Humanoid-X poses retargeted from human video.
- HumanPlus: Humanoid Shadowing and Imitation from Humans — sim-trained controller on human motion, shadowing and skill learning on a humanoid.
- OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning — kinematic pose as a control interface.
- DexMV: Imitation Learning for Dexterous Manipulation from Human Videos — translating human hand and object poses into robot-hand demonstrations.
- VideoDex: Learning Dexterity from Internet Videos — priors from human video for a dexterous hand.
- DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation — portable hand motion capture without force sensing.
- Embodied Hands: Modeling and Capturing Hands and Bodies Together — the MANO hand model.
- Humanoid Policy ~ Human Policy — unified human/humanoid state-action space.
- EgoMimic: Scaling Imitation Learning via Egocentric Video — co-training tracked human data with robot data.
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots — latent actions and pseudo-actions grounded by teleoperation.
- Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers — shared trunk with embodiment-specific stems.
- Open X-Embodiment: Robotic Learning Datasets and RT-X Models — transfer across robot embodiments.