Matching Demonstration Camera Viewpoints to a Humanoid's Sensors

The viewpoint gap in concrete terms

A trained visual policy learns a mapping from images to actions under the camera geometry it saw in training. Change the geometry and the same scene produces different pixels. The gap between a human wearer's camera and a humanoid's has several parts:

• Height and offset. A human's eyes sit at a different height than a robot's head cameras, and the camera sits at a different offset from the shoulders and hands.
• Orientation. People tilt their heads down for close manipulation and look up to find objects; a robot's head may have a fixed or limited tilt range.
• Head motion. Human gaze moves constantly and quickly; robot heads move slowly or not at all. A policy trained on shaky head video may see motion it never produces.
• Field of view and lens. Wearable cameras, a robot's stereo pair and a wide-angle fisheye differ in view angle and distortion.
• Resolution, frame rate and exposure. These affect motion blur and what small parts look like.
• Body in view. The robot's own arms and hands appear in its images in specific places; human arms and hands appear differently.

The wider embodiment problem is in the human-to-humanoid embodiment gap. This page is the capture-side response to the camera part of that gap.

What published work does about it

Published humanoid work treats egocentric human data as a form of cross-embodiment training data and spends effort on alignment. Humanoid Policy ~ Human Policy collects human demonstrations on robot-aligned tasks with hand-finger 3D poses from consumer VR devices, and trains a model with a unified state-action space for humans and humanoids; it reports about 27,000 human demonstrations against about 1,500 humanoid demonstrations and better generalization and robustness with the human data (Qiu et al., 2025). EgoMimic captures human demonstrations with Project Aria glasses and co-trains a single policy on human and robot data, reporting 34-228% improvement over robot-only data (Kareer et al., 2024).

GR00T N1 places human egocentric video at the base of its data pyramid and real-robot teleoperation at the top, with real-world evaluation on a humanoid (NVIDIA, 2025). The pattern: human video supplies breadth; robot data from the target body anchors it. Viewpoint matching narrows the distance the model has to bridge, but these studies do not show that matching removes the need for robot data.

What matched-viewpoint capture can and cannot support

Can support

• Co-training where human and robot observations are fed to the same encoder.
• Perception pretraining closer to deployment imagery.
• Evaluation of how much viewpoint offset hurts, if you record the same task at several offsets.

Cannot support

• Robot actions. A matched camera still has a human behind it.
• Identical dynamics. Even with matched height, human head motion remains.
• Forces and contact state.
• Transfer to a different robot model with different sensors. If you target several robots, a single matched rig matches only one.

Parameters to specify

Supply the robot's calibration files if you can: intrinsics, extrinsics relative to the torso, and sample frames from the robot's own cameras. A set of reference images from the robot's perspective lets a collection team test their rig before recording.

Options when exact matching is impractical

1. Chest or helmet mount at robot height. Cuts height and tilt differences but changes the feel of work. Test demonstrator comfort and safety.
2. Fixed-pose head. Instruct demonstrators to keep their head orientation within a range, accepting less natural behavior.
3. Over-capture and correct. Record wider and use calibration plus IMU logging to reproject toward the robot's view in post-processing. This approximates, and large rotations are not recoverable.
4. Multiple viewpoints per task. Record the same task at two or three heights so the model sees variation.
5. Robot-side capture for a calibration set. Pair a small robot dataset recorded through the robot's own cameras with human data; this lets you measure the gap directly.

None is free. Option 3 depends on camera tracking quality, and option 1 limits who can wear the rig.

Illustrative viewpoint specification

# Illustrative only - synthetic values; supply your robot's real numbers
target_robot_cameras:
  head_stereo:
    mount_height_m: 1.55            # example; standing
    pitch_deg: -15 to -35           # example operating range
    fov_h_deg: 90
    resolution: 1280x720 @ 30 fps
    baseline_mm: 60
recording_rig:
  position: chest + helmet pair, per-demonstrator measurement recorded
  tolerance: height +/- 5 cm; pitch +/- 10 deg (buyer-set)
  logging: IMU orientation per frame, calibration file per day
  reference_frames: 20 robot-view images supplied by buyer for rig check
delivery:
  raw_video: original resolution
  corrected_video: optional reprojected view, correction documented
  metadata: [demonstrator_height_cm, mount_offset, site_017, take_id]

Acceptance tests

1. Record a calibration scene with the rig and the robot cameras; compare horizon, object scale and field of view.
2. Check measured mount height and pitch per demonstrator against the tolerance.
3. Sample frames in which the human body occludes the scene differently from the robot's arms.
4. If you have robot data, train a small encoder on human data and probe how far its features separate human and robot images; use the result to decide whether to adjust the rig. Treat this as a diagnostic, not proof of transfer.
5. Confirm IMU logs line up with video.

Include these in the demonstration video acceptance criteria. Details for head-worn rigs are in the head-mounted camera capture specification, and wrist views are in wrist-mounted cameras on human demonstrators. For the decision between first- and third-person views, see egocentric vs exocentric workplace video.

Permissions and provenance

A head-mounted camera at human height records faces and screens at eye level. Matching the robot does not change that. Ask whether consent covers the specific rig, and whether bystanders are in view. Redaction does not by itself resolve permission questions; see worker consent and identifiable people.

Questions to ask collection teams

1. How will you measure and record mount height and orientation for each demonstrator?
2. Can the rig match our field of view and lens distortion, or only approximate it?
3. How is head motion logged, and can we filter by it?
4. What do you correct in post, and how is the correction documented?
5. Can you test the rig against reference frames from our robot's cameras?
6. How does the rig affect demonstrator comfort, safety and pace?
7. Can you record at more than one viewpoint offset for the same task?

Discuss your requirements

If your project needs demonstrations that match a particular humanoid's cameras, describe the sensor geometry and task requirements to SourceX. If no existing asset fits, SourceX can scope a commissioned collection of new recordings; SourceX's role is sourcing, rights review, licensing and delivery coordination, with specialist capture partners engaged case by case. A request does not guarantee a match. Start from the humanoid training data guide and the workplace task demonstration hub; the catalog page on first-person video of skilled manual work describes related footage.

Describe the workplace demonstration data your robotics project needs

Sources

• Humanoid Policy ~ Human Policy — robot-aligned egocentric human demonstrations for humanoids.
• EgoMimic: Scaling Imitation Learning via Egocentric Video — co-training on human and robot data.
• GR00T N1: An Open Foundation Model for Generalist Humanoid Robots — human video at the base of the data pyramid.