Short answer
A multi-view recording captures the same task attempt from several cameras on a shared clock, usually one or more egocentric cameras plus fixed external ones. Useful rigs specify camera roles, calibrated intrinsics and extrinsics, a synchronization method with a measured offset, and consistent file naming. Without synchronization and calibration, extra views add storage cost, not information.
What a second camera buys you
One camera answers one question: what does the scene look like from here? A second answers a different one. A head camera shows what the demonstrator attends to. A fixed camera shows body posture, workspace layout and the object after the hand leaves the frame. A wrist camera shows contact. Whether a buyer needs several of these depends on the model. A head-view-only policy needs no side view at inference but may still benefit from side views during training, labeling or evaluation.
The perspective choice itself is covered in egocentric vs exocentric workplace video. This page assumes you want several views and are specifying the rig; the single-wearer case is in the head-mounted camera capture specification.
Published rigs as reference points
Published datasets show the range of rig designs:
- Ego-Exo4D records skilled activities such as bike repair simultaneously from an egocentric camera and multiple exocentric cameras, with multichannel audio, eye gaze, 3D point clouds, camera poses and IMU. It reports 740 participants in 13 cities and 1,286 hours of video (arXiv v4) (Grauman et al., 2023).
- Assembly101 records toy-vehicle assembly and disassembly with 8 static and 4 egocentric cameras across 4,321 videos, unscripted and including mistakes (Sener et al., 2022).
- IKEA ASM uses three camera views for 371 furniture assembly videos, with depth, atomic action, object segmentation and human pose annotations (Ben-Shabat et al., 2020).
These are research rigs in controlled or semi-controlled settings. Workplace rigs have to fit around staff, safety rules and production pace, which usually means fewer cameras and simpler mounts. A reasonable question for any collection team is which of these reference designs they would adapt, and what they would remove.
What multi-view data can and cannot support
Can support
- Cross-view training and labeling: use the side view to label what the head view occludes.
- Body pose and hand pose estimation with triangulation, if cameras are calibrated.
- Evaluation of a head-view model against an external view of the same attempt.
- Mistake and outcome labels checked from a clear angle.
Cannot support
- Robot actions, joint states or forces. Several cameras do not change this.
- A match to every robot viewpoint. The cameras cover the angles you chose, not arbitrary ones.
- Dense coverage of long, moving tasks across rooms. Fixed rigs cover one workspace.
- Reliable cross-view geometry without calibration and synchronization.
Synchronization: what to specify
Unsynchronized views are a common failure in multi-camera data. Two cameras that each report "30 fps" drift apart over minutes, and consumer devices apply different startup delays.
Options, from lightest to heaviest:
| Method | How it works | Typical use |
|---|---|---|
| Event sync | A visible, audible or flash event at start and end of each take | Cheap; handles offset and drift if events are at both ends |
| Shared timecode | Cameras take timecode from a common source | Professional cameras |
| Hardware trigger | One signal starts exposure on all cameras | Machine-vision rigs |
| Network time | Devices share a clock over Ethernet using a protocol such as the Precision Time Protocol, which IEEE 1588-2019 defines and which can reach sub-microsecond accuracy (IEEE, 2019) | Wired sensor rigs |
Match the method to the tolerance. A task analysis of slow assembly steps tolerates a larger offset than a fast manipulation model that learns from sub-frame hand motion. State the tolerance as a buyer-set requirement, and ask the collection team to report the measured offset per take, not just the method.
Calibration and metadata
- Intrinsics: focal length, principal point and distortion per camera, re-checked when lenses are touched.
- Extrinsics: pose of each camera in a shared frame, from a calibration target at the start of each session and after any bump.
- Moving cameras: per-frame pose from SLAM or IMU; record the dropout rate.
- Naming: one take ID shared by every stream, with a camera role field.
- Time: per-frame timestamps, not only a nominal frame rate.
Illustrative rig and delivery specification
# Illustrative only - synthetic values the buyer must adapt
take_id: site_017_take_00231
cameras:
- role: ego_head # head-mounted, 1080p, 30 fps
- role: exo_front # fixed tripod, 1080p, 30 fps
- role: exo_side_left # fixed tripod, 1080p, 30 fps
- role: wrist_right # optional, 1080p, 60 fps
sync:
method: hardware trigger or event sync at both ends of take
measured_offset_ms: reported per camera pair per take
calibration:
intrinsics: per camera, per day
extrinsics: static cameras per session; moving cameras per frame (SLAM)
files:
video: mp4 per camera, constant frame rate
timestamps: csv per camera
calib: json per session
labels: [task_id, step_segments, outcome, operator_A12]
acceptance:
max_sync_offset: buyer-set (ms)
min_views_usable: buyer-set share of takes with all views usable
Storage and operational impact
Each added view multiplies storage, transfer and QA effort. A 4-camera rig produces four times the bytes before depth or audio; plan the pipeline with storage and transfer planning for video before approving the rig. Depth cameras raise the load further; see RGB-D capture. If robot logs or machine logs must be aligned too, read aligning video with system logs.
Fixed cameras on live sites bring site questions of their own: mounting approval, bystanders, areas that must stay out of frame. See recording protocols for live work sites.
Acceptance tests for a pilot
- Play all views side by side on a take with a sharp event and verify alignment frame by frame.
- Triangulate a known-size object across two views; check size error.
- Compare intrinsics at start and end of a session.
- Check that no view is blocked during key steps in the sampled takes.
- Confirm file naming lets you assemble every stream for a take without manual matching.
Run them as part of in-collection quality control.
Questions to ask collection teams
- Which synchronization method do you use, and what offset have you measured on a recent shoot?
- How are cameras calibrated, and how often?
- What happens when a camera is bumped mid-session?
- How many takes lose one or more views, and how is this reported?
- How do you handle bystanders in fixed views at a live site?
- What is the data volume per task-hour for the proposed rig?
- Can you deliver the raw streams as well as any processed versions?
Discuss your requirements
If your task needs several synchronized views, describe the rig, task and permitted uses to SourceX. If no existing asset fits, SourceX can scope a commissioned collection of new recordings. SourceX's role is sourcing, rights review, licensing and delivery coordination; specialist capture partners can be engaged case by case, and a request does not guarantee a match. See the humanoid training data guide and the workplace task demonstration hub, or the catalog page on video recordings.
Describe the workplace demonstration data your robotics project needs
Sources
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives — simultaneous ego and exo recording with additional sensor channels.
- Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities — 8 static and 4 egocentric cameras.
- The IKEA ASM Dataset — three-view assembly recording with depth and annotations.
- IEEE 1588-2019 Precision Clock Synchronization Protocol — networked clock synchronization.