Short answer
Teleoperation data comes from a real robot driven by a trained operator, not from business archives, so the specification must define the embodiment, the operator interface, the sensors, what is logged as actions and at what rate, calibration, synchronization, and how episodes are reset, checked and rejected. Leave any of these open and the dataset may not match your robot or your policy.
What you are specifying
A teleoperated episode records what the robot sensed and what the operator commanded through it, so it matches the robot's own observation and action spaces by construction. That is why it can train a control policy directly, and why it is expensive: every episode needs hardware, a trained operator and a reset. Human footage cannot supply these fields; see human demonstrations vs teleoperated robot demonstrations for the comparison and handheld gripper demonstrations for a cheaper middle route.
Supply is specialist. Teleoperated data comes from robotics labs, collection vendors and teams that run robots at scale, or from a business that already operates and logs its own robots. Operating businesses can also host a collection by providing sites, tasks and objects. If no existing asset fits, SourceX can scope a commissioned collection of new recordings. SourceX's role is sourcing, rights review, licensing and delivery coordination; it can also arrange annotation and engage specialist capture partners, scoped case by case.
Illustrative collection specification
# Illustrative only — synthetic values; every item is a decision the buyer sets
collection: TELEOP_2026_002
embodiment:
robot: humanoid_X (synthetic) # make, revision, firmware, controller version
arms: 2 x 7-DoF
hands: 6-DoF dexterous hands # or parallel grippers
base: fixed | wheeled | legged
kinematic_description: supplied (URDF or equivalent)
operator_interface:
type: vr_headset_and_controllers # alternatives: leader-follower arms, exoskeleton, mocap retargeting
operator_visual_feedback: stereoscopic head-slaved view
retargeting: documented; both operator-device pose and robot command logged
operators:
count: set by diversity plan
training: documented onboarding; proficiency tier recorded per operator
ids: operator_A12 style, one per episode
tasks:
list: [tote_unpacking, shelf_facing] # plus success criterion per task
success_criterion: written, checkable from logged data or video
scene_randomization: object set, placement and lighting schedule agreed
resets: scripted; reset verified before each episode starts
failures: kept and flagged (do not silently discard)
language: one instruction per episode
sensors:
cameras: [head_stereo, wrist_left, wrist_right, third_person]
video: constant frame rate, declared resolution, lossless or declared codec
depth: yes | no
proprioception: joint position and velocity; torque if available
hand_or_gripper_state: logged
force_or_tactile: logged if the robot has it
actions:
logged: commanded targets and measured state at the same step
space: defined per the action-space page
control_rate_hz: set by the buyer
calibration_and_sync:
intrinsics_extrinsics_hand_eye: per session, files delivered
clock: single shared clock; max offset reported
episode_qa:
automated: [dropped_frames, timestamp_monotonic, joint_limits, action_jumps, camera_occlusion]
review: buyer-drawn sample watched by a person
reject_rules: written before collection starts
delivery: robot-trajectory spec; manifest with checksums
rights: operators consented; site permission; third-party objects reviewed
How to fill the blocks
| Block | What to decide | Why it matters |
|---|---|---|
| Embodiment | Exact robot, revision, hands, base and controller | Actions and states only mean something relative to this; a different revision is a different dataset |
| Interface | How the operator drives the robot and what they see | The interface shapes motion style, speed and what tasks are feasible. Open-TeleVision used immersive VR with stereoscopic active visual feedback to mirror an operator's arm and hand movements on two humanoids (Cheng et al., 2024); OmniH2O used kinematic pose as the control interface, driven by VR headset, verbal instruction or RGB camera (He et al., 2024) |
| Operators | Who is recorded, how trained, how graded | Quality varies by operator. In one study results depended on demonstration quality, and its 300-demonstration multi-operator sets from 6 operators were grouped by proficiency (Mandlekar et al., 2021) |
| Protocol | Tasks, success criteria, randomization, resets, failures | Without resets and randomization schedules, episodes cluster in one easy configuration |
| Sensors | Cameras, depth, proprioception, force | Match your policy's observation space; add what the task needs, such as wrist views for fine manipulation |
| Actions | What is commanded, in which space, at what rate | See action spaces and control frequency |
| Calibration and sync | Intrinsics, extrinsics, hand-eye, shared clock | See calibration requirements and sensor synchronization requirements |
| Episode QA | Automated checks, human review, reject rules | The cheapest place to find a bad station is before the lot is delivered |
What published programs used
These are reference points for what specific studies did. They are not minimums, and the right figures depend on your architecture, task, embodiment and intended use.
| Program | What it reports |
|---|---|
| DROID | About 76,000 trajectories and 350 hours across 564 scenes and 86 tasks, from 50 collectors over 12 months; one rig replicated at 13 institutions; Meta Quest 2 controller teleoperation at 15 Hz; camera calibration, depth and language per episode (Khazatsky et al., 2024) |
| Mobile ALOHA | Whole-body teleoperation with a tethered operator; 50 demonstrations per task; co-training with static bimanual data raised success by up to 90% (Fu et al., 2024) |
| ACT | Six real-world fine-grained tasks at 80–90% success from about 10 minutes of demonstrations (Zhao et al., 2023) |
| GR00T N1 | Real-robot teleoperation at the top of a data pyramid; neural trajectories expanded 88 hours of teleoperation data to 827 hours (NVIDIA, 2025) |
| Open X-Embodiment | More than 1 million trajectories pooled from 22 embodiments, with positive transfer across platforms (Open X-Embodiment Collaboration, 2023) |
Pooling across platforms helped in that study, but it does not tell you how much data your own embodiment needs; see cross-embodiment robot datasets.
Acceptance tests to attach
| Test (illustrative) | Pass criterion the buyer might set | Evidence requested |
|---|---|---|
| Timing | Timestamps strictly increasing; measured rate within a set tolerance of the declared rate | Timing report per episode |
| Completeness | Every step has all declared streams; dropped-frame share below a set limit | Per-stream counts |
| Action validity | Values within joint limits; no jumps above a set threshold between steps | Range and discontinuity report |
| Calibration | Intrinsics, extrinsics and hand-eye files present for every session | File inventory by session |
| Success labels | Labeled outcome agrees with an independent reviewer on a buyer-drawn sample | Audit sheet |
| Reset fidelity | Initial scene matches the randomization schedule | Start-frame review |
| Operator coverage | Episodes spread across operators as agreed, none dominant | Counts by operator and task |
| Rights | Operator agreements and site permissions on file | Register extract |
The general delivery process is in acceptance testing for dataset deliveries, and the episode fields in the robot trajectory dataset specification.
Permissions
Rights questions are usually narrower than for workplace footage because no worker's job is being filmed, but they are not absent. Operators appear, and their hands, voices and faces may be recorded; see biometric privacy and workplace video. Collection at a business site adds site permissions, bystanders, confidential content in the background and third-party objects; see confidential business information in footage. Check who owns the episodes, derived data and any controller or retargeting software the provider used. This is not legal advice.
Limits and preparation
- No business-process context at scale. A teleoperated lab or cell shows robot behavior on set tasks. It does not carry the variety of live operations; combine with human demonstration video where that matters.
- Embodiment lock-in. The data fits the robot and interface it was recorded on. Another robot needs retargeting or fresh collection.
- Operator style is in the data. Speed, smoothness and strategy come from the operators; record who they were.
- Preparation is the cost. Hardware setup, calibration, synchronization and episode QA decide whether the delivery is usable. Budget for them rather than only for operator hours.
- A pilot comes first. Collect a small lot, train on it and inspect failures before committing to volume.
Questions to ask providers
- Which robot, revision and controller version will be used, and can you collect on mine?
- Which teleoperation interface do you use, and what do operators see and feel?
- How many operators will record my tasks, how were they trained, and will each episode carry an operator ID?
- What exactly is logged as action: operator-device pose, retargeted command, measured state, or all three?
- At what rate are actions, joint states and each camera logged, and how do you report offsets between them?
- How and when is calibration done, and what happens when a camera is bumped?
- What automated and human QA runs on every episode, and what are your reject rules?
- How do you handle failed attempts and recoveries?
- Who owns the episodes and any derived data, and what consent covers the operators?
Discuss your requirements
Describe the robot, tasks, sensors and action logging you need, and say whether existing robot logs might fit. Discuss your requirements with SourceX and it will investigate whether US businesses hold suitable material. If no existing asset fits, SourceX can scope a commissioned collection of new recordings. SourceX reviews the requirement and follows up about potential sources; a request does not guarantee a match, and nothing is contracted until a supplier agrees. More context: the procurement hub, the humanoid training data guide and the catalog's robotics and embodied AI use case.
Discuss your humanoid training-data requirements with SourceX
Sources
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset — scale, rig, controller, rate and per-episode metadata of a multi-site teleoperation program.
- Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation — whole-body teleoperation, 50 demonstrations per task, co-training result.
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware — ACT and its small-demonstration results.
- Open-TeleVision: Teleoperation with Immersive Active Visual Feedback — immersive teleoperation of two humanoids.
- OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning — kinematic-pose interface and released dataset.
- What Matters in Learning from Offline Human Demonstrations for Robot Manipulation — dependence on demonstration quality and operator proficiency.
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots — teleoperation in the data pyramid.
- Open X-Embodiment: Robotic Learning Datasets and RT-X Models — pooled multi-embodiment corpus.