Bimanual Task Demonstrations for Two-Handed Humanoid Manipulation

Why two-handed data is its own purchase

A humanoid has two arms, and much of the workplace work it is bought for needs both. Opening a tote means bracing it with one hand while the other pulls the lid. Fastening a bracket means holding the part in place while the second hand drives the screw. Spreading a towel needs two grasp points pulling in opposite directions.

Data recorded one hand at a time teaches none of this. The model needs to learn which hand takes which role, when the roles switch, and how the timing of one hand depends on the other. Those are properties of the pair, so they must be captured and labeled as a pair.

Workplace video makes this harder. The supporting hand is the one most often lost: it braces a part below the frame, sits behind the object, or is hidden by the worker's body.

Coordination patterns to specify

Write the patterns your target tasks use into the requirement. Each needs different capture and labels.

Label a role per hand for each segment, not only an action per segment. The contact and grasp layers are specified in hand-object interaction annotation; this page adds the pairing between the two hands.

What published bimanual work shows

Dual-arm robot data. Action Chunking with Transformers was introduced on low-cost bimanual hardware and learned 6 fine-grained real-world tasks at 80–90% success from only 10 minutes' worth of demonstrations (Zhao et al., 2023). Mobile ALOHA collected 50 demonstrations per task with whole-body teleoperation and found that co-training with static bimanual data from the earlier setup raised success by up to 90% (Fu et al., 2024), a sign that two-armed data from a related rig can carry over.

At larger scale, AgiBot World reports more than 1 million trajectories over 217 tasks collected on 100 dual-arm humanoid robots with 6-DoF dexterous hands (AgiBot-World Team, 2025). RoboMIND's embodiments include a dual-arm platform and a humanoid with two dexterous hands (Wu et al., 2024). The π0 model was trained on single-arm, dual-arm and mobile manipulator data and shown on dexterous tasks such as laundry folding and box assembly (Black et al., 2024).

Two-hand human data. H2O provides egocentric video with markerless 3D annotations of both hands and the manipulated objects, including per-frame 3D hand poses, 6D object poses and interaction labels (Kwon et al., 2021). ARCTIC holds 2.1M video frames with 3D hand and object meshes and dynamic contact information for two-handed articulation of objects such as scissors and laptops, filmed from eight allocentric views and one egocentric view with motion-capture ground truth (Fan et al., 2023). TACO records 2.5K bimanual motion sequences across tool-action-object combinations from 12 third-person views plus one egocentric view (Liu et al., 2024). That ground truth came from multi-camera capture setups, with motion capture for ARCTIC and TACO, not from workplace footage. At larger scale, EgoDex pairs 829 hours of egocentric video with 3D hand and finger tracking from a headset (Hoque et al., 2025).

The link between the two sides is a shared representation. Humanoid Policy ~ Human Policy recorded hand and finger 3D poses with consumer VR devices on tasks aligned with the robot's, and trained one policy across a unified human and humanoid state-action space (Qiu et al., 2025). Public research sets suit method development rather than workplace coverage, and each carries its own license terms.

Three supply routes

Dual-arm robot or teleoperation episodes. These carry joint states and commanded actions for both arms on a defined embodiment, which is what a control policy trains on. They come from specialist robot-data teams, not from business archives. The specification issues, from action space to synchronization, are covered in teleoperation data collection requirements, and the split between the two supply chains in human demonstrations vs teleoperation.

Commissioned human capture with both hands tracked. Workers perform the target tasks at operating businesses while a headset, multi-view rig or glove system tracks both hands. This is the route to two-handed co-training data with real objects and real working conditions. Capture options and their accuracy limits are compared on the 3D hand pose data page. The site describes a separate physical-world path for new recordings of hands-on work, where each opportunity has its own requirements.

Existing workplace footage. Training videos and body-camera recordings often show two-handed work, rarely with both hands in clear view. They help identify which target tasks are bimanual, how often each pattern occurs and how steps are structured, but they are weak co-training material unless a sample shows both hands visible for most of each task, and even then 3D hand poses have to be estimated after the fact.

Illustrative package: commissioned two-hand demonstrations

# Illustrative only — synthetic specification, not an available dataset
package: bimanual_workplace_demos_v0
tasks: [carton_open_and_unload, linen_bag_tie_and_lift, bracket_hold_and_fasten]
sites: [site_011, site_014, site_019]
demonstrators: 12 (handedness recorded; mix of left- and right-handed)
capture:
  head: egocentric RGB 1080p, 30 fps, with headset SLAM pose
  hands: both hands tracked, 21 keypoints per hand, 30 Hz, per-frame confidence
  exo: two fixed RGB views, calibrated, hardware-synchronized
  sync: one clock for all streams; per-session offset report
labels:
  hand_role: per hand per segment {stabilize, act, carry, transfer, idle, out_of_view}
  contact: start and end per hand per object
  handover: giver, receiver, transfer frame
  outcome: completed / recovered / failed
excluded: robot joint states, commanded actions, grip forces
rights: training and internal evaluation; demonstrator releases cover AI training

Change the keypoint format, frame rates and role vocabulary to match your model. If you plan to retarget human hands to a robot hand, ask for MANO-style parameters as well as keypoints; retargeting human hand demonstrations covers what that step needs.

What the data can and cannot support

Can support: recognizing two-handed tasks and predicting each hand's role; step structure for bimanual plans; co-training a shared human-humanoid representation when both hands are tracked in 3D on tasks that match the robot's; evaluation sets that test whether a model assigns roles and handovers correctly.

Cannot support on its own: dual-arm joint commands, the grip force each hand applies or how load is shared between arms. Human hands also differ from robot hands in degrees of freedom, strength and reach; the human-to-humanoid embodiment gap explains why that limits direct transfer. Contact-heavy bimanual tasks, such as seating a part with both thumbs, need force or tactile data as well.

Quality and acceptance checks

Set thresholds yourself; the checks below say what to measure.

• Both hands tracked. Share of each segment in which both hands have valid tracking above your confidence floor.
• Identity swaps. Left and right hand labels exchanged mid-sequence; count them on a sampled set of sessions.
• Inter-hand timing. Clock offset between streams and between the two hand tracks, reported per session.
• Role agreement. A second annotator labels hand roles on a subset; disagreement stays within your tolerance.
• Demonstrator mix. Handedness, experience and glove use recorded for every session.
• Failures included. A buyer-set share of episodes with slips, drops and regrasps, kept and labeled.

Questions to ask suppliers

1. Which of the requested tasks require both hands, and how did you decide?
2. How are both hands tracked, and what happens to tracking when one hand is occluded by the object or the body?
3. What share of frames in a sample has valid tracking for both hands at once?
4. Are all streams, including both hand tracks, on one clock, and how is the offset measured?
5. How is each hand's role labeled, and what is the agreement between annotators?
6. How many demonstrators are left-handed, and is handedness recorded per session?
7. Are slips, drops and regrasps kept in the delivery or cut out?
8. Do demonstrator releases cover AI training and licensing to third parties?
9. For robot episodes, which embodiment, action space and control frequency were used for each arm?

Discuss your requirements

If you need two-handed data for a humanoid, describe the tasks, coordination patterns and capture you need to SourceX, along with the uses you want licensed. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. SourceX's role is sourcing, rights review, licensing and delivery coordination; it can also engage specialist capture partners, scoped case by case. It reviews each requirement and follows up about potential sources; a request does not guarantee a match.

For the wider map, see the workplace task demonstration hub and the humanoid training data guide. Deformable two-handed work is treated as a task on garment folding and textile handling, and the catalog entry for first-person video of skilled manual work describes the related asset type.

Describe the workplace demonstration data your robotics project needs

Sources

• Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware — fine-grained bimanual tasks learned from a small demonstration budget.
• Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation — demonstrations per task and co-training with earlier bimanual data.
• AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems — large-scale data from dual-arm humanoids with dexterous hands.
• RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation — teleoperated data that includes dual-arm and dual-hand humanoid embodiments.
• π0: A Vision-Language-Action Flow Model for General Robot Control — generalist policy trained with dual-arm data on dexterous tasks.
• H2O: Two Hands Manipulating Objects for First Person Interaction Recognition — 3D annotations of both hands and manipulated objects in egocentric video.
• ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation — bimanual manipulation of articulated objects with motion-capture ground truth and contact.
• TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding — bimanual tool-action-object sequences from multi-view capture.
• EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video — egocentric video with 3D hand and finger tracking.
• Humanoid Policy ~ Human Policy — shared human-humanoid state-action space for two-handed tasks.