Short answer
Manual load handling demonstrations for whole-body humanoids show how people lift, carry, turn and set down loads, and they need full-body pose, load description and stance labels, not only hand views. Video of lifting can inform whole-body motion priors and ergonomic annotation, but it carries no load mass, joint torques or balance data from the robot.
Task anatomy
Carrying a box looks like one task and is really a chain of whole-body phases.
- Assess. Judge size, weight, handles and path. Workers shift a box to test it before lifting.
- Position. Approach, set feet, align the body to the load.
- Lift. Bend knees and hips, grip, raise to carrying height; sometimes a two-person lift.
- Carry. Walk with the load, turning corners and passing through doors, sometimes up steps.
- Place. Lower to a shelf, cart or floor, with control and releasing grip.
- Reset. Recover posture, check the load, return.
Variation is large: load size and mass, hand couplings (handles, cut-outs, none), height at origin and destination, distance carried, frequency, one- versus two-person handling, and the terrain between. Failure modes include loads that shift, slips, drops, blocked sightlines and unplanned stops. Workers use compensating strategies, such as resting a box on a hip, that a robot may not be able to copy.
Why whole-body data differs from hand-focused data
Most demonstration pages in this cluster focus on hands and objects. Here the load changes the entire body: center of mass shifts, stance widens, gait slows, and the arms act more like struts than manipulators. A humanoid executing the same task must coordinate legs, torso and arms, with balance as a constraint.
Published humanoid work reflects this. HumanPlus trained a low-level policy in simulation with reinforcement learning on 40 hours of human motion data, enabling real-time shadowing of human body and hands from a single RGB camera, with skill policies learned by behavior cloning at up to 40 demonstrations per task and 60-100% success on a customized 33-DoF, 180 cm humanoid (Fu et al., 2024). OmniH2O uses kinematic pose as a universal control interface for whole-body teleoperation by VR headset, verbal instruction or RGB camera, and releases a whole-body control dataset of 6 everyday tasks (He et al., 2024).
Both results show that human body pose is a usable interface to humanoid control. Neither says that video of workers carrying loads suffices to train a humanoid to carry. Pose from video is an input to a motion pipeline whose other stages are simulation, retargeting and robot data. For the broader gap, see the human-to-humanoid embodiment gap.
Capture specifics
| Stream | Purpose | Notes |
|---|---|---|
| Multi-view video | Full-body pose, load, path | At least two views at different angles; more for turns |
| Body pose | Joint kinematics over time | Motion capture, IMU suit, or estimation from video |
| Load description | Mass, dimensions, coupling | Weigh and photograph every item |
| Floor plan or route | Distances, turns, obstacles | Mark origin and destination heights |
| Egocentric camera | Worker's view of path and load | Head-mounted; sightline blocked by tall loads |
| Optional: foot pressure | Balance, weight transfer | Insoles; instrumented capture |
Body models matter for how pose is stored. SMPL-X is a unified body model with fully articulated hands and expressive face, learned from thousands of 3D scans, and SMPLify-X fits it from a single RGB image (Pavlakos et al., 2019). A buyer can ask suppliers to deliver pose in an agreed skeleton or body model, with per-frame confidence. For where pose comes from and how accurate it is, see full-body pose from workplace video.
Heavy loads hide the body: a large box occludes the torso and arms from the front, so side and rear views carry the pose estimate. Plan the camera layout for the worst-case load, not the lightest.
Ergonomic reference data
Occupational safety guidance gives a vocabulary for describing lifts. The NIOSH Revised Lifting Equation manual explains how to estimate the risk of low back disorders for manual lifting, including asymmetrical lifts and lifts with less than optimal hand-object coupling (NIOSH, 2021). Task variables it works with, such as where the load starts and ends, how far it moves, how often, and how well the hands can grip it, make good metadata for each demonstration.
Using these fields does two things. It lets you filter and balance the dataset, for instance by coupling or asymmetry. It also lets you label risk as an annotation on human lifts. It does not set limits for a robot, whose structure and failure modes differ, and it is not an assessment of any worker's safety. Treat ergonomic scores as descriptive context.
What the data cannot support
- Load and torque. Video does not tell you how heavy an item was or what joint torques the lift required. Weigh loads and record them.
- Balance and contact. Ground reaction forces and center-of-mass motion need force plates or instrumented capture.
- Robot capability. A person's lift technique may exceed a humanoid's payload, speed or torque limits; actions have to be re-planned for the robot's body. See retargeting human hand demonstrations for the hand side of that problem and pushing carts and trolleys for a related whole-body task.
- Safety. Do not infer that a technique captured on video is safe. Workers' habits include shortcuts.
Illustrative specification
# Illustrative only - synthetic values
task_family: "carry boxes between stations"
loads: 20 boxes, 2-18 kg (weighed), 3 sizes, handles/cutouts/none
routes: 3 routes per site (flat 15 m, one door, one ramp)
views:
fixed: 3 cameras, 1080p, 30 fps, overlapping coverage
head_mount: 1080p, 30 fps
pose:
source: imu_suit 60 Hz + video estimate
format: SMPL-X parameters, per-frame confidence
labels:
phases: [assess, position, lift, carry, place, reset]
metadata: [mass_kg, size_cm, coupling, origin_height, dest_height, distance_m, handler_count]
safety:
lift limits set by site safety lead; workers may decline any load
excluded: joint torques, ground reaction force, robot actions
Quality, suppliers and permissions
Acceptance examples: every run has a weighed load record; pose confidence reported; full-body visibility in a set share of frames; handoff and two-person lifts labeled; failures retained. Suppliers include warehouses, movers, facilities and delivery operations; see warehouse and fulfillment data and furniture moving and room setup. Full-body capture identifies people more readily than hand views, so consent matters more; see worker consent and identifiable people. Workers must never be asked to lift beyond their normal practice for the camera.
Questions to ask suppliers
- Can each load be weighed and described before the run?
- What pose capture is available, and what accuracy is stated?
- Can side and rear views be covered despite aisle layouts?
- How are two-person lifts and handoffs recorded?
- What safety rules govern the session, and who can stop it?
- Which routes, heights and terrain are available?
- How are worker consent, withdrawal and face handling addressed?
- Are failures and near-misses retained or removed?
Discuss your requirements
If you need whole-body carrying demonstrations, describe the loads, routes and pose requirements to SourceX. SourceX reviews the requirement and follows up about potential sources; instrumented capture is scoped case by case, and a request does not guarantee a match. The demonstration hub and the humanoid training data guide show how this task fits the wider set.
Discuss your humanoid training-data requirements with SourceX
Sources
- HumanPlus: Humanoid Shadowing and Imitation from Humans — HumanPlus human motion data and humanoid shadowing.
- OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning — OmniH2O kinematic pose interface and whole-body dataset.
- Expressive Body Capture: 3D Hands, Face, and Body from a Single Image — SMPL-X body model with articulated hands.
- Applications Manual for the Revised NIOSH Lifting Equation (DHHS (NIOSH) Publication No. 94-110) — NIOSH Revised Lifting Equation manual.