Workplace Movement Data: Walking With Loads, Stairs and Ladders

Why locomotion buyers need more than video

Much of the workplace footage that exists was recorded to watch hands: a bench, a shelf, a counter. Locomotion is the opposite problem. The signal lives in the legs, pelvis, trunk and arms, in how weight shifts when a person carries a box up a flight of stairs, and in what the feet do on a rung. A camera mounted for process monitoring usually crops the legs, sits too far away to resolve foot placement, or loses the subject behind a stair rail.

If your team trains a whole-body controller, a locomotion prior or a motion generator, the question is what kind of human motion data feeds it and what must still come from simulation or the robot. This page covers the human-motion side. For the physical reasons the mapping is imperfect, see the human-to-humanoid embodiment gap.

What published work shows human motion data can support

Human motion data is already used as a prior for humanoid control, in several forms:

• Marker-based motion archives. AMASS unifies 15 optical marker-based motion-capture datasets into one body-model representation, covering more than 40 hours, over 300 subjects and over 11,000 motions (Mahmood et al., 2019). It is a reference for what a clean, calibrated, body-model-fitted corpus looks like, and it is built from laboratory-style capture rather than workplaces.
• Tracking policies trained on human motion. HumanPlus trained a low-level humanoid policy in simulation with reinforcement learning on 40 hours of human motion data, then shadowed a person's body and hands from a single RGB camera (Fu et al., 2024).
• Pose data mined from video. The Humanoid-X work built more than 20 million humanoid poses with text motion descriptions from human videos, through video mining, captioning and human-to-humanoid retargeting (Mao et al., 2024).
• In-the-wild whole-body capture. Nymeria recorded 300 hours of daily activity across 1,200 recordings, 264 participants and 50 locations, pairing full-body motion ground truth with egocentric multimodal recordings and a third-person observer device (Ma et al., 2024).

None of these is a stair-or-ladder workplace corpus, and none contains robot joint commands. They show the shape of the data a locomotion team may want to specify, not evidence that any particular amount covers a task.

Locomotion tasks and what each one demands

For loaded walking in a specific job, the task-level view is in manual load handling and carrying and pushing carts and trolleys. This page stays on the movement itself.

Capture routes and what each produces

Optical motion capture. Reflective markers and fixed cameras give accurate joint positions in a calibrated volume. It works for a stair set or ladder installed in a studio, and rarely for a live building without dedicated setup time.

Wearable inertial sensors. Body-worn units record orientation and acceleration without line-of-sight, so they survive occlusion on a ladder or in a stairwell. They drift, need per-subject calibration poses and give relative rather than absolute position. See body-worn IMU demonstrations.

Multi-camera pose estimation. Several synchronized views let a pose estimator reconstruct the body in 3D without markers. Accuracy depends on camera count, placement and clothing, and failures cluster at occlusions. See full-body pose from workplace video.

Egocentric plus body capture. Head-worn cameras show what the person sees while a second system records the body, which is the structure used in the Nymeria study above. It suits perception-conditioned locomotion, where the robot must decide where to step.

Foot pressure or contact sensing. Insoles or instrumented treads give contact timing and load. Without them, contact labels are inferred or hand-annotated, which is the weakest part of video-only data.

Illustrative specification: loaded stair and ladder session

# Illustrative only - values are synthetic examples, not an available dataset
session: locomotion_pilot_site_017
subjects: 6 adults, varied height and build, ids operator_A01..A06
tasks:
  - loaded_walk: level corridor, 12 m, load = crate with weight logged per trial
  - stair_up_down: 14-tread straight stair, handrail left, with and without load
  - ladder_climb: fixed vertical ladder, 8 rungs, harness and no-harness variants
sensors:
  body: 17 wearable IMUs, per-subject calibration pose at start and end
  video: 4 synchronized exocentric cameras, 60 fps, 1080p, shared clock
  egocentric: 1 head-worn camera, 30 fps
  contact: optional insoles, otherwise annotated heel-strike / toe-off events
labels: task phase, foot-contact events, load weight, stair/rung index, failure flag
metadata: footwear, floor/tread surface, lighting, camera extrinsics, clock offsets
excluded: robot joint targets, torques, balance-recovery commands
rights: model training and internal evaluation; no redistribution

Change every value to match your embodiment. A humanoid with different leg length, foot geometry or ankle range needs the stair geometry and rung spacing recorded so the motion can be scaled or filtered in retargeting.

What this data cannot support

• Robot actions. Human motion contains no joint targets, torque limits or actuator delays. Converting it means retargeting, then tracking in simulation, then robot validation.
• Dynamics. Video-derived motion does not give ground reaction forces, and carried-load weight is only known if someone records it.
• Balance and fall recovery. People rarely fall on camera, and falls are exactly the cases a safe capture protocol avoids. Recovery behavior generally comes from simulation or robot trials.
• Robot sensing. A humanoid perceives stairs through its own cameras or depth sensors from its own height. Human head-height footage does not replicate that view. See matching camera viewpoint to humanoid sensors.
• Generalization claims. One stair design, one ladder and one floor surface cover one case. Treads, rises, rail heights, surface friction and lighting all vary between buildings.

Preparation a buyer should plan for

1. Write down the robot's leg kinematics, foot size and tolerable step height, so capture covers geometry the robot could attempt.
2. Choose the motion representation (body model, joint angles, root trajectory) before capture, and ask whether the supplier's skeleton can be converted to it.
3. Decide how contact will be labeled: sensor-derived, annotated or estimated.
4. Plan a held-out evaluation split by site, subject and stair or ladder design. Do not randomize by clip.
5. Brief demonstrators to work at a safe, natural pace, and have a spotter on ladders. Safety takes priority over capture, and a staged harness-free climb is not an acceptable shortcut. See safety behaviors in demonstrations.

Sourcing routes

Existing footage rarely helps here. A facilities team may hold stair-area security video, but that footage was recorded for other purposes, is wide-angle, and typically has no sensor sync or labels. It can inform environment diversity at best.

Purpose-built capture is the realistic route. Facilities, warehouses and building operators have the stairs, loading docks and ladders, and can host staged sessions. The capture plan, calibration, consent and QA have to be designed in, and rights are usually workable because new consent can be built into the session. Compare the two routes in existing footage vs commissioned recording, and agree site rules using the recording protocol for live work sites.

If you can describe the movements and sensors you need, tell SourceX about the locomotion data requirement and it will investigate whether US businesses hold suitable material or whether a commissioned session is worth discussing.

Questions to ask suppliers

1. Which sensors were used for body motion, and what were their calibration procedures per subject?
2. Are camera clocks and inertial clocks synchronized, and how is drift reported?
3. How are foot-contact events obtained: sensors, annotation or estimation, and what is the error check?
4. Is carried-load weight recorded per trial?
5. What stair geometry (rise, tread depth, rail position) and ladder type were used, and how many distinct installations?
6. Were subjects wearing footwear and clothing typical of the target job?
7. Does the skeleton or body-model format convert to our representation, and who validates the conversion?
8. Are unsafe or aborted attempts logged rather than deleted?
9. What did participants consent to, including use of body-shape data in model training?
10. Can a held-out site or subject set be reserved?

Discuss your requirements

If you are specifying locomotion data for a whole-body model, describe the movements, sensors and permitted uses so SourceX can review the requirement and follow up about potential sources. A request does not guarantee a match, and nothing is contracted until a supplier agrees. For broader context, start at the humanoid training data guide or the workplace task demonstration hub. The catalog entry for first-person video of skilled manual work covers a related collection program, and the robotics and embodied AI use-case page gives wider context.

Describe the workplace demonstration data your robotics project needs

Sources

• AMASS: Archive of Motion Capture as Surface Shapes — scale and structure of a marker-based motion archive.
• Learning from Massive Human Videos for Universal Humanoid Pose Control — humanoid poses derived from human video through retargeting.
• Nymeria: A Massive Collection of Egocentric Multi-modal Human Motion in the Wild — in-the-wild full-body motion with egocentric and observer devices.
• HumanPlus: Humanoid Shadowing and Imitation from Humans — low-level humanoid policy trained on human motion data.