Short answer
Doors, drawers and cabinets are articulated objects: each has a joint type, an axis, a handle and sometimes a spring, latch or lock. Demonstrations should show many fixtures with labeled joint and handle types and record the approach, grasp, open, pass-through and close. Video does not capture resistance, so closers, fire doors and heavy drawers need extra data.
Why this prerequisite matters
A mobile humanoid in a workplace meets articulated fixtures constantly: doors between rooms, supply cabinets, filing drawers, refrigerators, lockers, equipment panels. If it cannot operate them, it cannot reach what it came for. This is why such tasks appear as building blocks in many workflows rather than as an end in themselves; see mobile manipulation demonstrations for how movement joins them.
Simulation has made articulated objects a standard research topic. SAPIEN is a physics-based simulator built around a large collection of articulated objects for household robotics, with benchmarks for part detection and motion recognition and manipulation tasks tackled with heuristics and reinforcement learning (Xiang et al., 2020). That helps with joint structure, but real workplaces include worn hinges, stiff closers and fixtures nobody modeled. Recorded work fills that gap.
Task anatomy
| Fixture | Joint | Handle or actuator | Complications |
|---|---|---|---|
| Hinged door | Revolute, vertical axis | Lever, knob, push bar, pull bar | Door closers, swing direction, badge reader, latch |
| Double door | Two revolute | Panic bar, flush pull | Sequence of leaves, one fixed |
| Sliding door | Prismatic | Recessed pull, rail | Tracks, automatic sensors |
| Drawer | Prismatic | Bar pull, knob, recessed | Full-extension slides, soft-close, lock |
| Cabinet door | Revolute | Knob, magnetic catch, push latch | Magnet resistance |
| Refrigerator or locker | Revolute | Pull handle, latch | Seals, thermal handles |
Container lids and packaging follow different mechanics and are covered in opening containers, lids and packaging.
Steps: locate the fixture and handle; approach; reach; grasp or push; actuate the latch; move through the articulation; hold or pass through; release; close or leave open. For doors, the person usually does something else at the same time, such as carrying, pushing a cart or keeping a door open for others. That makes "hands busy" a first-class condition: opening a door with an elbow, hip or foot while carrying is common and is not captured by clean tabletop demonstrations. See pushing carts and trolleys.
What a dataset needs
Fixture catalog. Photograph and describe every fixture: type, joint, handle, hinge side, swing direction, closer, lock, width, and mounting height. A video clip with no fixture record is hard to use.
Variety. The number of distinct fixtures matters more than the number of repetitions of one door. Robot-collected datasets report diversity the same way: BridgeData V2 states 60,096 trajectories across 24 environments (Walke et al., 2023), giving both a trajectory count and an environment count. Ask suppliers for the equivalent pair, fixtures and sites, and weight the second heavily.
Close and wide views. A close-up of hand and handle plus a wide view that shows the swing and clearance. At the research end, ARCTIC used motion capture, eight allocentric views and one egocentric view to obtain 3D hand and object meshes and contact for bimanual articulation of objects such as laptops and scissors (Fan et al., 2023). Workplace capture rarely reaches that level, and does not need to for recognition and step labels.
Labels. Joint type, axis side, handle type, door angle over time if estimable, phase, outcome, assistive body part, and whether the fixture was held open.
Sequencing a collection
Fixtures differ in how much of the task is about the fixture and how much about what happens after it opens. Plan the collection in three layers. The first is the bare operation: approach, open, release, close, with free hands, many fixtures and little else. The second adds context: carrying a box through a door, pulling a drawer while holding a folder, opening a cabinet and taking out a labeled item. The third connects the operation to a larger workflow, such as entering a supply room, restocking and leaving it as found.
The first layer is cheap and benefits from breadth. The second needs scripted conditions and more careful safety. The third overlaps with long-horizon task demonstrations and should be scoped there. Doing the layers in that order lets a pilot reveal which fixtures cause trouble before the more complex runs are recorded.
Expect surprises in the survey phase. Doors that look the same may open the opposite way; a cabinet may have a hidden catch; a drawer may need a lift before it slides. A walk-through of each site with a facilities contact, noting the fixtures and any quirks, saves recording time and is often the point at which the supplier first learns what you actually need.
What the data cannot support
- Forces. Door closer resistance, magnetic catch strength and drawer friction are not visible. A humanoid hand must learn them from interaction.
- Access control. Badge readers, keypads and locks involve credentials. Recordings that show codes, badges or key positions are a security issue. Redact or avoid them; see redaction in demonstration video.
- Robot geometry. Doorway width and handle height matter differently to a humanoid with a different shoulder width, reach and arm length.
- Safety systems. Fire doors, alarms and restricted rooms bring rules that a policy must respect.
Illustrative specification
# Illustrative only - synthetic values
task_family: "operate workplace fixtures"
fixtures: 60 distinct, from 4 sites (30 doors, 20 drawers, 10 cabinets)
fixture_record: [type, joint, handle, hinge_side, closer, lock, width_cm, height_cm, photo]
runs_per_fixture: 3, varied hand-busy condition (free, carrying, pushing cart)
views:
close_up: 1080p, 60 fps
wide: 1080p, 30 fps, shows swing
head_mount: 1080p, 30 fps
labels: [phase, joint_open_angle (estimated), outcome, body_part_assist, held_open]
excluded_zones: [secured areas, code entry, tenant spaces]
not_included: resistance, torque, robot joint states
Quality and acceptance criteria
Examples: each fixture has a complete record; at least a set share of fixtures appear at more than one site; hand-busy and free-hand conditions both present; failures such as door closers slamming and drawers jamming kept; the handle visible in the approach phase of nearly all runs. Check a sample by recomputing the joint type from the video and comparing it with the record.
Supplier types and permissions
Facilities teams, supply rooms, kitchens and offices have the fixtures; hotels and hospitals have the variety. Recording takes place in spaces where the site owner controls access, and where other people use the doors. Ask who approves recording, which areas are off limits, and how bystanders will be handled. For permission patterns in live sites, read recording protocol on live work sites and worker consent and identifiable people.
Questions to ask suppliers
- How many distinct fixtures, by type, can be recorded?
- Can you provide a fixture record with each clip?
- Can runs include hands-busy conditions safely?
- Which areas are excluded for security or safety?
- How are access codes, badges and keys kept out of frame?
- Are door closers, locks and sliding doors included, or only simple hinged doors?
- Do you allow close-up capture of handles?
- Who approves recording at each site?
Discuss your requirements
If you need articulated-fixture demonstrations, describe the fixtures, sites and label needs to SourceX. SourceX reviews the requirement and follows up about potential sources; a request does not guarantee a match. Read the demonstration hub and the humanoid training data guide for neighboring pages.
Describe the workplace demonstration data your robotics project needs
Sources
- SAPIEN: A SimulAted Part-based Interactive ENvironment — SAPIEN articulated object simulator.
- ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation — ARCTIC bimanual articulated hand-object capture.
- BridgeData V2: A Dataset for Robot Learning at Scale — BridgeData V2 trajectory and environment counts.