Short answer
Each-picking demonstrations record people taking single items from totes, shelves or flow racks and placing them into order containers. For a humanoid program they show where hands go, how people choose a grip in clutter and how exceptions are handled. Footage alone has no robot actions or forces; scan and weight events can supply labels, but fixed overhead cameras rarely show hands closely enough. Fulfillment operators and 3PLs are the main suppliers.
The buyer's problem
Order picking is the most labor-intensive and costly warehouse activity in a classic review, which cites an estimate of as much as 55% of warehouse operating expense (de Koster et al., 2007). Picking single items, or "eaches", for customer orders is the form of that work this page covers, and a visible candidate for automation because of that cost.
Deployment teams need footage at the pick face, from a viewpoint a robot could share, with outcomes attached. What operators often have is station or aisle video from above, installed for other purposes, plus system events with no pictures. This page covers how to turn either into usable demonstrations. For the images of the tote itself, see bin-picking image datasets.
Task anatomy
| Step | What happens | Typical failure | Data that helps |
|---|---|---|---|
| Receive and route | A pick instruction arrives by list, scanner, voice or light; the picker goes to the location | Wrong or stale location | Task event with location ID |
| Confirm location | Scan or read the location label | Misread label | Scan timestamp; location label images |
| Identify item | Match the SKU to what is in the pick face | Look-alike variant | SKU metadata; item images |
| Reach and grasp | Take one item from among neighbors | Multi-pick; slip; disturbing the stack | Close hand-level video; head and wrist views |
| Extract | Lift, tilt or slide the item free | Snagging; tipping neighbors | Video of the whole extraction |
| Verify | Scan the item or check weight | Scan fails; weight outside tolerance | Scan and weight events with timestamps |
| Place | Put the item into the order tote, cart or onto a conveyor | Drop; crushed item; wrong container | Placement view; destination ID |
| Handle exception | Short pick, damaged item, unreadable label | Skipped or unrecorded | Exception codes; deliberately scripted cases |
The pick face changes the task: shelf bays, flow racks, pallets and totes on carts present different reach angles and clutter, as covered in pick-face types. Parcel-oriented and pallet-oriented sites also differ, a point developed in e-commerce vs B2B distribution data.
Item variety is the main source of difficulty. Stratify any collection by packaging class: rigid boxes, polybags and soft goods, blister and hanging packs, cylinders that roll, and glossy or transparent packaging that can confuse depth sensors. Grocery tote picking has its own mix, in micro-fulfillment and grocery tote picking.
What published work shows
The public work cited here is robot work. ARMBench was collected in a warehouse pick-and-place workcell and holds images, videos and metadata for more than 235,000 pick-and-place activities on more than 190,000 unique objects, with benchmark tasks for segmentation in clutter, identification and defect detection (Mitash et al., 2023). It shows how much labeling can surround a pick when a site instruments a cell; it contains robot picks, not human demonstrations.
The first Amazon Picking Challenge surveyed 26 teams and concluded that reactive control must be combined with deliberative planning, while the choice of mechanism mainly affected execution speed (Correll et al., 2016). The practical reading is that seeing the item is only part of the problem: picking success also depends on control and planning, which human footage does not provide by itself. Published results with human video report it helping robot data, not replacing it; see what human demonstration video can and cannot do.
Where each-picking data comes from
| Source | What it shows | Suited to | Limits |
|---|---|---|---|
| Fixed station or aisle cameras | Operator and tote from above or the side | Activity recognition, timing, exceptions | Hands small or occluded; viewpoint far from a robot's |
| Scanner, WMS and weight events | Task start, location, SKU, confirm, weight | Labels, timing, outcomes | No pictures; clocks need synchronizing |
| Pick-face images before and after a pick | State of the bay or tote either side of an action | Perception in clutter; what was removed | Capture practice varies; ask whether they exist |
| Head-mounted or wrist cameras on pickers | Hand-item contact at the pick face | Co-training inputs once hands are tracked; grip strategy | Commissioned; needs consent, safety review and client approval of SKUs |
| Robot-cell logs | Robot picks and failures | Robot-side data | Not human demonstrations; robot vendors may control them; see existing warehouse robot logs |
Overhead pack-station video falls in the first row, and pack-station camera recordings covers how to evaluate it. For commissioned sessions, the practical sourcing context is in 3PL providers as data suppliers.
Joining footage to system events
A scan or a scale reading timestamps what the picker did and which item was involved, so a joined clip carries a task ID, a location, a SKU and an outcome without hand labeling. Pick confirmation via scans and weights as automatic labels sets out the join. Measure the clock offset between camera and system on a sample before trusting any label; a small drift shifts every event to the wrong action.
What the footage can and cannot support
Can support: visual pretraining on cluttered pick faces; step and progress recognition; picking-cycle timing; examples of how people slide, tilt or brace items and handle soft packaging, once hands are annotated; exception examples; and the design of evaluation scenarios.
Cannot support on its own:
- Robot actions and forces. Human picking has no joint commands, gripper states or contact forces. Multi-pick and slip detection that relies on weight or force needs those signals.
- Your end effector. A grip that works for a human hand says little about a vacuum gripper or a robot hand; see the human-to-humanoid embodiment gap.
- Pace. Human pick rates do not transfer to a robot.
- Close hand detail from distant cameras. Overhead views rarely resolve fingers and item contact.
Illustrative recording plan
# Illustrative only: synthetic ids; every parameter is the buyer's to set
campaign: eachpick_pilot_A
sites: [site_DC01 (e-commerce eaches)]
pick_faces: [shelf_bay, flow_rack, tote_on_cart]
cameras:
head: egocentric, eye-level mount
wrist: right (left optional)
pickface_fixed: one fixed view into the bay or tote
sync: shared clock with WMS; offset measured per shift
sku_strata: [rigid_box, polybag, blister_pack, cylinder, glossy_or_transparent, soft_goods]
client_approvals: {client_A: approved, client_B: approved, client_C: excluded}
sessions: consented pickers; peak and off-peak shifts
scripted_exceptions: [short_pick, damaged_item, unreadable_label, wrong_variant]
joined_events: task_id, location_id, sku_hash, scan_ts, weight_check, confirm_ts, exception_code
annotations: step_segments, hand_visibility, multi_pick_flag, outcome
excluded: consumer names and addresses, order values, client pricing, robot actions, forces
rights: training and internal evaluation; client approvals attached
Plan coverage deliberately: pick-face type, packaging class, tote density from sparse to cluttered, item weight band, shift volume, picker experience and lighting. Coverage planning for demonstrations and the head-mounted camera capture specification cover the general method.
Quality and acceptance criteria
- Visibility. The share of grasp and extract frames showing both hand and item is reported.
- Clock offset. The video-to-system offset is measured, and label error is checked on a sample.
- Join completeness. Every clip has a task ID, location, SKU (hashed if needed) and outcome.
- Exceptions kept. Multi-picks, drops and exceptions are retained and flagged; see pick errors and mispicks.
- Coverage. Counts by packaging class, pick-face type and tote density.
- Splits. Held-out sets are separated by SKU, picker and site, following the pick-and-place evaluation protocol.
- Client approvals. Approved SKUs match the SKUs on camera.
- Agreement. Step boundaries and outcome labels are double-labeled on a sample.
Questions to ask suppliers
- What cameras cover pick faces, and what do they see: hands, items or only stations?
- How are footage and system events time-aligned, and what is the measured offset?
- Which pick-face types and item classes appear, and how many distinct SKUs?
- Are multi-picks, drops and exceptions kept, and how are they flagged?
- Whose goods appear, and have those clients approved use for AI training?
- Are pickers employees or agency staff, and what notice or consent applies?
- Do screens, labels or totes show consumer names or addresses?
- How long is footage retained, and who can export it?
- Could a site host head-mounted recording on selected shifts with approved SKUs?
Permissions and provenance
Picking footage shows client-owned goods, order screens and people at work. The rights positions of the operator, its clients and the pickers are separate questions; client-owned inventory in 3PL footage covers the clause topics, and worker consent and identifiable people the people side. Blurring a label or a face reduces exposure but does not settle whether a recording may be licensed. This is not legal advice.
Discuss your requirements
If you are scoping a picking program, describe the pick-face types, item mix and permitted uses to SourceX. SourceX investigates whether US businesses hold suitable footage and records, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. If no existing footage fits, SourceX can scope a commissioned collection. A request does not guarantee a match; SourceX follows up about potential sources.
Start from the warehouse and fulfillment hub or the humanoid training data guide. The catalog entry for supply chain and logistics operations records and the logistics buyers page give wider context.
Describe the workplace demonstration data your robotics project needs
Sources
- Design and control of warehouse order picking: A literature review — order picking as the most labor-intensive and costly warehouse activity.
- ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation — warehouse workcell pick-and-place data and labels.
- Analysis and Observations from the First Amazon Picking Challenge — control and planning lessons from robot picking.