Short answer
Pick-error data pairs warehouse picks with their verified outcomes: wrong item, wrong quantity, more than one item lifted, short pick, drop or damage. Fulfillment centers hold scan mismatches, weight-check failures, quality-control rejections and return reasons, sometimes with images. It can train and test error detection, but it skews toward errors that were caught, rarely shows why a grasp failed, and does not show how a robot should recover.
Why error examples are a separate purchase
A picking system that cannot tell when it got the pick wrong has no verification layer. Programs usually have plenty of successful picks and almost no labeled mistakes, because warehouses are run to avoid mistakes and records focus on throughput. An evaluation lead who wants to measure how often a detector catches a wrong item or a double pick needs those negatives deliberately, and with the same imaging as the successes.
Three neighboring needs are easy to blur. Human demonstrations of picking are covered in each-picking demonstrations. Failures produced by robots, such as slips and collisions, are covered in robot manipulation failure datasets. This page covers the error records that fulfillment sites already keep about picks, and what they are worth.
Error types and how a warehouse detects each
Definitions vary by site, so ask for the taxonomy and the codes behind it. The table shows the common patterns.
| Error | What happened | Usually detected by | When the label arrives | Label reliability |
|---|---|---|---|---|
| Wrong item | A different SKU or variant was picked | Scan verification at pick or pack; QC audit; customer complaint | Seconds at a scan; days after a complaint | High at a scan; complaint labels are noisy |
| Short pick | Fewer units than ordered, or the item was not found | Quantity entry; exception code | Seconds | High for the event; the cause (inventory or miss) is unknown |
| Wrong quantity | More or fewer units than the order | Scan count at pack; weight check | Seconds to minutes | Medium to high |
| Multi-pick | More than one unit lifted when one was wanted | Weight check; scan count; vision | Seconds to minutes | Medium; depends on weight tolerance |
| Drop or slip | Item fell during the pick | Operator report or camera | Often never recorded | Low |
| Handling damage | Item damaged in picking or packing | QC; returns; carrier claim | Days to weeks | Medium; cause often unclear |
| Wrong location or container | Item taken from, or placed into, the wrong place | Location scan; put-wall mismatch | Seconds | High |
The timing column matters for training. A label that arrives from a scan is tied to a moment you can find in video. A label that arrives from a customer complaint is tied to an order, not to a moment.
What published work shows
Public failure data is mostly from robots. ARMBench was collected in a warehouse pick-and-place workcell and includes defect detection among its benchmark tasks alongside segmentation and identification (Mitash et al., 2023). Read its defect categories before assuming they match your error taxonomy; its value to a buyer is as a picture of what labeled failures look like when a site instruments one cell.
Other public sets show different routes to failure data. RoboMIND includes 5K real-world failure demonstrations annotated with causes (Wu et al., 2024), and REFLECT introduced RoboFail, a dataset of failed executions in simulation and the real world (Liu et al., 2023). AHA's FailGen takes the synthetic route, perturbing successful simulated demonstrations to generate failure trajectories (Duan et al., 2024). Generated failures cover the failures someone thought to generate; warehouse records contain the ones that happened, including look-alike SKUs and packaging quirks nobody anticipated.
Where the records live
| Source | What it holds | Strength | Watch for |
|---|---|---|---|
| Warehouse management and pick-verification logs | Scan mismatches, exception codes, short picks | Timestamps and SKU IDs | No images; inconsistent codes across sites |
| Check-weigh and in-motion scale logs | Expected and measured weight per tote or carton | A direct multi-pick signal | Tolerance bands; mixed-SKU containers |
| Quality-control audit stations | Photos of audited orders with human verdicts | Confirmed labels with pictures | Audited orders are a biased sample; the viewpoint differs from the pick face |
| Pack-station cameras | Video of items going into cartons | Time-aligned exceptions | Short retention; see pack-station camera recordings |
| Customer service and returns | "Wrong item received" reason codes | Escapes that passed every check | Late, noisy, tied to orders; see returns processing workflow |
| Robotic picking cells | Grasp outcomes and sensor-based multi-pick flags | Failure modes of automated picking | Usually vendor-controlled; see existing warehouse robot logs |
The main suppliers are fulfillment operators, including third-party logistics providers, e-commerce sellers and retail distribution centers, and sites that run robotic picking cells. Joined scan and weight events are described in pick confirmation as labels; how sites handle exceptions in general is in warehouse exception handling.
What to request
Ask for one record per pick event, with the image evidence attached where it exists. The fields below are a starting point to adapt in a request to SourceX.
# Illustrative only — synthetic identifiers, codes and values
event_id: pk_site012_77410
site: site_DC12, station: PICK_14, picker_id: operator_B33 # pseudonymous
order_line: {expected_sku: SKU_10442, expected_qty: 1}
observed: {sku: SKU_10443, qty: 1, weight_g: 212, expected_weight_g: 205}
outcome: wrong_item # wrong_item | short | multi_pick | drop | damage | ok
detected_by: scan_mismatch # scan_mismatch | check_weigh | qc_audit | customer_return
detection_delay_s: 4
confirmed_by_human: true
images: [tote_before.jpg, tote_after.jpg, scan_station.jpg] # capture moments stated
look_alike_group: LA_0061 # SKUs commonly confused
split_key: [site, sku_family, week]
excluded: [customer details, order value, client pricing, robot actions]
Also request matched negatives: correct picks captured through the same station and cameras. Without them a model can learn the station or lighting instead of the error.
What this data can and cannot support
Can support: training and testing detectors for wrong item and multi-pick from images and weights; calibrating verification thresholds; hard negatives for item identification, especially look-alike SKUs; evaluation scenarios for verification layers; and the design of a failure taxonomy for your own robot data.
Cannot support:
- Grasp physics. A record that says "multi-pick" does not say why the second item came along. Cause needs robot or close-up data.
- Recovery behavior. Human exception handling is a procedure, not a robot action.
- True error rates. Records count detected errors. Errors that slipped through, or were never logged, are missing.
- Equivalence across stations. Error images taken at a QC station differ in viewpoint and lighting from ordinary picks.
Quality and acceptance criteria
- Taxonomy. Definitions are supplied and mapped to yours, with an
othercode and its share reported. - Ground truth. The share of labels confirmed by a person, rather than only flagged by a system, is stated per error type.
- Matched negatives. Correct picks are included from the same sources, with the selection rule written down.
- Image timing. For each image the capture moment is stated: before the pick, at the scan or after packing.
- Counts. Reported by error type, site and SKU class, so rare combinations are visible; see rare defects and class imbalance.
- Leakage checks. Reject totes, stickers, screen overlays and station signage that reveal the label are removed or flagged.
- Splits. Evaluation data is held out by site, SKU family or time, following training vs evaluation datasets.
- Pseudonymization. Pickers appear only as pseudonymous IDs.
Limitations and preparation
Expect to harmonize codes from several sites, and to relabel a sample where records are ambiguous. Rare classes mean small counts, so a single site rarely supports a held-out evaluation per error type. Records of escapes are weakly linked to events. If you plan to expand a small set synthetically, treat generated variants as training material only and keep real records for evaluation. Pick-and-place evaluation design is covered in the warehouse pick evaluation protocol.
Questions to ask suppliers
- Which error types are recorded, with what codes, and how did definitions change over time?
- Which records carry images, and at what moment in the pick are they captured?
- What share of labels was confirmed by a person?
- Are correct picks available from the same stations and cameras?
- How are multi-picks detected: weight, scan count, vision or operator report?
- Are escapes, such as customer returns, linked back to the pick event?
- Are records tied to named pickers, and can they be pseudonymized?
- Whose goods appear in the images, and have those clients approved AI training use?
- Which SKU groups are most often confused, and are they represented?
Permissions and provenance
Error records tied to a named picker are performance data about an employee, so ask for pseudonymous IDs or removal. Images show client-owned goods and sometimes order details; see client-owned inventory rights and worker consent and identifiable people. Complaint and return records may include consumer data that should not be delivered.
Discuss your requirements
If you need error examples for training or evaluation, describe the error types, labels, image needs and permitted uses to SourceX. SourceX investigates whether US businesses hold suitable records, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match; SourceX reviews it and follows up about potential sources.
Start from the warehouse and fulfillment hub or the humanoid training data guide. The catalog entries for supply chain and logistics operations records and images and inspection photos give wider context.
Tell SourceX what your evaluation set needs to cover
Sources
- ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation — warehouse workcell data with a defect detection task.
- RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation — real-world failure demonstrations with causes.
- REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction — RoboFail dataset of failed executions.
- AHA: A Vision-Language-Model for Detecting and Reasoning Over Failures in Robotic Manipulation — synthetic failure generation from successful demonstrations.