Short answer
Bin-picking image datasets show cluttered totes and bins from above or at an angle, usually as RGB or RGB-D images labeled with masks, object identities, poses, occlusion order and sometimes grasp points. Public sets pair large synthetic collections with small real ones. Real totes from fulfillment centers add the SKU variety and lighting that research sets lack, but they show client-owned goods and cannot say whether a grasp succeeded unless a robot logged it.
Why a tote is a hard perception problem
A picking tote holds touching, stacked, partly hidden and look-alike items, some deformable, some reflective, some transparent. Before a hand or gripper moves, perception has to separate instances, say what each one is, work out which are reachable, propose grasps and choose an order. Errors upstream become mispicks and multi-picks downstream, which is why image data deserves its own sourcing plan apart from each-picking demonstrations.
This page covers warehouse totes and bins of mixed items. Identical machined parts in industrial bins are a different problem, covered in industrial bin picking of parts, and generic clutter on benches and shelves in cluttered scenes for recognition.
What public datasets contain
| Dataset | Real or synthetic | Contents | Worth knowing |
|---|---|---|---|
| ARMBench | Real, warehouse workcell | Images, videos and metadata for 235K+ pick-and-place activities on 190K+ unique objects; segmentation in clutter, identification, defect detection (Mitash et al., 2023) | Robot-cell data; shows what a well-instrumented site can label |
| MetaGraspNet | Mostly synthetic | 217k synthetic RGB-D images across 82 article types with detection, amodal, keypoint, manipulation-order and grasp labels, plus 2.3k+ real annotated RGB-D images in 5 difficulty levels and an unseen-object set (Gilles et al., 2022) | Parallel-jaw and vacuum grasp labels; real set is small |
| SynPick | Synthetic | Dynamic bin-picking scenes modeled on the Amazon Robotics Challenge in a BOP-compatible format, with physics-simulated targeted and untargeted picks and random moves (Periyasamy et al., 2021) | Items shift after picks, which static images miss |
| GraspNet-1Billion | Real, cluttered scenes | 97,280 RGB-D images with over one billion grasp poses (Fang et al., 2020) | Data terms are listed as non-commercial; check before commercial use |
| ClearGrasp | Synthetic training, real benchmark | Estimates and corrects depth for transparent objects from one RGB-D image; 50,000+ synthetic images and a 286-image real benchmark (Sajjan et al., 2019) | Shows why depth needs checking on transparent items |
The pattern among these sets leans synthetic: SynPick is fully synthetic, ClearGrasp trains on synthetic images, and MetaGraspNet pairs 217k synthetic images with 2.3k+ real ones. A buyer who needs fulfillment-style SKU variety, real packaging and real lighting should expect to source real images from an operating site; synthetic data's role is discussed in real-world data vs synthetic and simulation data. Before using any public set commercially, run the checks in using public robot and egocentric datasets commercially.
Label types
| Label | Supports | Note |
|---|---|---|
| Instance mask | Segmentation of each item | Touching identical items are the hard case |
| Amodal mask and occlusion order | Reasoning about hidden parts and what lies on top | Included in MetaGraspNet; costly to annotate |
| Item identity | SKU or class recognition | Expected tote contents from the warehouse system can seed it |
| Pose or keypoints | Grasp planning for rigid items | Weak for deformables; see 6D pose annotation |
| Grasp labels | Grasp proposals | Tied to an end effector; suction and parallel-jaw labels do not describe a dexterous hand; see grasp annotations for products |
| Outcome labels | Picked, multi-picked, dropped | Only from robot or system logs; images alone cannot supply them; see pick errors and mispicks |
| Difficulty band | Stratified evaluation | Needs a written definition, such as clutter density or occlusion level |
Where real tote images exist
| Source | Viewpoint | Labels that may exist | Watch for |
|---|---|---|---|
| Goods-to-person and induction stations with tote cameras | Fixed, top-down | Tote ID, expected contents, pick events | Client-owned goods; labels showing consumer data |
| Existing robotic picking cells | Fixed, robot-mounted | Robot success and failure logs | Vendor-controlled; see existing warehouse robot logs |
| Inventory audit and cycle-count photos | Handheld, variable | Location and SKU | Irregular framing and quality |
| Pack-station and quality cameras | Overhead | Order contents | Items laid out, not cluttered |
| Commissioned capture | Chosen, calibrated | Designed labels | Cost, site access, client approval |
Fulfillment centers and 3PLs are the supplier types; the sourcing context is in 3PL providers as data suppliers.
Viewpoint and sensor fit for a humanoid
Existing tote cameras are often fixed and look straight down. A humanoid will look into a tote obliquely from head height, with its own arm in the way. That mismatch affects occlusion patterns, apparent object shape and depth quality, so ask for the sensor model, the depth technology, intrinsics, extrinsics and depth alignment. Depth behaves differently on dark, shiny and transparent surfaces, which is the problem ClearGrasp addresses for transparent objects. The general guidance is in matching demonstration camera viewpoints to a humanoid's sensors and calibration requirements; transparent and reflective objects covers the hard item classes.
Using system data as weak labels
A warehouse system often records what a tote should contain: SKUs and quantities. That seeds identity and counting labels at scale, to be verified on a sample. Pairing an image before a pick with one after it, joined to the confirmation scan, shows what was removed; see pick confirmation via scans and weights as automatic labels. Treat system-derived labels as weak until a double-checked sample shows their error rate.
Illustrative package
# Illustrative only: synthetic ids, bands and sources, not an available dataset
package: tote_images_v0
source: fulfillment_operator_D (2 sites), induction-station cameras
sensors: rgbd_cam_model_X (structured-light depth); intrinsics and extrinsics per station
frames:
type: before_pick and after_pick pairs, top-down tote views
per_frame: rgb, depth (aligned), depth_validity_mask, timestamp, station_id
tote: tote_id, tote_type, expected_contents (sku_hash, qty)
labels:
instance_mask: human-annotated sample; remainder model-assisted and reviewed
sku_identity: from expected_contents; verified on a sample
pick_order_rank: annotated on the densest difficulty bands
outcome: pick_ok | multi_pick | drop # from system log where available
difficulty_bands: C1 sparse ... C5 densely overlapped # written definitions attached
item_strata: [rigid_box, polybag, cylinder, glossy_or_transparent, soft_goods]
splits: eval holds out SKUs and one site; frames from one tote stay in one split
excluded: grasp success inferred from images, robot actions, forces; shipping labels blurred
rights: training and internal evaluation; client approvals attached per SKU
What tote images cannot support
- Grasp outcomes. An image of a tote after a pick does not say how the grasp went unless a robot or system recorded it.
- Dynamics. Items slide, tip and settle after a pick. Static frames miss this; SynPick simulates physics to model it, and real sequences need video or multiple frames.
- Mass, fragility and compliance. Images do not show how heavy or squashable an item is; see physical property annotations.
- Manipulation skill. Recognizing and segmenting items does not teach a hand how to extract one; see object recognition vs manipulation data.
- Your grasp types. Labels made for suction or parallel-jaw grippers need rework for a humanoid hand.
Quality and acceptance criteria
- Depth validity. The share of valid depth pixels is reported per difficulty band, with holes on reflective and transparent items called out.
- Mask quality. A double-annotated sample reports overlap between annotators, including for touching identical items.
- Difficulty bands. Definitions are written, and counts per band are reported.
- SKU coverage. Counts per SKU and packaging class, with the long tail visible.
- Splits. Evaluation data is separated by SKU, tote and site, and near-duplicate consecutive frames stay on one side.
- Calibration. Intrinsics, extrinsics and depth alignment are documented per station.
- Label provenance. Each label type is marked human, model-assisted or system-derived.
- Outcome labels. Source and coverage are stated; unlabeled picks are flagged, not assumed successful.
- Approvals. Client approvals map to the SKUs that appear.
Questions to ask suppliers
- Which cameras and depth sensors produced the images, and are calibration files kept?
- Do images show whole totes before picks, after picks, or only the picked item?
- Does the system record expected tote contents, and how accurate is that record?
- How many distinct SKUs and packaging classes appear, including soft and transparent goods?
- Were images auto-labeled or filtered by a model before they reached the archive?
- Are failed picks, multi-picks and drops linked to specific images?
- Whose goods appear, and have those clients approved use for AI training?
- Do shipping labels, order screens or workers' hands appear, and what redaction was applied?
- How long are images retained, and who can export them?
Permissions and provenance
Tote images show client-owned goods and their packaging, and top-down views can catch shipping labels with consumer names and addresses. Workers' hands and order screens can appear too. Client-owned inventory in 3PL footage sets out the clause topics. Blurring a label reduces exposure but does not settle whether a client has agreed to licensing. This is not legal advice.
Discuss your requirements
If you need real tote imagery, describe the item mix, sensors, label types and permitted uses to SourceX. SourceX looks for US businesses that hold suitable images, checks the data and the supplier's licensing permissions, and coordinates pricing, allowed uses and delivery in a license. If no existing images fit, SourceX can scope a commissioned collection. SourceX reviews the request and follows up about potential sources; a request does not guarantee a match.
The warehouse and fulfillment hub and the humanoid training data guide place this asset among the others. The catalog pages on images and inspection photos and supply chain and logistics operations records describe related record types.
Discuss your humanoid training-data requirements with SourceX
Sources
- MetaGraspNet: A Large-Scale Benchmark Dataset for Scene-Aware Ambidextrous Bin Picking via Physics-based Metaverse Synthesis — synthetic and real bin images with detection, amodal and grasp labels.
- SynPick: A Dataset for Dynamic Bin Picking Scene Understanding — physics-simulated dynamic bin-picking scenes.
- ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation — real warehouse workcell pick-and-place data.
- GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping — RGB-D clutter images with grasp poses.
- ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation — depth correction for transparent objects.