Short answer
An object recognition dataset for robots pairs images, 3D models and attributes of real items so a humanoid can detect, identify, localize and plan grasps on them. Retailers, distributors and manufacturers already hold much of this as catalog images, dimensions and CAD. It shows what objects look like and measure, not how they behave when handled, so manipulation still needs interaction data.
Scope of this cluster
A humanoid working in a store, warehouse, kitchen or plant meets thousands of distinct items, many of them changing every season. Perception engineers need data that teaches the robot to find an item, tell it from look-alikes, estimate its pose and state, and know enough about its size, weight and material to plan a grasp. This cluster covers where that data comes from and how to judge it.
It deliberately separates two needs. Recognition data covers detection, identification, attributes and state; physical-understanding data covers dimensions, mass, material, articulation and grasp annotations that a planner consumes.
Both are about objects at rest. How an object moves, flexes or slips in a hand is interaction data, which comes from workplace task demonstrations or robot data. The keystone explanation is object recognition data vs manipulation training data.
What object data can support
| Need | Asset type | What published work shows |
|---|---|---|
| Item identification at SKU level | Catalog images and listing metadata | ABO pairs 147,702 product listings with 398,212 catalog images and up to 18 metadata attributes per product (Collins et al., 2021) |
| 3D shape and material | Artist meshes, 3D scans | ABO adds 7,953 products with artist-made meshes and physically based materials; Google Scanned Objects provides over one thousand scanned household items prepared for simulation (Downs et al., 2022) |
| 6D pose | CAD plus annotated RGB-D | BOP gathers eight pose datasets in a unified format with an error measure that handles symmetric objects (Hodan et al., 2018) |
| Grasp planning | Grasp labels on images or geometry | GraspNet-1Billion annotates 97,280 RGB-D images with over one billion grasp poses (Fang et al., 2020) |
| Identification in a real workcell | In-cell images with pick metadata | ARMBench records 235K+ pick-and-place activities on 190K+ unique objects in a warehouse workcell (Mitash et al., 2023) |
| Repeatable physical evaluation | Physical object sets with models | YCB provides everyday objects varying in size, texture, weight and rigidity, with scans, physical properties and benchmark protocols (Calli et al., 2015) |
Business supply maps most directly onto the first three rows. Retail and e-commerce catalogs hold images, titles, descriptions and attributes; product master data holds measurements. The GS1 Package and Product Measurement Standard defines nominal measurement attributes from consumer unit to pallet, and GDSN lets trading partners share that master data through certified data pools (GS1, 2026). Manufacturers hold CAD and part masters, often under tight design-IP control.
What it cannot support
- Behavior under handling. Catalog and CAD data do not show a bag sagging, a lid sticking, a cable springing back or a box crushing. Even large grasp datasets are often simulated: ACRONYM's 17.7M grasps on 8,872 objects were labeled by physics simulation (Eppner et al., 2020), which is not the same as a recorded real-world outcome.
- In-situ appearance. Packshots are clean, centered and evenly lit; shelves, totes and workbenches are not. See packshots vs in-situ images.
- Objects in the hand. Occlusion by fingers and gloves changes appearance in ways catalog images never show.
- Currency. Packaging changes and SKU churn age a catalog quickly; see packaging variants and SKU appearance changes.
- Hard materials. Transparent and reflective items defeat depth sensors and need data of their own; see transparent and reflective objects.
Sourcing routes
Existing assets. Retailers, brands and e-commerce operators hold catalog images and attributes; industrial and automotive distributors hold parts catalogs with cross-references; manufacturers hold CAD, PLM records and part master data. Image rights are the usual snag: many catalog photos were shot by agencies or supplied by brands, so the retailer may not be able to license them. See rights in product images and product catalog data for robot perception.
Commissioned capture. When catalog imagery is too clean or too sparse, multi-view photography or 3D scanning of the target items fills the gap, and physical samples let you image items on your own rig and lighting. The multi-view product photography specification and workplace benchmark object set cover both. If no existing asset fits, SourceX can scope a commissioned collection of new recordings.
Annotation. Pose, grasp, affordance and state labels can be added to licensed images or scans. SourceX's role is sourcing, rights review, licensing and delivery coordination; it can also arrange annotation and engage specialist capture partners, scoped case by case.
How to evaluate object data
- Rights chain per image. Who shot it, who owns it, and whether AI training is permitted.
- Attribute completeness. Share of SKUs with dimensions, weight, material and units filled consistently (a buyer might set 95% for dimensions on the target SKUs).
- View coverage. Number of views per item and whether the underside, back and labels are included.
- Context ratio. Proportion of in-situ images to packshots, and whether the contexts match your deployment.
- Date range. Capture dates against packaging changes, so the data reflects what is on shelves now.
- 3D fidelity. For CAD or scans, measured agreement with physical items and whether physics properties are present.
- Holdout. Whether a set of items can be kept unseen for novel-object evaluation.
Illustrative object package for a tote-picking pilot
# Illustrative only — synthetic SKUs and values, not an available dataset
package: object_pack_pilot_v0
scope: 1,200 target SKUs from retailer_R07; 150 held out for evaluation
per_sku:
images: 6 studio views (front, back, left, right, top, bottom) + 4 in-tote photos
attributes: gtin, title, category, length_mm, width_mm, height_mm, net_weight_g, packaging_material
geometry: scanned mesh (obj) for 300 SKUs; none for the rest
labels: deformable (y/n), transparent (y/n), fragile (y/n)
excluded: grasp outcomes, slip or deformation data (not present in catalog sources)
rights: studio images owned by retailer_R07; brand-supplied images excluded pending permission
The excluded line matters. A tote-picking policy still needs demonstrations or robot episodes; the object package makes them easier to label and evaluate.
Key pages in this cluster
| Group | Page | What it answers |
|---|---|---|
| Foundations | Recognition vs manipulation data | The keystone distinction for this cluster |
| Foundations | Synthetic CAD renders vs real images | When renders substitute for photos |
| Foundations | Instance vs category recognition | Which labels your task needs |
| Foundations | Object data for hands vs grippers | Multi-finger hands need different object data |
| Sources | Product catalog data for robot perception | The most accessible object supply |
| Sources | Grocery and retail product recognition | Store items and look-alikes |
| Sources | Manufacturer 3D models and CAD | Exact geometry and its IP constraints |
| Sources | Industrial distributor catalogs | Parts data at scale |
| Sources | Retailer vs manufacturer object data | Which holder fits which need |
| Sources | Product dimensions and weights | Measurements for grasp and motion planning |
| Capture and annotation | Multi-view product photography spec | An adaptable capture specification |
| Capture and annotation | Object attribute schema | Fields for robot-ready product records |
| Capture and annotation | Grasp annotations for products | Labeling grasps on product data |
| Capture and annotation | Affordance and part labels | Handles, lids, buttons and parts |
| Capture and annotation | 6D pose annotation | Pose labels for workplace objects |
| Capture and annotation | Physical property annotations | Mass, friction, fragility and compliance |
| Capture and annotation | Object state recognition | Open, closed, full, empty, clean, dirty |
| Hard cases | Transparent and reflective objects | Items depth sensors miss |
| Hard cases | Fasteners and small hardware | Tiny, similar parts |
| Hard cases | Equipment controls and interfaces | Buttons, switches, dials and screens |
| Hard cases | Cluttered scenes | Workbenches, shelves and sinks |
| Evaluation and rights | Novel-object generalization evaluation | Test sets of unseen items |
| Evaluation and rights | Workplace benchmark object set | Physical items for repeatable tests |
| Evaluation and rights | Product master data quality audit | Checking attributes before robot use |
| Evaluation and rights | Rights in product images | Photographers, brands and retailers |
Neighboring clusters
Bin-picking imagery and pick errors are in warehouse and fulfillment, including bin-picking image datasets. Tool and spare-part identification is in maintenance and tool use; parts presentation at workstations in manufacturing and assembly; condition and damage in inspection and failure data. The section overview shows how objects fit the full stack. For catalog context, see licensing product catalogs and descriptions and licensing CAD and engineering drawings.
Discuss your requirements
List the item categories, the number of SKUs, the views and attributes you need, and how many items you will hold out. Describe the object data to SourceX: it looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. SourceX reviews each requirement and follows up about potential sources; a request does not guarantee a match.
Discuss your humanoid training-data requirements with SourceX
Sources
- ABO: Dataset and Benchmarks for Real-World 3D Object Understanding — catalog images, metadata and 3D models at product scale.
- Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Items — scanned items prepared for simulation.
- BOP: Benchmark for 6D Object Pose Estimation — unified 6D pose datasets and error measure.
- GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping — grasp annotations on RGB-D images.
- ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation — object data collected in a warehouse workcell.
- Benchmarking in Manipulation Research: The YCB Object and Model Set and Benchmarking Protocols — physical object set for repeatable evaluation.
- GS1 Package and Product Measurement Standard — standard measurement attributes and master-data sharing.
- ACRONYM: A Large-Scale Grasp Dataset Based on Simulation — simulated grasp outcomes.