Short answer
Catalog packshots show a product alone, evenly lit, centered, on a plain background, in a pose chosen to sell it. A robot sees it partly hidden, at an angle, in uneven light, among neighbors. Packshots are good for building the identity of each product; in-situ images are needed to train and test detection in context. Most programs need both, plus a plan to measure the gap.
The buyer's problem
Catalogs are the cheapest large source of labeled product images, and they come with names, categories and attributes. The temptation is to train on them and expect the model to work on a shelf, a cart or a countertop. It often does not, because every nuisance factor the studio removed returns at the point of use: clutter, glare, motion, partial views and compression artifacts from the robot's camera.
This page helps you decide how much in-situ imagery to buy and how to test for the gap. For what catalogs contain in general, see product catalog data for robot perception. For multi-view studio capture, see the multi-view product photography specification.
What differs between a packshot and a robot's view
| Factor | Packshot | In-situ image from a robot or workplace camera |
|---|---|---|
| Background | Plain or removed | Shelves, benches, other products, people |
| Lighting | Diffuse, controlled | Mixed sources, glare, shadows, backlight |
| Viewpoint | Front, canonical set | Oblique, top-down, wrist-level, partial |
| Occlusion | None | Neighbors, hands, grippers, shelf lips |
| Scale and resolution | Large, sharp | Small in frame, motion blur, compression |
| Object state | New, clean, upright | Dented, tilted, wet, stacked, half-open |
| Labels | One product per image | Many products; instance masks or boxes needed |
Each row is a distinct failure source; the rows can be tested separately if the data carries the right metadata.
What published datasets show about the gap
Amazon Berkeley Objects is a catalog-side reference. It pairs 147,702 listings and 398,212 unique catalog images with metadata, 3D models for 7,953 products and 360-degree turntable views for 8,222 products (Collins et al., 2021). It supports 3D understanding and multi-view retrieval benchmarks, which is catalog-grade identity and geometry, not the visual conditions of a workplace.
SKU-110K sits on the other side. It is an annotated set of densely packed retail shelves, targeting tightly packed, often identical objects, and its authors introduced detection components designed for that density (Goldman et al., 2019). Standard detectors assume objects are separated; the shelf case breaks that assumption.
The RPC checkout dataset makes the gap explicit in its own structure: single-product images under controlled conditions plus multi-product checkout images, with up to 83,000 images and 200 product categories in 17 meta categories (Wei et al., 2019). The existence of that split in a published benchmark is a useful design pattern. You can reproduce it in your own evaluation by requiring a clean exemplar set and a cluttered test set per product.
Decision table: what to buy for what purpose
| Purpose | Packshots | In-situ images | Note |
|---|---|---|---|
| Learn product identity and names | Primary | Supporting | Catalog attributes ground language |
| Train detection in clutter | Weak | Primary | See cluttered scenes for recognition |
| Evaluate real-world accuracy | Not suitable | Primary | Hold out by site and camera |
| Pose and grasp planning | Partial (multi-view) | Needed for occlusion | Add depth or 3D models |
| Count items on shelves | Not suitable | Primary | See planogram compliance images |
| Fill the long tail of rare SKUs | Primary | Hard to get | See long-tail object coverage |
Closing the gap: options and trade-offs
Synthetic compositing and renders. Placing packshot cutouts or 3D models into varied backgrounds can widen coverage cheaply. It does not reproduce real sensor noise or the true statistics of clutter; see synthetic renders versus real images.
Existing in-situ images. Retailers, warehouses and food-service sites hold photographs from audits, receiving and incident records. They were taken for other purposes, so viewpoint, resolution and labels are uneven, and they may include people or confidential layouts.
Commissioned in-situ capture. You specify products, placements, cameras and conditions and a capture is designed around them. It costs time and needs site access and permissions. Best used for the evaluation set, where control matters most.
An evaluation protocol for the gap
- Pick a product list and collect or buy packshots for each.
- Collect in-situ images of the same products in at least two distinct sites.
- Train on packshots only; test on in-situ images from one site.
- Add in-situ images from the second site to training; test again on the first site.
- Report accuracy by occlusion level and by lighting condition, not only overall.
The difference between steps 3 and 4 estimates what in-situ data is worth for your pipeline. Requirements vary by architecture, camera and products, so no universal image count applies.
Camera-mismatch checklist
Before accepting in-situ images as a proxy for your robot, compare these parameters with your own platform: camera height above the floor, tilt, field of view, resolution after any streaming compression, shutter type and exposure behavior, and whether the camera moves while capturing. Retail audit photos taken with a handheld phone at chest height differ from a head camera that sweeps past a shelf, and both differ from a wrist camera close to the item. If the supplier can tell you the device and mount for each image, you can stratify results and see which conditions drive errors. The novel object generalization evaluation page explains how to hold items out, and grocery and retail product recognition covers the retail setting where this mismatch is most visible.
Limitations
Even in-situ images from retail or workplace cameras may not match your robot's sensor height, field of view or motion. If the robot's camera is on a moving humanoid head, handheld or fixed-camera photographs only approximate it. The images also record how products are displayed, not how they are handled. Handling needs object-in-hand images or interaction data.
Permissions and provenance
In-situ images often capture people, screens, price tags and neighboring brands. Ask what was in frame, who consented, and who can license training use. This is not legal advice, and blurring parts of an image does not settle permission questions on its own.
Questions to ask suppliers
- Which share of images shows products in context versus isolated?
- What cameras, heights and distances were used, and can they be matched to your robot's?
- How many distinct sites, lighting setups and times of day are covered?
- Are the same products present as both packshots and in-situ images, with a shared ID?
- How are partially visible products labeled, and with what visibility field?
- What was removed or masked before delivery, and does it affect objects of interest?
- Who holds rights in the in-situ images, and were people or screens in frame?
- Can you provide a sample that I can score against my own detector before purchase?
Discuss your requirements
If you can name the products, the camera you plan to use and the places the robot will work, describe the requirement to SourceX. SourceX reviews it, follows up about potential US sources, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. Where no existing set fits, SourceX can scope a commissioned collection of new recordings; a request does not guarantee a match. Start from the humanoid training data guide or the product and object data hub. The site also describes a physical-world data path for new recordings coordinated with businesses.
Discuss your humanoid training-data requirements with SourceX
Sources
- ABO: Dataset and Benchmarks for Real-World 3D Object Understanding — ABO: catalog images with metadata, artist-made 3D models and 360-degree turntable views.
- Precise Detection in Densely Packed Scenes — SKU-110K: densely packed retail shelf images with tightly packed, often identical objects.
- RPC: A Large-Scale Retail Product Checkout Dataset — RPC: controlled single-product images versus multi-product checkout images.