Short answer
Planogram compliance data consists of shelf images labeled against a target layout: which product should sit in each facing, which is actually there, and where gaps, misplacements and mis-orientations occur. To be useful it needs the planogram version for each image, SKU-level boxes, and gap and misplacement labels. Plain shelf photos with product boxes but no target layout support detection, not compliance.
The buyer's problem
Retail shelf work is a plausible early task for a mobile or humanoid robot: scan an aisle, find gaps, spot misplaced items, restock. The perception problem is hard. Shelves hold rows of nearly identical packs, with price tags, promotional signage, reflections and customers in the way. Compliance adds a second layer, comparing what is seen with a stored intent that changes by store, season and promotion.
This page covers sourcing shelf images that carry that intent. For general shelf and store recognition, see grocery and retail product recognition. For appearance drift across pack versions, see packaging variants and SKU appearance.
What published shelf data shows
SKU-110K is an annotated dataset of densely packed retail shelves. It targets scenes where objects are tightly packed and often identical, and the authors introduced a Soft-IoU layer and an EM-Merger unit to handle overlapping detections in that density; they also evaluated on the CARPK and PUCPR+ counting datasets (Goldman et al., 2019).
Two points matter for compliance buyers. First, the detection problem in dense shelves is different from standard object detection: boxes overlap and adjacent items are the same product. Second, a detection dataset alone says where items are, not whether they match a plan. To get from detection to compliance, you need the planogram, a mapping from each detected item to a SKU, and a notion of facing, row and position. Those are not described in the SKU-110K publication, so ask for them explicitly.
Layers of labeling
| Layer | Label | Needed for |
|---|---|---|
| 1. Detection | Box or mask per item or facing | Counting, locating |
| 2. Identification | SKU or product ID per box | Matching to plan |
| 3. Layout | Shelf, row, position, facing count | Spatial comparison |
| 4. Plan link | Planogram version and expected SKU per slot | Compliance |
| 5. Exceptions | Gap, wrong item, wrong orientation, wrong price tag | Alerting |
| 6. Context | Store format, aisle, time, lighting | Splits and analysis |
Many shelf datasets stop at layer 1 or 2. If your goal is out-of-stock and misplacement detection, layers 4 and 5 are where the value is, and they are the layers most dependent on retailer cooperation.
Illustrative record
# Illustrative: synthetic values only
shelf_image:
image_id: shf_004417
store_id: store_031 # pseudonymous
planogram_id: pg_snacks_v12 # version at capture time
shelf: {bay: 3, shelf_level: 2}
facings:
- {slot: 7, expected_sku: sku_55120, observed_sku: sku_55120, box: [x1,y1,x2,y2], status: ok}
- {slot: 8, expected_sku: sku_55121, observed_sku: null, box: [x1,y1,x2,y2], status: gap}
- {slot: 9, expected_sku: sku_55130, observed_sku: sku_55140, box: [x1,y1,x2,y2], status: misplaced}
exceptions: [price_tag_missing_slot_8]
capture: {device: handheld_phone, height_m: 1.4, lighting: store_fluorescent}
people_in_frame: false
Fields such as planogram_id and status come from retailer systems or audit staff. A dataset without them is a detection set.
Supplier types and routes
Retailers hold store audit photos and planograms, and their category management teams own the plans. Photos may have been taken for internal checks and include staff, customers and competing products.
Merchandising and field-audit firms take shelf photos on behalf of brands and retailers, sometimes at scale, and often with a standard capture app. They typically hold the images under contracts with the brand or retailer, so ask who can authorize training use.
Brands hold their own execution audits, covering their products but not neighbors.
Existing assets are the sensible route here, since a purpose-built collection needs retailer cooperation across many stores, which is slow. A commissioned capture can add images from your robot's camera height and path, to test the transfer from handheld audit photos. Handheld photos are taken at a different height and angle than a robot sees, as the packshot versus in-situ images page discusses for catalogs.
Edge cases to define in advance
Compliance labels depend on definitions that look obvious until annotators meet real shelves. Decide how to treat an item pushed to the back of the shelf, a facing that is partly filled, a correct product placed in the neighboring slot, a promotional display that replaces the planogram slot, a product turned sideways, and a facing hidden by a customer or cart. Decide whether a missing or wrong price tag counts as an exception. Write each rule down with an example image, and ask suppliers whether their labels follow rules of this kind. Without them, two datasets with the same column names can mean different things. The cluttered scenes for recognition page covers occlusion and overlap more generally.
Acceptance criteria
- Plan linkage present for a majority of images, with the version at capture time.
- SKU identification accuracy checked on a sample against the physical shelf or a second annotator.
- Gap and misplacement labels defined, with examples of edge cases such as a pushed-back item or a half-empty facing.
- Store diversity. Report number of stores, formats and chains. A single chain may use one fixture style.
- Time spread. Seasonal sets and promotions change appearance; see SKU churn and catalog refresh.
- Split by store. Do not place images of the same shelf in train and test.
- Taxonomy mapping. Product categories mapped to a classification you use; see product taxonomies.
What this data cannot support
Shelf images do not show how to pick, place or face a product. They contain no grasps, forces or shelf-lip geometry, and a robot restocking shelves needs manipulation data that these images cannot provide. They also describe the shelf at a moment: stock changes within hours. Planograms are business-confidential and sometimes unavailable even when images are. A model trained on one retailer's fixture style and label design may not transfer to another's.
Permissions and provenance
Shelf images show competing brands' packaging and trademarks, price information, in-store signage, and sometimes shoppers or staff. Ask who owns the photos, whether the retailer or brand can license training use, and whether the planograms are covered by the same grant. See trademarks in training images and product image rights. Blurring faces does not by itself settle permission questions. This is not legal advice.
Questions to ask suppliers
- Does each image link to a planogram version, and can the plan be shared?
- Are SKUs identified per facing, and how was identification verified?
- How are gaps, misplacements and orientation errors defined?
- How many stores, chains, formats and dates are represented?
- What device and height were used for capture?
- Are people and screens visible, and what has been done about them?
- Who owns the images and the planograms?
- Can I score a sample against my detector before purchase?
Discuss your requirements
If you can describe the store formats, categories and the compliance checks your robot must perform, describe the requirement to SourceX. SourceX reviews it, follows up about potential US retailers or merchandising firms, and checks the data and licensing permissions before a license is agreed. A request does not guarantee a match. Start from the humanoid training data guide or the product and object data hub. See also the site's entries on images and inspection photos and supply chain and logistics data.
Explore sourcing and licensing options with SourceX
Sources
- Precise Detection in Densely Packed Scenes — SKU-110K: annotated densely packed retail shelf images with tightly packed, often identical objects; also evaluated on CARPK and PUCPR+ counting.