Retail Shelf Restocking and Facing Demonstrations

The deployment problem

Restocking looks simple and is not. Associates work from carts of mixed cases, often overnight, against a planogram that fixes each product's position and number of facings. They open cases with blades, check dates, push older units forward, place new ones label-out, and pull everything to the shelf edge. Shelves are deep, high and close together, and in daytime there are shoppers in the aisle.

Retail recognition data is plentiful. Data showing how a person or robot actually moves product into a crowded shelf is scarce. A deployment team needs both, and it needs to know which suppliers can provide which.

Task anatomy

Two fixtures change the task. Peg hooks need items threaded onto a rod rather than set down. Refrigerated cases add a door the robot must hold open while placing product, which is a two-handed pattern covered on bimanual task demonstrations. Case opening is its own skill; see opening containers and packaging.

What public retail data covers

Public datasets concentrate on recognition:

• Dense shelves. SKU-110K annotates images of densely packed retail shelves, where objects are tightly packed and often identical (Goldman et al., 2019).
• SKU-level identity. Products-10K labels 10,000 SKU-level products with in-shop and customer images arranged in a hierarchical label graph (Bai et al., 2020).
• Controlled vs cluttered. RPC pairs single-product images taken under controlled conditions with multi-product checkout images (Wei et al., 2019), which makes the gap between catalog-style and in-scene images explicit.

A relevant robot-side precedent comes from warehouses, not stores. ARMBench released images, videos and metadata for more than 235,000 pick-and-place activities on more than 190,000 unique objects from a robotic workcell (Mitash et al., 2023). It shows what object-centric manipulation records look like, but a workcell bin is not a store shelf.

Recognition data tells a robot what is on the shelf and where. It does not show reaching past a front row, tilting a box to fit under a shelf lip or catching a bottle that starts to tip. The keystone page on object recognition vs manipulation data explains why both are needed.

Business data that fills the gaps

• Planograms and shelf tags. The target state for every slot, which turns placement into a checkable outcome; see retail planograms.
• Shelf photos. Merchandising services and store teams photograph shelves before and after work as proof of completion. These become compliance and success labels; planogram compliance images covers what to check.
• Product master data. Packshots, dimensions and weights keyed to GTINs, described on the catalog page for product catalogs and descriptions and, for robot perception, on grocery and retail product recognition.
• Training material. Videos and job aids on rotation, facing and case handling standards.
• System records. Gap scans, out-of-stock alerts and inventory adjustments, which time-stamp when a slot was empty and when it was refilled.
• Commissioned capture. Head-mounted recording of overnight crews, the main route to hand-level stocking demonstrations.

Store security footage rarely helps with manipulation; fixed-camera footage limits explains why.

Capture and labels

Record when the store is closed or quiet, so shoppers are not on camera. A head-mounted camera captures hands and product; a second camera on the cart or shelf upright gives a stable view of the slot before and after.

# Illustrative only — synthetic label record for one restocking episode
episode_id: stock_site044_0912
store: {format: grocery, aisle: 07, bay: 03, shelf_level: 4_of_5, fixture: shelf}
planogram: {pog_id: POG-07-03-v12, slot: S14, gtin: "00000000000000", facings_target: 3}
associate: associate_D21 (overnight crew, 1 yr)
segments:
  - {label: open_case,       tool: box_cutter, start_s: 0.0,  end_s: 6.4}
  - {label: rotate_existing, units_moved: 4,   start_s: 6.4,  end_s: 12.1}
  - {label: check_date,      units_checked: 2, start_s: 12.1, end_s: 14.0}
  - {label: place_units,     units_placed: 9,  start_s: 14.0, end_s: 31.7}
  - {label: face_row,        start_s: 31.7,    end_s: 35.2}
images: {before: shelf_before.jpg, after: shelf_after.jpg}
outcome: {slot_correct: true, facings_after: 3, label_out: true, damaged_pulled: 1}
excluded: robot actions, contact forces, shopper footage (recorded after close)

Supplier types

• Grocery and convenience chains with overnight stocking crews and standard store formats.
• Big-box and drugstore retailers with planogram systems and store reset programs.
• Third-party merchandising services that stock and reset shelves for brands across many retailers and photograph their work.
• Direct-store-delivery teams of beverage and snack brands, who restock their own products.
• Store reset and remodel contractors, who rebuild whole aisles to new planograms.

A merchandiser's photos show a retailer's shelves and a brand's products, so ask who can license them. Packaging carries third-party trademarks and artwork, and store layouts may be confidential. New recording follows the site's separate physical-world path for hands-on work, where each opportunity has its own requirements.

Limits for this task

• Manipulation data is the gap. Images teach recognition; reaching, tilting and pushing in tight shelves need robot or teleoperation episodes.
• Weight and fragility. Glass jars and soft bags look stable in images; their handling needs physical property data or demonstrations.
• Packaging churn. SKUs and packaging change often, so recognition data ages and needs refresh terms.
• Human technique. Associates sweep a row with a forearm or place two units at once; a humanoid may not.
• Store variety. Fixture types, shelf depths and lighting differ by format and region.
• Daytime conditions. Overnight capture misses shoppers, carts in aisles and daytime lighting.

Questions to ask suppliers

1. Which store formats, fixture types and departments would the data cover?
2. Are planograms versioned and linkable to the date and slot of each recording?
3. Do before-and-after shelf photos exist, and who owns them: the retailer, the brand or the merchandiser?
4. Can recording happen overnight or after close, with no shoppers present?
5. Are gap scans or out-of-stock events logged with timestamps and slot IDs?
6. How are damaged and expired units handled and recorded?
7. Which product images include third-party artwork, and how would the license treat them?
8. Do associates' releases cover AI training and licensing to a third party?

Discuss your requirements

If you are scoping a restocking deployment, describe the store formats, fixtures and labels you need to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. It reviews each requirement and follows up about potential sources; a request does not guarantee a match. Related tasks are listed on the workplace task demonstration hub, and the humanoid training data guide covers the full section.

Describe the workplace demonstration data your robotics project needs

Sources

• Precise Detection in Densely Packed Scenes — densely packed retail shelf images (SKU-110K).
• Products-10K: A Large-scale Product Recognition Dataset — SKU-level product recognition with in-shop and customer images.
• RPC: A Large-Scale Retail Product Checkout Dataset — controlled single-product images vs multi-product scenes.
• ARMBench: An Object-centric Benchmark Dataset for Robotic Manipulation — robotic pick-and-place records in a warehouse workcell.