Short answer
An instance segmentation spec for products defines what counts as one instance, how occlusion, transparency and shadows are handled, the mask format, and how quality is sampled and accepted. Masks on catalog or warehouse images support detection and picking perception. They show where an item is, not how it behaves when grasped, so manipulation still needs robot data.
The buyer's problem
Existing product images usually have labels at the wrong level. A catalog image has a SKU. A warehouse photo has a tote ID. Neither has a pixel mask for each item. A perception team that wants to train or evaluate an instance segmenter, for example to isolate one item from a pile before picking, needs that mask layer and needs it to follow rules that match the downstream use.
In practice, ambiguous annotation rules can undermine segmentation work as much as model choice does. Is a six-pack one instance or six? Does a mask include a hand holding the item? What happens where a bag is crumpled over a box? This specification is meant to be adapted: copy the structure, change the values to fit your architecture and evaluation.
Scale reference from published work
The COCO dataset gives a sense of the annotation effort involved. It reports 328k images, about 2.5 million labeled instances and 91 object categories, with per-instance segmentation collected through crowd workers (Lin et al., 2014). It is a general-object benchmark, not a product dataset, and its categories do not match retail or industrial items. A buyer should read it as a precedent for the annotation format and process, not as a coverage promise.
What the data can support and what it cannot
Instance masks can support:
- Training and evaluating segmentation and detection models on products.
- Separating touching items in totes and on shelves before pose estimation.
- Deriving bounding boxes, crops and areas for downstream tools.
- Seeding grasp-point or pose labels, which need further annotation. See grasp annotations for products.
- Evaluation sets for picking perception when held out properly.
They cannot support:
- Handling. A mask does not say how an item deforms, slips or weighs. Control needs robot or teleoperation data, not images alone. See object recognition data vs manipulation training data.
- 3D structure. A 2D mask does not give depth or pose without extra sensors or models.
- Unseen conditions. Masks cover the images supplied. Different lighting, packaging revisions and clutter levels need their own samples.
- Rights. A mask does not change who owns the source image.
Illustrative annotation specification
The specification below is Illustrative. Choose each rule to fit your task; the values are examples.
# Illustrative only - not an available dataset
task: instance_segmentation_products_v0
images: licensed catalog and in-situ photos # source set decided by buyer
classes:
scheme: sku_level or category_level # buyer decides; do not mix
unknown_item: "other_product"
instance_rules:
multipack: one instance for retail pack; inner units not labeled
sets_and_kits: one instance per sealed package
items_in_bags: mask the bag, not the contents
held_items: mask the object only; exclude hand and glove
occlusion:
visible_only: true # mask covers visible pixels
amodal_field: optional # separate polygon for full extent, if requested
min_visible_fraction_to_label: 0.10 # example threshold
boundaries:
transparent_items: mask outer edge of the container; flag transparent=true
shadows_reflections: excluded
labels_and_hang_tabs: included as part of the item
format:
masks: COCO-style polygons or RLE; per-instance ID
image_ids: stable; map to source and rights record
quality:
second_review_sample: 10% # example; buyer sets
boundary_iou_check: on sampled instances
disputed_cases: logged with rule number
Rules that most affect downstream results
| Rule | Why it matters | Failure if unspecified |
|---|---|---|
| Visible-only vs amodal masks | Picking logic may need full extent | Models learn inconsistent edges |
| Multipack handling | Defines the pick unit | Duplicate or merged instances |
| Hand and glove exclusion | Keeps masks to the object | Hand pixels leak into object class |
| Transparent items | Edge is hard to see | Annotators trace contents instead |
| Minimum visible fraction | Controls noisy tiny labels | Tiny slivers swamp metrics |
| Class scheme | SKU, category or generic | Mixed levels break evaluation |
Acceptance tests
- Boundary agreement. Have a second annotator re-label a random sample and report mask overlap. The buyer sets the threshold.
- Rule conformance. Spot-check named hard cases: multipacks, bags, transparent items, tangled cables.
- Completeness. Count unlabeled visible items in a sample.
- Consistency across images. The same SKU should be masked under the same rules.
- Format validity. Masks load, IDs are unique and every mask maps to a rights record.
- Held-out integrity. Evaluation images are separated by SKU, site or time as your claim requires. See object recognition evaluation metrics.
Training and evaluation masks deserve different review budgets. Small labeling errors in a large training set may be tolerable, but errors in an evaluation set change the score you report. Consider a full second review for evaluation images, keep them out of any rule-revision pilot, and never move them back into training once results have been reported.
Source images and rights
Masks are only as usable as the images under them. Ask whether the source images are catalog packshots, warehouse captures or purpose-built photos, and who holds the photographer and brand rights. A new annotation layer is usually a separate piece of work from the image license, and the supplier may not be the person who can license either. See product image rights. Images with people, such as worker hands in warehouse shots, bring consent questions. Blurring faces does not by itself settle them.
How commissioning typically works
A buyer who has images but no masks needs an annotation step. Annotation is a separate service from sourcing images and is scoped case by case. Start with a small pilot batch, run the acceptance tests above and revise the rules before scaling. For hard scenes, see cluttered scenes for recognition and transparent and reflective objects.
Questions to ask suppliers
- Who annotates, and what training and guideline documents do they work from?
- Are masks polygon, brush or model-assisted, and how is assisted output reviewed?
- How is second-review sampling done, and what disagreement is logged?
- How are multipacks, bags, transparent items and hands handled?
- Can you deliver amodal masks as a separate field?
- What formats and tools are supported, and are IDs stable across revisions?
- Which images are licensed for annotation and for training?
- Can the annotator pass back disputed cases for rule updates?
Choosing between SKU-level and category-level classes is its own decision; see instance vs category recognition.
Discuss your requirements
If you need masks on product images, or images that already have them, describe the classes, image types, mask rules and permitted uses to SourceX. SourceX reviews the requirement, looks for US businesses that may hold suitable images, checks the data and the supplier's licensing permissions, and follows up about potential sources. Annotation can be arranged case by case, and a request does not guarantee a match. See the objects hub, the humanoid training data guide and the catalog entry for expert annotations and labels.
Tell SourceX what your evaluation set needs to cover
Sources
- Microsoft COCO: Common Objects in Context — scale and process reference for per-instance segmentation.