Short answer
Specify defect annotations by geometry (box, polygon or mask), by container (COCO-style JSON, per-image mask files or another schema), and by the metadata that must travel with them: class map, coordinate convention, split files and a manifest with checksums. Conversion between geometries loses information, so choose the finest geometry your model needs before delivery, and settle who owns the annotations.
Why format belongs in the contract
Defect labels reach a buyer in whatever form the supplier's tool exported: a spreadsheet of coordinates, XML files from a labeling tool, PNG masks in a folder, or a proprietary vision-system log. A contract that says only "labeled images" invites all of these. The cost shows up later, when boxes turn out to be too coarse, class names differ between batches, or half the masks use a different value for background.
This page covers specification and delivery terms. For how to choose the label geometry for a model, see pixel-level defect segmentation; for class definitions, defect taxonomy and label schema.
Geometry: choose it before collection ends
| Geometry | Suited to | Loses |
|---|---|---|
| Image-level label | Pass/fail or class screening | Location and extent |
| Bounding box | Detection, counting, localization of compact defects | Shape; wasteful for thin or diagonal defects such as cracks |
| Polygon | Irregular defect outlines at moderate cost | Interior holes and fine edge detail depending on vertex count |
| Raster mask | Pixel-level segmentation, area measurement | Storage economy; instance identity unless instance masks are used |
Going down the table is easy: a mask yields a box or an image label. Going up is not. A box cannot be turned into a faithful mask without new annotation. Mixed supervision work on surface-defect detection reports that a few pixel-labeled samples performed comparably to full supervision at much lower annotation cost (Božič et al., 2021), so it can be reasonable to request masks for a subset and boxes or image labels for the rest. That reflects one study's setting; test it on your own data.
Container formats
Common containers include a single COCO-style JSON file listing images, annotations and categories; per-image XML in the Pascal VOC style; one text file per image in the normalized-coordinate style used by some detector toolkits; and mask images stored as PNG with class or instance values. COCO itself is a public instance-annotated dataset with 328k images and about 2.5 million labeled instances across 91 categories, with per-instance segmentation collected via crowd workers (Lin et al., 2014); "COCO format" in practice commonly refers to a JSON layout derived from it, with variants. Because the details have variants, write the exact conventions into the specification and confirm them against the format's own documentation, not against this page.
Fields that must be pinned down regardless of container:
- Coordinate origin, units (pixels or normalized) and box convention (corner-corner or origin-plus-size).
- Polygon winding, closure and how holes are represented.
- Mask encoding: bit depth, background value, class value versus instance value, and whether run-length encoding is used.
- Overlap handling: can masks overlap, and how are crowded defects represented?
- How image size, rotation and EXIF orientation relate to coordinates.
Class map and metadata
A class map is the contract for meaning. It lists numeric IDs, names, definitions, example images and the version of the taxonomy. Without it, a mask value of 3 means nothing. Include a documented process for unknown or ignored regions ("ignore" labels for ambiguous areas), because treating uncertain pixels as background trains the wrong behavior.
Also request per-annotation attributes where they matter: annotator or role, review status, confidence or "borderline" flag, and the imaging mode. Imaging conditions metadata lists what to attach to the image itself.
Illustrative delivery manifest
# Illustrative only - names, counts and checks are examples a buyer would set
delivery: inspection_annotations_v0
version: 1.0.0
images_dir: images/
annotations:
container: coco_style_json
file: annotations/instances.json
masks_dir: masks/ # optional, PNG, 8-bit, background=0
class_map: class_map.yaml # id, name, definition, taxonomy_version
conventions:
box: [x_min, y_min, width, height]
units: pixels
origin: top_left
splits:
files: [splits/train.txt, splits/val.txt, splits/test_heldout.txt]
split_key: lot_id # chosen by buyer
manifest: manifest.csv # file path, sha256, bytes, image_id
documentation: datasheet.md # composition, collection process, known gaps
checks:
- every annotation image_id resolves to a file
- every class id appears in class_map
- no image in more than one split
- polygons closed and non-self-intersecting
excluded: robot actions, joint states, forces
A datasheet in the style proposed by Datasheets for Datasets covers motivation, composition and collection process and is a sensible companion file.
Delivery mechanics and acceptance
Specify transfer method, file naming, directory layout and checksums, and require a machine-readable manifest. Run acceptance tests against the manifest checks above before sign-off, and agree how corrections are delivered: a patch with new version number is better than overwriting files in place. For splits, require the split files to be delivered by the supplier or specified by you, and never allow test images to be re-shuffled into training; the leak patterns are described in leakage in defect datasets.
Legacy labels often need conversion rather than recollection; relabeling legacy defect archives covers the choices.
Rights and terms to settle
Annotations are a separate asset from the images, and ownership can differ: a supplier may own its original labels, an annotation partner may claim rights to work product unless the contract says otherwise, and the buyer's own relabeling creates derivative labels. Questions for the license include whether annotations may be used for training and evaluation, whether the buyer may keep and modify them, whether the supplier or a partner retains reuse rights, and what happens on termination. See licensing terms for robotics training data and the site guide to data license terms. This is not legal advice.
What a format cannot fix
A neat container does not make labels correct. Consistent COCO-style JSON can hold inconsistent class definitions, noisy boxes or leaked splits. Format also cannot add content the data never had: annotations on inspection images describe defects in pictures and contain no robot actions, joint states or force data, so they cannot train manipulation or humanoid control, which needs robot or teleoperation data. And no format is universally best; the right choice depends on the model, tooling and evaluation method.
Questions to ask suppliers and annotation partners
- In what tool and export format are your labels held today, and can you deliver the format in our specification?
- Are annotations stored as boxes, polygons or masks, and was the geometry assigned at labeling or derived?
- Is there a class map with definitions and a taxonomy version?
- How are ambiguous and ignore regions represented?
- What review or consensus process produced the labels, and is review status stored?
- Can you provide checksums, a manifest and split files?
- Who owns the annotations, including any work done by a third-party annotator?
- Can corrections be delivered as versioned patches?
- What documentation accompanies the delivery on composition and known gaps?
Discuss your requirements
If you are specifying an inspection delivery, describe the geometry, format, splits and permitted uses to SourceX. SourceX's role is sourcing, rights review, licensing and delivery coordination; annotation can be arranged case by case, and a request does not guarantee a match. Start from the inspection data hub or the humanoid training data guide, or see the catalog entry on expert annotations and labels.
Tell SourceX what your evaluation set needs to cover
Sources
- Microsoft COCO: Common Objects in Context — scale and instance-level annotation of a public dataset; format details to be verified against the format's own documentation.
- Mixed supervision for surface-defect detection: from weakly to fully supervised learning — mixed supervision with few pixel-labeled samples.
- Datasheets for Datasets — documentation practice for delivered datasets.