Short answer
Recycling recognition data consists of images of waste streams labeled by item, material and contamination, ideally from real sorting facilities. Public work such as ZeroWaste shows the setting: deformable, often translucent items in heavy clutter. Image labels support detection and segmentation, but sorting a stream also needs pick demonstrations, belt speeds and facility-specific rules.
What makes waste recognition different
Most object datasets show clean, intact, well-lit items. A recycling stream shows crushed bottles, film wrapped around cardboard, items covered in food residue and everything piled together. The label a sorter needs is often a material or a recovery grade, not a product name, and the same item can belong to different classes depending on state: a rinsed PET bottle versus one full of liquid, a flattened carton versus a crushed one.
The ZeroWaste dataset describes itself as the first in-the-wild industrial-grade waste detection and segmentation dataset, targeting highly deformable, often translucent objects in extreme clutter on a recycling sorting line. Its authors note that less than 35% of recyclable waste is recycled in the U.S. The project page lists subsets of 4,503 images (ZeroWaste-f), 6,212 (ZeroWaste-s) and 1,410 (ZeroWaste-w) (Bashkirova et al., 2021).
Those figures show the scale of a research-grade collection from one facility type. They do not tell you whether it covers your region's streams or your facility's belt. This page is a sourcing guide for recognition imagery; sorting demonstrations are a different asset and are not covered here.
What this data can and cannot support
It can support detection and instance segmentation of waste items, material classification (paper, plastic types, metal, glass, organic), contamination estimates, and stream composition monitoring. Segmentation masks matter because items overlap; see segmentation masks for products.
It cannot support picking. Whether an item can be lifted with a suction cup or a gripper, how a bag of mixed content deforms, and how fast the belt moves are properties of the robot, the line and the item, and none are in a labeled image. Images from a fixed camera above a conveyor also differ from a humanoid's moving view; if the robot is a mobile manipulator in a bin area, a ceiling camera dataset is likely to transfer only partially. And visual material classification has limits: look-alike plastics, black plastics and multilayer packaging can be indistinguishable in RGB images. Spectral sensing (for example near-infrared) may be needed, and that is a different sensor modality that image archives rarely include.
Classes: define them before you buy
| Decision | Options | Consequence |
|---|---|---|
| Label basis | Item type, material, recovery grade, brand | Item labels are easier to see; material labels are what the line needs |
| Granularity | Broad (paper, plastic) vs specific (PET bottle, HDPE jug) | Specific classes need more examples per class |
| Contamination | Clean, soiled, liquid-filled | Affects value and pick strategy |
| State | Whole, crushed, flattened, bagged | Changes appearance and grasp |
| Out-of-scope items | Hazardous, electronics, textiles | Needs a reject class and a safety path |
| Unknown | Not identifiable | Needs an explicit label, not forced guesses |
Recycling programs differ by municipality and facility. A class scheme from one material recovery facility may not match another's, so write your scheme in the contract and require a mapping from the supplier's.
Supplier types
Material recovery facilities and recyclers are the main source: they see the stream, run cameras for process control, and can show real belt conditions. Their value is realism; their constraints are site access, shift schedules and the presence of staff and customer material in views.
Waste haulers and municipalities may hold route-level audit photos, which show input composition but not belt conditions.
Packaging producers and brand owners hold clean images of their own items but not discarded ones.
Recycling technology vendors often hold labeled imagery from their own systems, but rights and terms differ; ask who owns the labels and the images.
Compare with the sibling topic cluttered scenes for recognition for general density, and object state recognition for state labels.
Illustrative specification
The block is Illustrative; field values are synthetic.
# Illustrative waste-stream image spec - synthetic values
site: mrf_example_03
sensor: {type: RGB, mount: overhead_fixed, height_m: 2.0, fps: 5, resolution: 2048x1536}
conditions: [three shifts, wet and dry loads, belt speed recorded, dust on lens logged]
labels:
instance_mask: required
material_class: [paper, cardboard, pet, hdpe, other_plastic, film, metal_ferrous,
metal_nonferrous, glass, organic, textile, unknown]
item_type: free_text_plus_controlled_list
contamination: [clean, soiled, liquid_filled, unknown]
state: [whole, crushed, flattened, bagged, tangled]
occlusion_fraction: 0-1
hazard_flag: [battery, pressurised_can, sharps, none_seen]
splits: by day and by load, not by frame
documentation: datasheet with sampling method (all frames vs events)
Acceptance criteria
- Class mapping. A document that maps supplier classes to yours, with examples of each.
- Clutter and occlusion statistics. Distribution of items per image and visible fraction.
- Unknown share. A reported share of "unknown", not a forced label on every item.
- Annotation audit. A sample of masks checked for edge quality on transparent and thin items, which is where weak annotation shows. Related: transparent and reflective objects.
- Temporal leakage. Consecutive frames from a belt are near-duplicates; confirm splits by time.
- Seasonal and regional range. Streams change by season, holidays and local collection rules.
- Hazard handling. How batteries, aerosol cans and sharps were flagged.
Permissions and provenance
Waste streams can contain customer-originated material: letters, medical items, personal documents or devices. Images may show workers on the line. Ask what filtering and notice apply, who owns labels made by system vendors, and whether the facility's agreements with municipalities or customers limit image use. Redaction does not settle permission questions on its own. This is not legal advice.
Aligning images with the line
If the robot will work beside a conveyor, ask whether the supplier can tie image frames to the belt speed, the position along the line and the timestamps of any downstream diverter or pick events. That linkage lets you check which items were actually removed and which passed, which is a stronger label than a human's guess from a still image. It is rarely present in archives and often has to be arranged at capture time with the facility's controls team.
Limitations and extra preparation
Expect to remap classes, remove duplicate frames and add your own robot-view imagery. For deformable and bagged items, add physical properties (deformable product properties) and packaging data (packaging material properties). Document what you received with a datasheet following Gebru et al.. Do not treat recognition accuracy on images as sorting purity; purity is measured on the line.
Evaluating a sample
Ask for a sample covering at least two different days and loads. Sort the frames by number of items and review the densest ones, because easy frames flatter any dataset. Open a few masks on film and transparent bottles at full zoom and check where the boundaries fall. Finally, ask the supplier to point out five images they consider hard to label; a supplier who can do that has probably looked closely at their own data.
Questions to ask suppliers
- Which sensor, mount, resolution and frame rate produced the images?
- Are images from a belt, a tipping floor, a bale or a bin?
- Which classes are used, and who defined them?
- How were masks made, and what is the audit result on thin and transparent items?
- Are contaminated, wet and crushed items included?
- How were duplicates and near-duplicates handled in the sampling?
- What notice covers workers visible in frames?
- Do you hold any spectral or depth data aligned with the images?
Discuss your requirements
Start with the material classes, the sensor and the facility type. When you describe the recycling recognition requirement to SourceX, SourceX reviews it and follows up about potential sources among US businesses, and whether new imaging at a facility is feasible is discussed case by case. A request does not guarantee a match. See the humanoid training data guide and the object data hub.
Discuss your humanoid training-data requirements with SourceX
Sources
- ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes — waste detection and segmentation in cluttered recycling scenes; subset sizes.
- Datasheets for Datasets — documenting composition and collection process.