Short answer
Recycling and waste-sorting data mostly supports perception: detecting and segmenting deformable, dirty, overlapping items on a moving stream. Images from a sorting line and video of people picking can also support pick-priority and timing studies. They cannot provide robot grasp actions, forces on crushed or wet items, or the sorting rules a facility applies.
The task and why it is hard
A material recovery facility (MRF) moves mixed material past sorters on a conveyor. Items overlap, bags hide contents, labels are torn, liquids leak, and a bottle can be flattened, crushed or filled with dirt. A human sorter chooses which item to pick, in a fraction of a second, from a pile that keeps moving.
Waste picking combines three problems: finding items in clutter, deciding the target material class, and grasping something with unpredictable shape and compliance. Each needs different data. Buyers who label all three as "waste sorting data" risk ending up with imagery that addresses the first and not the third.
Task anatomy
| Element | Example | Data consequence |
|---|---|---|
| Stream | Mixed paper, plastics, metals, residue on a belt | Belt speed, density and fill vary with the day and the route |
| Target | Pull one material class, such as clear PET | Class definitions and purity thresholds set by the facility |
| Item state | Crushed, wet, soiled, bagged, translucent | Appearance varies within a class |
| Pick | Grab, lift, release into a chute | Timing is tied to belt motion |
| Miss handling | Item continues down the line | Missed picks and rework are part of normal operation |
| Contaminant removal | Hoses, film, textiles tangled in machinery | Safety-critical and rare |
Failure modes: translucent items missing from a detector, objects occluded by others, grabbing two items at once, picking a contaminant, and belt-speed changes that break a timing assumption.
What the data can support
- Detection and segmentation of deformable items in clutter. ZeroWaste was presented as the first in-the-wild, industrial-grade waste detection and segmentation dataset, targeting highly deformable, often translucent objects in extreme clutter on a recycling sorting line. Its project page lists subsets of 4,503 images (ZeroWaste-f), 6,212 (ZeroWaste-s) and 1,410 (ZeroWaste-w) (Bashkirova et al., 2021). The same paper notes that less than 35% of recyclable waste is recycled in the U.S. It is a useful reference for what a labeled sorting-line dataset looks like and how hard segmentation in clutter is.
- Material classification. Images tagged with material class support class-level decisions, though visual class is not always the true material.
- Pick behavior. Video of human sorters shows target choice, reach path and timing against belt motion; see working pace in demonstrations.
- Rare-event coverage. Hazards such as batteries, needles or propane canisters can be flagged in images for alerting; see rare events in demonstrations.
- Evaluation. Held-out belts, days and facilities test generalization.
What it cannot support
- Grasp actions. Images and even human pick video do not give gripper poses, finger closure or suction states. For contact-rich context, RH20T pairs more than 110,000 manipulation sequences with force, audio and action data per sequence and a human demonstration video each (Fang et al., 2023); this page cites no equivalent package for waste picking.
- Forces on compliant items. A half-full bottle, a plastic bag and a crushed can respond differently to a grasp.
- Facility decisions. Which material to pull this hour depends on bale specifications and market grades that are not visible.
- Material truth. A visual class is not a lab-verified polymer. Ask how classes were assigned.
- Seasonal and regional change. Waste composition varies with region, collection program and season.
Capture considerations
Fixed overhead cameras are standard at MRFs and give consistent geometry; fixed-camera footage limits explains what that viewpoint cannot show. Lighting, dust and belt vibration affect image quality, so specify exposure and a frame rate high enough to avoid blur at belt speed. Labeling approach matters: instance masks cost more than boxes, and deformable, translucent items are slow to outline. See object tracks and masks for tracking labels in video.
Stream shape also changes by position on the line. Pre-sort, post-sort and residue lines look different, so record where each camera sits.
Illustrative dataset specification
# Illustrative only; synthetic values
dataset: mrf_stream_v0
facilities: mrf_003, mrf_009
views:
overhead_rgb: fixed, 20 fps, belt speed logged
pick_station_video: sorter view, optional
labels:
instance_masks: item polygons, class, occlusion_level
classes: pet_clear, hdpe, aluminum_can, paper_mixed, film, textile, contaminant, other
attributes: crushed, wet, bagged, translucent
pick_events: item_id, picker_id (pseudonymous), outcome (sorted / missed / dropped)
metadata: belt_speed_class, stream_position, shift, date, weather_class
splits: by facility, week and stream position
excluded: grasp poses, forces, bale specifications
Acceptance criteria
- Class definitions are written, with examples per class, and the same definitions are used across sites.
- Mask quality is checked on a sampled subset with reported agreement between annotators.
- Occluded, translucent and wet items are represented, not filtered out as hard.
- Belt speed and stream position are recorded with each clip.
- Contaminants and hazards appear in enough examples to evaluate.
- Held-out evaluation sets come from different days and facilities.
Suppliers and provenance
Likely suppliers are material recovery facilities and recyclers, some of which run camera-based sorting equipment. Ask whether imagery belongs to the facility or to the sorting-equipment vendor. Facilities also hold information about incoming customers and haulers, and photos may show branded packaging or documents inside bags. Workers on the line may be visible; see worker consent and identifiable people. This is not legal advice. For deformable-object handling beyond waste, see deformable object handling.
Questions to ask suppliers
- Were images captured from the facility's own cameras or from a sorting-equipment vendor's system?
- How were material classes assigned, and by whom?
- Is belt speed, line position and time logged with each image?
- Does the set include translucent, crushed, wet and bagged items?
- Are hazard items such as batteries represented?
- Are there video views of human pickers, with consent?
- Does the data span seasons and incoming-route changes?
- What do customer or hauler contracts say about imagery?
Discuss your requirements
If you are scoping sorting data, describe the material streams, modalities, label needs and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. If nothing existing fits, SourceX can scope a commissioned collection of new recordings. A request does not guarantee a match; SourceX follows up about potential sources.
Start with the humanoid training data guide or the workplace task demonstration hub. Related catalog entries: images and inspection photos and robotics and embodied AI use cases.
Describe the workplace demonstration data your robotics project needs
Sources
- ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes — Waste detection and segmentation dataset for deformable items in clutter on a recycling line
- RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot — Contact-rich sequences pairing force data with human demonstrations