Short answer
Produce packing and grading recordings show how workers inspect, sort, orient and place natural items that differ in size, color, ripeness and blemish. They can support grade classification, placement conventions and evaluation. They do not supply robot actions or bruising forces, and grade labels are only as consistent as the graders who made them.
What makes produce different
Natural products have no CAD model. Two apples of the same variety differ in diameter, shape, color distribution, surface defects and firmness. A packer judges each one in a fraction of a second, decides its grade, and places it in a tray, clamshell or carton in an orientation that prevents damage and meets a count or weight target.
That combines two skills that data should separate. The first is a quality judgment: which grade does this item get? The second is handling: how to pick it up without bruising, and where to put it. Image and label data support the first. The second needs demonstrations, and in the end robot or instrumented-hand data. Do not assume a labeled grading dataset teaches gentle handling.
Task anatomy
| Stage | Typical work | Data that helps |
|---|---|---|
| Receiving and tipping | Bins emptied onto a line or table | Volume and condition of incoming lots |
| Inspection | Remove culls, foreign material, damaged fruit | Images with reject/accept labels |
| Grading | Sort by size, color, defect class | Grade labels and standard references |
| Orientation | Stem up, face up, label side out | Placement conventions |
| Packing | Fill trays, clamshells, bags or cartons to count or weight | Pack-pattern images and counts |
| Close and label | Lid, seal, label, stack | Outcome checks |
Failure modes: grade drift between graders or shifts, bruising during placement, fruit rolling after release, wrong count, missed defects on the underside, and mislabeled variety.
What the data can support
- Grade and defect classification. Images of produce with grader labels support classifiers for size class, color maturity and visible defect. This is closer to defect image datasets than to manipulation data.
- Placement and pack patterns. Finished-tray images with counts show conventions a planner can follow.
- Step and cycle structure. Video of packers supports step recognition and pace analysis.
- Contact-rich sequences as a model. RH20T paired more than 110,000 contact-rich real-world manipulation sequences, each with visual, force, audio and action data, with human demonstration video and language descriptions (Fang et al., 2023). That is the kind of package a handling policy needs; packhouse video on its own has only the visual side.
- Evaluation sets. Held-out lots, varieties and seasons test whether a grade classifier or picking policy generalizes.
For narration-based annotation of egocentric video that a packhouse session could borrow, EPIC-KITCHENS-100 describes a pause-and-talk approach with transcription and temporal segmentation (Damen et al., 2020).
What it cannot support
- Handling force. Bruise risk comes from grip force, drop height and contact speed, none of which are visible in video. Consider fragile item handling and deformable object handling for the data you would add.
- Firmness and internal quality. Ripeness, sugar content and internal defects are not visible. A grader may use touch or a meter. Ask for those measurements.
- Robot actions. Packers use two hands, fast pinch grasps and rolling motions. See the embodiment gap.
- Seasonality. One harvest, variety or storage condition shapes the appearance. A dataset from one month may not transfer to another.
- A single true grade. Graders disagree at boundaries. Treat grade labels as human judgments with a measured agreement rate.
Images or video?
| Need | Better modality | Reason |
|---|---|---|
| Grade classifier | High-resolution still images, multiple views per item | Defects need pixels; motion adds little |
| Packing pattern | Images of finished trays | Counts and orientation checked at the end |
| Handling and sequence | Video from head-mounted and fixed cameras | Shows the motion and timing |
| Rolling and settling | High-frame-rate fixed view | Fast dynamics |
Specify the lighting. Packhouse lines often use controlled lighting for graders, which can differ greatly from the lighting a robot will see.
Illustrative packhouse package
# Illustrative only; synthetic values
package: produce_pack_v0
crop_variety: apple_var_03 (synthetic ID)
lots: lot_0112..lot_0127 across two harvest weeks
sites: packhouse_007, packhouse_012
images:
per_item_views: 4 (top, bottom, two sides), 12 MP, calibrated color target in frame
labels: size_class, color_class, defect_types[], grade, grader_id (pseudonymous)
video:
head_camera: packer view, 30 fps
fixed_camera: pack station overview
labels: step segments, tray_complete, count_error, damage_event
measurements:
optional: firmness_reading, weight_g (where logged)
agreement:
double_graded: 10% of items, agreement reported per class
splits: by lot, harvest week and packhouse
Acceptance criteria
- Double-graded items are included and agreement is reported by grade, not only overall.
- Color targets or consistent lighting allow comparison between sites.
- Cull and defect classes include rare types, with counts.
- Video clips include damage and count-error events, not only clean runs.
- Splits keep a lot, and ideally a variety, wholly in either training or evaluation.
Suppliers, permissions and provenance
Typical suppliers are packhouses and growers. Packhouses may hold grading images from their sorting machines, which are not the same as demonstration video; ask which of the two exist and whether the machine-vision vendor retains any rights to the images. Customer specifications (retailer grade standards) may be confidential. Workers appear in video; see worker consent and identifiable people. Seasonal workers may be employed through labor contractors, so ask who can give consent on their behalf. This is not legal advice.
Groceries and packed goods share some downstream handling with grocery bagging and order assembly, and growing environments with greenhouse and nursery tasks. For how outcome labels are built, see success and outcome labels.
Questions to ask suppliers
- Which varieties, seasons and lots are covered, and how many items per grade?
- Were grades assigned by humans, a sorting machine or both?
- Is there a measured agreement rate between graders?
- Do images and video come from the same items so they can be linked?
- Are damage events and rework preserved?
- Is lighting and camera placement documented?
- Who owns the images captured by a sorting machine on the line?
- Do retailer specifications restrict licensing of grade standards or labeled images?
Discuss your requirements
If you are scoping packing or grading data, describe the produce types, modalities, label needs and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. If nothing existing fits, SourceX can scope a commissioned collection of new recordings. A request does not guarantee a match; SourceX follows up about potential sources.
Start at the humanoid training data guide or the workplace task demonstration hub. Related catalog entries: images and inspection photos and first-person video of skilled manual work.
Describe the workplace demonstration data your robotics project needs
Sources
- RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot — Contact-rich sequences with force and audio alongside human demonstration video
- Rescaling Egocentric Vision — Narration-based annotation approach for egocentric kitchen video