Short answer
Dishwashing and dish-racking demonstrations show how people scrape, sort, spray, stack and load wet, cluttered items under time pressure. Recorded human work can support step recognition, object and state understanding and evaluation. It does not provide robot actions, grip forces or reliable views through spray and steam, so buyers combine it with wet-environment-specific robot or instrumented data.
Why the dish room is a distinct task
A dish room is repetitive, wet and visually hostile. Items arrive mixed and soiled, move through a sequence of stations, and must be placed into racks in orientations that let water drain and spray reach every surface. The person doing the work uses fast, coarse motions for most items and slower, careful ones for glassware.
This differs from commercial kitchen food prep, where the objects are food and the hazards are cutting and heat. It also differs from front-of-house work covered in restaurant service, bussing and reset. Here the defining features are repetition, moisture and tight stacking tolerances.
Task anatomy
| Station | What happens | Hard parts for a learner |
|---|---|---|
| Receiving | Trays and bus tubs arrive, items are unloaded | Mixed items, food residue, overlapping stacks |
| Scraping and pre-rinse | Scrape plates, spray soil off, sort silverware | Spray occludes view; wet hands and gloves |
| Sorting | Plates, bowls, cups, glassware, flatware, pans separated | Fast category decisions on soiled items |
| Racking | Place items into color- or shape-specific racks | Orientation, spacing, nesting, avoiding chips |
| Machine loading | Push rack into machine, close door, start cycle | Heavy racks, steam, door mechanics |
| Unloading | Remove hot, wet racks, check for residue | Heat, slippery items, quality check |
| Stowing | Dry and stack clean items by type | Stack stability, delivery to stations |
Failure modes: broken glassware, nested bowls trapped with water, flatware falling through peg gaps, a plate placed upright so it blocks the spray arm, racks overloaded beyond a safe weight, and items sent for re-wash after a visible residue check.
What the data can support
- Step and station recognition. Segmented video lets a model separate scraping, sorting, racking and unloading. Annotation style from EPIC-KITCHENS-100, which used narration followed by transcription, parsing and temporal segmentation across 45 kitchens and 90,000 actions, is a model for labeling long, repetitive kitchen sequences (Damen et al., 2020).
- Object category and state. Soiled vs clean, wet vs dry, nested vs separated. These labels support detection and state classification, which are separate from handling skill. See object recognition vs manipulation data.
- Placement conventions. Which items go in which rack, in what orientation and spacing. This is structured, learnable task knowledge.
- Pace and rhythm. Recordings show how fast an experienced worker cycles, relevant to working pace in demonstrations and cycle-time targets.
- Evaluation clips. Racked-load images and unload inspection outcomes provide a way to score whether a policy loaded correctly.
What it cannot support
- Robot actions and grip force. A person holding a wet plate with one hand while the other sprays has a grip strategy that cannot be read off video. Fragile and slippery handling needs force or tactile sensing, and a robot hand's compliance matters; see fragile item handling.
- Deformable and nested items. Soft sponges, bags, cloths and stacked bowls behave in ways that video shows poorly; see deformable object handling.
- Through-spray perception. Fogged lenses, water droplets and steam degrade the image. Footage from a phone at the dish pit shows what a human sees, not what a robot camera would see in the same spray.
- Safety under heat and chemicals. Machine chemistry and temperature limits are not visible.
- Coverage of one kitchen's ware. Restaurants, hospitals and universities use different crockery, trays and rack types. One site's data may not transfer.
Capture choices
| Option | Best for | Constraint |
|---|---|---|
| Head-mounted camera with protective housing | Operator view of hands, items and rack | Lens fouling; sealed or shielded housing needed |
| Fixed camera over rack station | Packing density and orientation labels | Mount away from spray; fixed viewpoint limits |
| Fixed camera at unload | Residue and quality check outcomes | Steam; add lens heating or timing |
| Two viewpoints, synchronized | Coverage of both hands and rack | Clock sync; see multi-view synchronized recording |
Existing footage from dish rooms is rare because few businesses record them. More often, a supplier hosts a commissioned session with scripted tasks and recruited workers; existing footage versus commissioned recording covers the tradeoffs.
Illustrative dish-room collection spec
# Illustrative only; synthetic values
dataset: dish_room_v0
sites: 3 (restaurant, hospital kitchen, campus dining)
operators: >= 6 per site, mixed experience, gloved and ungloved
scenes:
load_mix: [plates_heavy, glassware_heavy, flatware_heavy, mixed_tubs]
soil_level: [light, moderate, heavy]
cameras: head (sealed housing) + fixed over rack station
labels:
steps: scrape, presort, rack_plate, rack_glass, rack_flatware, load_machine, unload, stow
rack_state: per-rack image at load completion with item count by category
outcomes: rewash_flag, breakage_event, spill_event
gloves: true/false per clip
excluded: robot actions, contact force, water temperature (not recorded)
Acceptance criteria
- Hands are visible for most of each step in a random sample, including during spray.
- Rack-state labels reconcile with item counts from the load log or a second reviewer.
- Breakage and rewash events are retained, not trimmed.
- Items include the glassware, stoneware and metal types your target kitchen uses.
- Held-out evaluation clips come from sites and operators not used in training; see demonstration video acceptance criteria.
Permissions and diligence
Dish-room staff may have little say over cameras mounted above their station, so ask how consent was obtained and whether declining carried any consequence; see worker consent and identifiable people. Guidance from the EDPB on video devices is a reference point for questions about proportionality, notice and retention (EDPB, 2020). This is not legal advice. Hospitals and universities add their own policies on cameras near patients, students and food-safety areas.
Questions to ask suppliers
- Was the footage recorded for training, safety or monitoring, and who knew it was running?
- Which ware types and rack types are represented?
- Are there views of both hands during spraying, or only a fixed overhead?
- Are glove use and wet conditions labeled?
- Are breakage, rewash and spill events included?
- How many distinct workers and sites appear?
- Can the supplier host a scripted session with a sealed head camera?
- Who owns recordings made on a hospital or university premises?
Discuss your requirements
If you are scoping dish-room data, describe the items, conditions, modalities and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. If nothing existing fits, SourceX can scope a commissioned collection of new recordings. A request does not guarantee a match; SourceX follows up about potential sources.
Start with the humanoid training data guide or the workplace task demonstration hub. Related catalog entries: first-person video of skilled manual work and egocentric video of manual work. For a related contact-rich primitive, see wiping and surface cleaning demonstrations.
Describe the workplace demonstration data your robotics project needs
Sources
- Rescaling Egocentric Vision — Egocentric kitchen video with narration-based action annotation
- Guidelines 3/2019 on processing of personal data through video devices (Version 2.0) — Guidance on personal data processed through video devices