Wiping, Scrubbing and Surface Cleaning Demonstrations

Why wiping needs its own data plan

Wiping looks simple and is not. A person adjusts pressure as soil type, surface material and cloth saturation change, usually without thinking about it. The visible result is a hand moving a cloth across a surface. The information that makes the motion work, such as normal force, shear, and the moment the cloth starts to skid, is not in the pixels.

That makes surface cleaning a contact-rich primitive. It is narrower than a full janitorial route, which adds sequencing, travel and restocking; for that layer see janitorial workflow demonstrations. This page covers the single reusable skill: applying a tool to a surface until a condition is met.

Task anatomy

A wiping or scrubbing episode usually has these parts:

1. Assess. Identify the soil (dried spill, film, grease, dust), the surface (stainless steel, laminate, glass, tile grout) and the allowed chemistry.
2. Prepare. Wet or dampen the cloth, apply cleaner by spray or pour, fold the cloth to expose a clean face.
3. Cover. Sweep with overlapping passes, in an S-pattern, circles or straight lines, moving from clean to dirty areas.
4. Scrub. Localize on stubborn soil with higher pressure, a pad or brush, and short strokes.
5. Check. Look at, touch or tilt the surface to judge residue and streaks.
6. Recover. Rinse or turn the cloth, repeat a missed region, move objects aside and put them back.

Failure modes worth collecting: streaks left by an oversaturated cloth, missed corners and edges, soil smeared instead of lifted, cleaner pooling on a tilted surface, objects knocked over during a pass, and cloth snagging on a seam.

What the data can support

Published work shows what a contact-rich dataset looks like when the goal is control. RH20T collected more than 110,000 contact-rich real-world manipulation sequences, each with visual, force, audio and action data, and paired each robot sequence with a human demonstration video and a language description (Fang et al., 2023). The pairing is the useful pattern: a human video shows the intended strategy while the robot sequence carries the signals the video lacks.

Cleaning recordings from people can support:

• Task and step understanding. Models that recognize "applying cleaner", "scrubbing", "rinsing" and "checking" as separate phases.
• Coverage and progress estimation. Learning when a surface has been fully traversed, a basis for reward or success detection.
• Visual grounding of soil. Distinguishing wet from dry, clean from soiled, residue from reflection.
• Language grounding. Matching "wipe the counter, then the sink rim" to observed sequences.
• Evaluation. Held-out recordings from other sites and operators to test whether a policy's behavior resembles accepted practice.

Egocentric annotation methods transfer here. EPIC-KITCHENS-100 covers 100 hours of head-mounted video across 45 kitchens and used a pause-and-talk narration stage, followed by transcription, parsing and temporal segmentation, to produce about 90,000 action segments (Damen et al., 2020). A cleaning dataset can reuse that narrate-then-segment approach, with verbs such as wipe, scrub, rinse and fold, plus hand-object interaction annotation for the grip and contact state.

What it cannot support

• Force and pressure. A camera cannot recover how hard someone pressed. Policies for wiping or scrubbing need force-torque readings, motor current, tactile sensing or an instrumented tool. See when video is not enough.
• Robot actions. Human arm motion is not a robot trajectory. Retargeting to a humanoid hand and arm loses information; see the human-to-humanoid embodiment gap.
• Cloth and fluid state. Saturation, cleaner concentration and cloth wear are rarely visible and are rarely logged.
• Outcome truth. Whether a surface is hygienically clean is not decided by a frame. If your claim needs it, add a measurement step such as an ATP swab or a supervisor's pass/fail record.
• Coverage of rare surfaces. Contractors clean the surfaces they have. Curved sinks, textured stone and glass walls may be absent.

Capture choices that matter

Existing footage usually has one camera and no instrumentation. Commissioned recording can add these, at the cost of recruiting demonstrators and arranging access. Existing footage versus commissioned recording covers that tradeoff.

Illustrative label schema for cleaning clips

# Illustrative only; values are synthetic examples
clip_id: clean_000417
site_id: site_017
operator_id: operator_A12
surface: {material: laminate, orientation: horizontal, area_class: medium}
soil: {type: dried_spill, severity: moderate}
tool: {kind: microfiber_cloth, state: damp, chemical: none_recorded}
phases:
  - {label: apply_cleaner, start_s: 2.1, end_s: 4.0}
  - {label: wipe_cover,    start_s: 4.0, end_s: 31.5}
  - {label: scrub_spot,    start_s: 31.5, end_s: 40.2}
  - {label: check,         start_s: 40.2, end_s: 43.0}
outcome: {result: passed, judged_by: supervisor_visual, residue_visible: false}
coverage_map: per-pass polygon masks on surface plane (optional, requires calibration)
contact_signal: none   # video-only; set to ft_sensor / imu if instrumented

Set the fields to your own architecture. A team that only needs phase recognition can drop the coverage map; a team training a contact policy cannot drop the contact signal.

Acceptance criteria a buyer can set

• At least a stated share of clips shows hand, tool and the entire working surface in frame, checked on a random sample.
• Phase boundaries agree between two annotators within a tolerance you define.
• Outcome labels come from a documented rule, not from the annotator's guess.
• Surface materials and soil types are listed with counts, so gaps are visible.
• Evaluation clips are held out by site and operator, never mixed into training.

Where cleaning footage comes from

Likely suppliers are janitorial contractors (facility-services contractors as suppliers), hotels (see hotel housekeeping room turnover) and other facility operators that train staff on surface-cleaning methods. Their archives are often training videos: staged, well lit and clean in sequence, but light on real soil and on mistakes. Footage from live operation is messier and more representative, and carries stronger consent and privacy questions. Recordings of cleaning staff are personal data in many jurisdictions; the EDPB guidelines on video devices frame the questions regulators ask (EDPB, 2020). This is not legal advice. Blurring faces does not by itself settle whether recording or reuse was permitted; see worker consent and identifiable people.

Questions to ask suppliers

1. Was the footage recorded for training staff or captured during live cleaning?
2. Which surfaces and soil types appear, and how many clips per combination?
3. Are cloth, cleaner and tool details recorded anywhere?
4. How are outcomes judged, and by whom?
5. Do any recordings include residents, guests or customers in frame?
6. Can the contractor license footage produced by a chemical vendor or an outside training company?
7. Is there a written cleaning procedure that maps to the video, such as a housekeeping or cleaning SOP?
8. Can additional instrumented sessions be arranged on the same surfaces?

Discuss your requirements

If you are specifying cleaning data, describe the surfaces, soil types, modalities and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. If no existing recordings fit, SourceX can scope a commissioned collection of new recordings. A request does not guarantee a match; SourceX follows up about potential sources.

For the wider picture, start at the humanoid training data guide or the workplace task demonstration hub. Related catalog entry: first-person video of skilled manual work.

Describe the workplace demonstration data your robotics project needs

Sources

• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot — Contact-rich sequences paired with force, audio and human demonstration video
• Rescaling Egocentric Vision — Kitchen egocentric video annotated through narration and temporal segmentation
• Guidelines 3/2019 on processing of personal data through video devices (Version 2.0) — Guidance on personal data processed through video devices