Short answer
Before-and-after photo records are images that field service, cleaning and retail operators capture to show a job's start and finished state, often tied to a checklist step or work order. They pair a procedure step with its visible outcome, which suits success detection and task-state verification. They show states, not the motion between them, are staged toward success, and raise privacy questions.
The buyer's problem
A robot that finishes a step needs to know whether the step worked. Closed-loop language-model planners depend on that signal. Inner Monologue, for example, plans and replans using natural-language environment feedback without additional training, with feedback sources that include success detection, scene description and human interaction (Huang et al., 2022). Success detection needs a model that can look at a scene and judge whether a described end state holds. That model needs labeled examples of finished and unfinished states in real workplaces.
Operators already generate such images. Cleaning crews photograph restrooms after service, field technicians photograph an installed unit, retail merchandisers photograph a reset shelf. Photos are proof of work for clients and supervisors. They are an existing, procedure-linked image asset, which is why this page sits in the procedures cluster. For the broader image category, see the catalog entry for images and inspection photos.
What the records look like
A typical record contains:
- One or more images, labeled before, during or after, from a phone or tablet.
- Task context: work order, route stop, checklist item or step ID, sometimes free text.
- Metadata: capture time, device, approximate or exact location, user or crew identifier.
- Status: approved, rejected for re-shoot, or flagged by a supervisor.
- Comments, which may reference defects, access problems or customer instructions.
Where they sit differs. Field-service and cleaning platforms store them against jobs; retail execution tools store them against planogram tasks or store visits; connected-worker systems attach them to steps (see connected-worker platform exports). The checklist that triggered the photo is discussed on completed checklists as execution records and checklists as success criteria.
What this data can support, and what it cannot
Completion photos can support:
- Success and state classifiers. Labeled "done" and "not done" states for judging whether a surface is clear, a shelf is faced, a panel is closed.
- Visual grounding of step outcomes. A step text such as "wipe counter and leave clear" paired with the image of the resulting state.
- Change detection. Before and after pairs of the same scene show what changed, useful for verifying an effect.
- Evaluation of verification models, held out by site and operator.
- Failure explanation inputs, where a rejected photo and its reason show what a wrong state looks like. Work on failure explanation, such as REFLECT, which summarizes multisensory robot data for an LLM and introduces the RoboFail dataset of failed executions (Liu et al., 2023), uses richer robot data than a still photo provides.
They cannot support:
- Manipulation learning. Two images do not show the hand path, grasp, force or sequence between them. They are states, not demonstrations; compare with object recognition versus manipulation data.
- Robot viewpoint. Phone photos are taken at a human's chosen angle and distance, often framed to look good.
- Balanced outcomes. Operators photograph successful work. Failed states appear only when a supervisor rejects a photo and it is retained.
- Proof of the procedure. A tidy photograph does not show every step was followed.
- Time-resolved evidence. Mid-task states are rare.
Illustrative record
# Illustrative - synthetic values
record_id: ph_0044120
job_id: job_58213
site: site_041 # pseudonymous
procedure_ref: {sop_id: sop_0092, step: "4", checklist_item: "restroom_sink_clear"}
images:
- file: img_0044120_a.jpg
role: before
captured_at: "synthetic-iso-time"
device_class: phone
resolution_px: [3024, 4032]
- file: img_0044120_b.jpg
role: after
captured_at: "synthetic-iso-time"
status: approved # approved | rejected | resubmitted
rejection_reason: null
location: {precision: site_level} # exact coordinates removed
people_visible: false
operator_id: operator_A12 # pseudonymous
removed: [customer_name, address, faces, signage_text]
Evaluation points
- Pairing integrity. Do before and after images show the same scene, from a similar angle? Many pairs do not.
- Link to the step. Is each photo tied to a specific step or checklist item, or only to a job?
- Outcome labels. Is there a status, and how reliable is it? Photos approved by a supervisor are more trustworthy than self-submitted ones.
- Negative examples. How many rejected or resubmitted photos are retained? Without them, a classifier sees only success.
- Metadata fidelity. Timestamps from the device clock can be wrong; stripped EXIF data may remove the time. Know what survived.
- Resolution and compression. Platform uploads often compress images; check what the supplier holds.
- Diversity. Count sites, operators, lighting and object types.
- Duplicates. Re-photographed scenes and routine repeats at the same location can inflate volume.
Sampling should be your own: pull records at random across sites, then review pairs by hand before agreeing volume.
Permissions and provenance
Completion photos are taken inside other people's premises. They can capture customers, employees, children, license plates, mail, screens, documents and security details, plus the layout of the property. The operator may not own the right to use them: customer contracts may restrict images, and workers may not have been told the photos could be licensed. Location metadata can reveal a home address.
Ask what notices and consents exist for customers and staff, whether contracts with clients restrict use, what was removed from images and metadata, and who the data controller is. Blurring faces is a partial step, and it does not settle consent or contract questions. This is not legal advice. See worker consent and identifiable people for the related framework on identifiable people in workplace recordings.
Limitations and additional preparation
- Curated toward success. Class balance is skewed; expect to need negative examples from another source or by design.
- Uncontrolled capture. Angle, lighting and framing vary, which helps diversity and hurts consistency.
- Weak labels. The only label may be "uploaded"; judgment of whether the end state truly met the standard needs reviewers.
- Redaction effects. Blurring or cropping to remove people and addresses may remove the evidence the image was meant to show.
- Linking to procedures. Matching photos to steps may need parsing of checklists and work orders; see housekeeping and cleaning SOPs and retail store operating procedures.
- Commissioned alternative. If you need controlled before-and-after captures with designed viewpoints and failures, that is a new collection; discuss feasibility rather than assuming it.
- Defect imagery has its own sourcing issues, covered in the inspection hub.
Questions to ask suppliers
- How many photo pairs are tied to a step or checklist item, not just a job?
- Who approved or rejected photos, and are rejected ones retained?
- What metadata survives, and how was location handled?
- Do images contain people, plates, documents or screens, and how were they treated?
- What do customer contracts say about image use?
- What is the original resolution, and has the platform compressed it?
- How many sites and operators are represented?
- Were workers informed that photos could be licensed?
- Can I draw a random sample for review before agreeing terms?
Turning photos into evaluation items
Completion photos are most useful as a verification benchmark when each item is a question with a checkable answer: given the step text and the after image, does the state meet the criterion? Build the answer key from supervisor approvals and rejections, then have a second reviewer independently judge a sample to measure how consistent the standard is. Expect disagreement on borderline cases such as streaks, partial facing or a slightly open panel; that disagreement is information about how loosely the criterion is written. Split by site and operator so a classifier cannot learn one crew's photo habits, and hold out a separate set that was never used for tuning thresholds. The result evaluates visual verification, which is one input to success detection, not the physical task itself.
Discuss your requirements
If you are looking for end-state images paired with procedure steps, describe the tasks, sites and permitted uses to SourceX. SourceX looks for US businesses that hold suitable images, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match, and SourceX follows up about potential sources. For the wider map, see the humanoid training data guide and the procedures hub. The catalog entry on field service maintenance records describes related records.
Tell SourceX what your evaluation set needs to cover.
Sources
- Inner Monologue: Embodied Reasoning through Planning with Language Models — closed-loop feedback including success detection.
- REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction — failure explanation and the RoboFail dataset.