Short answer
Returns processing is the receive, open, identify, inspect, grade and route workflow for goods sent back by customers. Its items are unpredictable and its decisions are judgment-heavy, so useful data pairs video of the handling with item identity, condition grade and disposition. Existing returns-center footage can support workflow understanding, but it holds consumer data and no robot actions.
Why returns are a hard task for robots
Forward fulfillment handles items the operator expects, in packaging designed for the flow. Returns reverse this. A box may hold the wrong item, a missing part, a used product, a damaged unit, or something that never came from this seller. Packaging is torn, taped over or reused. The worker has to decide what the item is, whether it matches the return authorization, what condition it is in, and where it goes next.
For a humanoid, the physical steps are ordinary: open a carton, remove contents, move an item to a station, place it in a bin. The difficulty is variety and judgment at each step. That makes data coverage, not any single motion, the central buying question.
Task anatomy
| Stage | What the worker does | What can go wrong | Data that describes it |
|---|---|---|---|
| Receive and sort | Scan or identify the return label, route to a lane | Missing or unreadable label | Station video, scan records |
| Open and unpack | Cut tape, open flaps, remove padding | Damaged contents, loose parts | Close-up video of hands and packaging |
| Identify item | Match item to catalog and return authorization | Wrong item, counterfeit, missing variant | Item images, SKU and catalog entries |
| Inspect and grade | Check function, cosmetic condition, completeness | Subtle damage, inconsistent grading | Photos, grade labels, checklists |
| Clean, repack, relabel | Restore saleable condition | Missing accessories | Station video, pack instructions |
| Disposition | Restock, refurbish, liquidate, recycle, destroy | Disagreement on route | Disposition record with reason code |
The grading step has its own sourcing issues, covered in returned item condition grading. This page concentrates on the workflow around it.
What each data type contributes
Station video. Fixed or head-mounted video of unpacking and inspection can support task and step understanding, object-handling analysis and exception recognition. It does not give robot joint states or forces.
Item images and catalog data. Matching a returned item to its catalog entry is a retrieval problem. Catalog resources such as ABO, which pairs 147,702 product listings and 398,212 catalog images with metadata of up to 18 attributes per product and 3D models for 7,953 products (Collins et al., 2021), show how reference data is structured. Real returns look different from catalog photography: they are used, worn, partially packed and photographed under station lighting. See product catalog data for robot perception.
Disposition and reason records. Return reason codes, grades and routing decisions provide labels for what the worker decided. They are only as consistent as the operation's grading rules.
Exception logs. Records of unreadable labels, mismatches and escalations connect to warehouse exception handling.
Limitations of returns data for robot training
The records describe decisions; the video shows a human doing the work. Neither contains robot actions, contact forces or tool state, so physical control still needs robot-side capture. Human video of unpacking supports visual and step understanding but not a policy by itself.
Existing recordings are often quality-assurance or training footage with fixed cameras at one site. Viewpoints may miss hands inside cartons, and packaging varies by retailer in ways a single site will not cover. Grade labels are the operator's, and different processors use different scales; a model trained on one scale does not automatically carry to another. A held-out set should split by site and by grading policy.
Finally, returns sets tend to over-represent common product categories and under-represent rare, high-value or hazardous items. Ask for category distributions, not just totals.
Permissions and provenance
Returns carry consumer data: names and addresses on labels, order numbers, photos with personal items, and sometimes devices holding user content. They may also contain third-party brand packaging, and the retailer's contract with brands or marketplaces may limit reuse. Ask which party owns the images, whether workers were notified about recording, and which fields were removed. Blurring a label does not by itself resolve whether the data may be licensed. This is not legal advice. For worker recording, see worker consent and identifiable people.
Illustrative package: a returns-station recording set
# Illustrative only - values are synthetic, not an available dataset
package: returns_station_v0
source: one returns processor, 2 stations, general merchandise
video:
views: 1 overhead, 1 head-mounted, constant fps, shared timecode
clips: one per return unit, from carton open to disposition
records:
per_unit: [unit_id, category, return_reason_code, grade, disposition, handling_seconds]
catalog_link: sku_id for the expected item; mismatch_flag
images:
item_photos: front, back, defects; station lighting
redaction:
removed: shipping labels, names, order numbers, personal items
excluded: robot actions, forces, customer correspondence
rights: training and evaluation use, no redistribution, grading policy document included
Quality and acceptance criteria
Judge a returns set on whether it preserves the decisions, not only the footage. Useful checks for a delivery sample follow.
- Complete units. Each unit's clip runs from carton open to disposition, with no cuts that hide the inspection step.
- Label consistency. Two reviewers regrade a sample of units from the images, and the buyer receives the rate of disagreement with the original grade.
- Reason-code integrity. Return reasons match what is visible; for example, a "wrong item" code should correspond to a catalog mismatch in the record.
- Category balance. The share of each product category is reported, and rare categories are not silently dropped.
- Hand and package visibility. Sampled clips show hands and contents during unpacking, not only the closed box.
- Residual personal data. A scan of sampled frames and metadata finds no labels, names or order numbers.
Hold out evaluation data by site and by grading policy, and keep a note of which brands appear in which split.
Returns share steps with inbound work, so compare receiving and inbound checks; restocking decisions may depend on SKU pack instructions for how a returned item should be repacked.
Questions to ask suppliers
- Which product categories and brands appear, and in what proportions?
- What grading scale is used, and is the grading policy document included?
- Do records distinguish wrong item, missing parts, damage and customer remorse?
- Which camera positions exist at the stations, and do they show hands inside cartons?
- What consumer and order information is visible, and how was it removed?
- Who owns the images of branded goods, and are marketplace or brand terms involved?
- How were workers informed and what consent is on file?
- Can the supplier link each clip to its SKU, return reason and disposition?
- Is a new recording of selected categories possible at the supplier's site?
Discuss your requirements
If you need returns-handling video and records, describe the categories, stations and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. A request does not guarantee a match; SourceX follows up about potential sources. For more context see the warehouse data hub, the humanoid training data guide, and 3PL providers as data suppliers. The catalog entry for supply chain and logistics data covers the related records.
Describe the workplace demonstration data your robotics project needs
Sources
- ABO: Dataset and Benchmarks for Real-World 3D Object Understanding — structure and scale of catalog images, metadata and 3D models.