Short answer
Curation turns continuous workplace recordings into clips worth training on: removing idle and empty footage, deduplicating near-identical segments, cutting task instances, filtering by visibility and quality, and documenting every rule. It reduces cost and noise but can also remove the rare events and variation a model needs, so specify what is dropped, keep an audit sample of rejects, and report coverage before and after.
The buyer's problem
A business with fixed cameras can produce thousands of hours a week. Most of it shows empty aisles, a worker standing at a screen, or the same pallet wrapped the same way. Paying to transfer, store and annotate all of it is wasteful; training on all of it teaches a model that nothing happens most of the time.
Curation sits between the archive and the labeling step. Done well it concentrates effort on the footage that shows work. Done badly it silently deletes the difficult cases. The decisions are yours to specify, even if the supplier or a preparation vendor performs them. For the choice between whole recordings and trimmed clips, see continuous shift recordings vs curated clips.
What curation sequences look like in published work
Ego4D shows curation and annotation at scale: 3,670 hours of daily-life video from 931 camera wearers across 74 locations in 9 countries, with consenting participants, de-identification procedures, more than 250,000 hours of annotator effort and five benchmark tasks (Grauman et al., 2021). Note the ratio. The annotator effort exceeds the footage hours many times over, which is a reminder that every retained hour carries labeling cost.
Web-scale corpora take the opposite approach. HowTo100M gathered 136 million clips from 1.22 million narrated instructional videos, with captions from automatic speech recognition and no manual annotation (Miech et al., 2019); follow-up work reports that narration is frequently not visually aligned with the clip. Scale there replaces curation, and quality problems are accepted. A workplace buyer rarely has that option, because the footage is smaller, the tasks narrower and the labels more precise.
Curated video is also an input to generative pipelines. DreamGen adapts image-to-video world models to a robot, generates synthetic "neural trajectories" and recovers pseudo-actions with a latent-action or inverse-dynamics model (Jang et al., 2025). That is a downstream use, not a substitute for the robot data the adaptation relies on.
Curation stages
| Stage | What it does | Typical method | Main risk |
|---|---|---|---|
| Inventory | Counts files, durations, cameras, gaps, corrupt files | Metadata scan | Underestimating missing time ranges |
| Empty/idle removal | Drops frames with no person or no task activity | Motion, person detection, schedule data | Removing slow, careful work |
| Deduplication | Drops repeated or overlapping footage | Hashes, embeddings | Collapsing legitimately repeated tasks |
| Task segmentation | Cuts continuous video into task instances | Step or event detection, system logs | Wrong boundaries |
| Quality filtering | Removes blur, glare, lens obstruction, hands out of view | Heuristics plus review | Bias toward easy conditions |
| Sensitive-content screening | Flags screens, documents, faces, bystanders | Detectors plus human review | Missed items; over-blurring |
| Balancing | Caps repeated cases, reserves rare ones | Sampling rules | Unrepresentative mix |
What curation can and cannot do
Curation improves signal-to-noise and cuts storage and labeling cost. It cannot add what was never recorded: a viewpoint nobody captured, hands that were never in frame, a failure that was never filmed. It also cannot resolve rights. Footage removed from the delivery still exists at the supplier, and footage that stays needs clear permission regardless of how tidy it is.
Rules that need explicit decisions
Idle definition. "Idle" for a packing task may mean no hands in the work zone for more than a set duration. A technician waiting for a part to cure is idle by that rule, yet waiting is part of the task. Define per task, and keep a sample.
Deduplication scope. Remove exact copies and overlapping exports freely. Near-duplicates need care: two workers repeating a task in the same cell are different demonstrations. Deduplicate by video content and time, not by task label.
Boundary policy. Decide whether clips begin at the first reach or include a lead-in and lead-out of a few seconds, so later models see the transition between tasks.
Rare-event protection. Mark falls, drops, interruptions and unusual objects as protected classes that balancing cannot remove. See rare events in demonstrations.
Redaction ordering. Redact before or after curation by agreement, but record the order; redaction can change detector behavior. See redaction in demonstration video.
Illustrative curation report
# Illustrative only - figures are synthetic placeholders, not a real archive
archive: site_017_cam_set_A
received: {files: 1,000, hours: 400, cameras: 6, date_range: 2026-01..2026-06}
stages:
- name: corrupt_or_unreadable
removed_hours: 8
- name: no_person_present
removed_hours: 192
- name: person_present_no_task_activity
removed_hours: 61
- name: exact_and_overlap_duplicates
removed_hours: 14
- name: quality_fail # blur, obstruction, glare
removed_hours: 22
- name: sensitive_content_fail
removed_hours: 6
retained:
hours: 97
clips: 3,840
median_clip_s: 91
audit:
rejected_sample_reviewed: 2% per stage by buyer
protected_rare_events_kept: 100% of 112 flagged
coverage_after: clips per task, per camera, per shift, per operator pseudonym
All figures above are invented for illustration. The structure matters: counts by stage, a reject audit and a coverage table before and after.
Acceptance criteria
- The curation report lists every rule and the hours or clips each removed.
- A random sample of rejected footage is available to you for review.
- Retained clips link back to source file and time range so decisions can be traced; see demonstration clip metadata schema.
- Coverage by task, camera, shift and site is reported before and after curation.
- Protected rare events survive curation.
- Clips are cut at the agreed boundaries and play without decode errors.
Curation for different downstream uses
The right curation depends on what the clips are for. Pretraining on diverse task video favors aggressive removal of idle and duplicate footage and tolerance for imperfect labels. Imitation-style training on a narrow task favors strict boundaries, quality filters and success labels. Evaluation sets favor little filtering, so that hard and ambiguous clips remain, plus a record of what was excluded. Preparing a single curated set for all three typically serves none of them well. Ask for the curation rules to be recorded per use, and keep the uncurated archive reference so a later purchase can revisit what was dropped. Curated clips used as input to world-model or video-generation pipelines still need robot data to ground actions, as the cited pipelines do.
Questions to ask suppliers
- Who performs curation, with what tools, and which steps are manual?
- How is idle defined for each task?
- Is the original archive retained, and can I request additional footage later?
- How are duplicates detected, and are repeated demonstrations preserved?
- Can you provide the rejected-footage sample and the removal counts?
- Which time ranges are missing because of outages or overwritten storage?
- How were sensitive items screened and by whom?
- Do curation rules favor easy lighting or camera conditions?
- Are rejected clips deleted, and does that match the permissions granted?
Discuss your requirements
If you hold or want to source raw workplace recordings, describe the tasks, volume and curation rules you need to SourceX. SourceX reviews the requirement and follows up about potential sources; preparation work is scoped case by case, and a request does not guarantee a match. See the humanoid training data guide and the workplace task demonstration hub for the wider picture, and existing footage vs commissioned recording for the sourcing decision that precedes curation. For labeling next steps, see step segmentation annotation.
Discuss your humanoid training-data requirements with SourceX
Sources
- Ego4D: Around the World in 3,000 Hours of Egocentric Video — scale, consent, de-identification and annotator effort.
- HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips — web-scale instructional video with ASR captions.
- DreamGen: Unlocking Generalization in Robot Learning through Video World Models — world-model adaptation and pseudo-action recovery.