Short answer
Record in a staged workcell when you need controlled viewpoints, fast resets, scripted variations and deliberate failures; record during live operations when you need real pace, clutter, interruptions and the variety of actual sites. Staged capture is easier to calibrate, consent and annotate but can look unlike deployment. Live capture is realistic but harder to schedule, cover and clear. Many programs pilot staged, then add live sessions.
The choice inside a commissioned collection
Once you have decided to commission recording rather than license what a business already holds (the trade-off is covered in existing workplace footage vs commissioned recording), the next decision is where the cameras go. In practice there are three settings, not two:
- Staged workcell. A replica station in a training room, spare bay or rented space, stocked with the real objects, tools and fixtures.
- Off-shift at the real site. The actual station, recorded before or after production with scripted tasks and no customers.
- Live operations. Real work recorded during a production shift, as it happens.
Each setting changes what the footage can teach, what it costs and who has to agree to it.
How the three settings compare
| Factor | Staged workcell | Real site, off-shift | Live operations |
|---|---|---|---|
| Camera and sensor placement | Free; rig can stay fixed for weeks | Set up and removed each session | Limited by safety, traffic and station design |
| Resets and repetitions | As many as needed | Several per session | One per real job; no repeats |
| Scene realism | Real objects, simplified surroundings | Real station, quiet conditions | Full clutter, noise, lighting and interruptions |
| Working pace | Tends to slow down | Close to normal | Production pace, with motion blur |
| Failures and recoveries | Can be scripted on request | Some can be scripted | Only what occurs naturally |
| People in frame | Demonstrators only | Demonstrators, perhaps staff | Co-workers, supervisors, sometimes customers |
Disruption to the host business rises from left to right. A staged cell costs space and setup; off-shift sessions cost overtime and supervised access; live recording must never slow production or interfere with guards, walkways or emergency stops.
What published collections show
Robot datasets that left the lab treated scene variety as a design goal and paid for it with standardization. DROID collected 564 scenes and 86 tasks with 50 data collectors across North America, Asia and Europe over 12 months, replicating the same rig at 13 institutions (Khazatsky et al., 2024). BridgeData V2 spans 24 environments (Walke et al., 2023). The Universal Manipulation Interface took collection into ordinary environments with a hand-held gripper whose only sensor is a wrist-mounted fisheye camera, and reports about 70% success in out-of-distribution tests (Chi et al., 2024); real settings were reached without bringing a robot into them.
Human-video datasets show the scripted-versus-natural trade-off directly. CaptainCook4D recorded recipes in real kitchens, including intentionally erroneous executions, under IRB-approved written consent (Peddi et al., 2023): scripted errors inside real environments. Assembly101 recorded unscripted assembly and disassembly that contains natural mistakes (Sener et al., 2022). IndustReal includes procedural and execution errors, many of which appear only in its validation and test splits (Schoonbeek et al., 2023). Ego4D captured unscripted daily activity from 931 camera wearers in 74 locations (Grauman et al., 2021). These are research designs, not evidence of what any workplace will allow.
What each setting misses
Staged recording misses production pace, incidental clutter, interruptions and multitasking, site-specific fixtures, real lighting and the shortcuts people take under time pressure. A model trained mostly in one replica can learn its quirks: the same bench, the same light, the same tote. Demonstrators also behave differently when they know every move is being studied.
Live recording misses control. You cannot ask for another angle, a repeat or a deliberate drop. Cameras get blocked, jobs get interrupted halfway, and pressure leads to skipped steps and off-task stretches. Calibration drifts when someone bumps a mount. Bystanders and customers enter the frame, which widens consent and redaction work.
Choosing by how the data will be used
| Use | Lean toward | Why |
|---|---|---|
| Hand-pose or co-training data from calibrated multi-view rigs | Staged or off-shift | Rigs need fixed positions and calibration that live stations rarely permit |
| Video pretraining and world models | Live | Breadth of real appearance matters more than control |
| Failure and recovery examples | Staged, scripted | Natural failures are rare and unpredictable |
| Step recognition at real speed | Live, or off-shift at normal pace | Pace changes motion, blur and visibility |
| Deployment-readiness evaluation | Live, at held-out sites | Evaluation should match deployment conditions |
| First pilot of a new rig | Staged or off-shift | The cheapest place to debug capture and labels |
Two sibling pages go deeper on specific rows. Scripted failures are covered in failed attempts and recoveries in demonstration data, and the effect of speed is covered in production-speed vs slowed-down demonstrations.
Hybrid designs
- Staged first, live second. Debug the rig, labels and consent in a replica, then record the same tasks live to measure the realism gap.
- Measured replicas. Build the staged cell from site measurements and the site's own fixtures so geometry matches deployment.
- Live with scripted inserts. During low-demand periods, demonstrators perform planned variations, such as an unusual object or a recovery, on the real station.
- Off-shift rotation across sites. Visit several real stations outside production hours to gain site variety without live-shift exposure.
Whichever design you choose, start small; scoping a pilot collection explains what a first run should prove.
Illustrative session plan
# Illustrative only — synthetic sites and quotas, not an available program
program: tote_kitting_v0
sessions:
- setting: staged_workcell
site: site_017_training_bay
purpose: rig validation, scripted failures (dropped item, wrong slot)
demonstrators: [operator_A12, operator_B07]
- setting: off_shift
site: site_017_line_2
purpose: real station geometry at normal pace
demonstrators: [operator_A12, operator_C33]
- setting: live
site: site_024_line_1
purpose: production pace, interruptions; held out for evaluation
demonstrators: rostered staff who opted in
excluded_from_frame: customer areas, screens, badge readers
labels: task, step, outcome; setting recorded as metadata on every clip
Permissions and site operations
Staged sessions involve recruited demonstrators who can consent to the specific purpose. Live sessions also capture co-workers, supervisors, visitors and sometimes customers, plus processes and screens the business may treat as confidential. Treat these as diligence questions for the host business and your own counsel: who must be notified, who can opt out, which areas are off limits, and whether a works council or union agreement applies. Worker consent and identifiable people in workplace recordings covers the permissions side, and recording protocols for live work sites covers day-to-day conduct on the floor. This is not legal advice.
Limitations
- No setting adds robot actions, joint states or forces; those require a robot, teleoperation, a handheld capture device or instrumented tools.
- Staged realism is bounded by the replica; live realism is bounded by the one or two sites willing to host.
- Live footage generally needs more redaction and more rejected clips, so usable minutes per recorded hour fall.
- Setting is a confound: if every failure was staged and every success was live, a model can learn the setting instead of the failure. Record the setting on every clip and balance it where you can.
Questions to ask collection providers and host sites
- Can the station be replicated off the floor with the same fixtures, tools and object set?
- Which hours are available for off-shift recording, and who must supervise?
- During live recording, where can cameras go without blocking walkways, guards or emergency stops?
- Who besides the demonstrators will appear in frame, and how will they be notified or kept out?
- Which failures and recoveries can be scripted safely, and which must never be staged?
- How many complete task instances does a live shift usually produce, and how often are tasks interrupted?
- Can production records, such as work orders or scan events, be linked to live clips to confirm outcomes?
- Which areas, screens or products are off limits?
Discuss your requirements
If you know the tasks but not yet the setting, tell SourceX what you need recorded and under which conditions. SourceX reviews the requirement and follows up about potential sources; the site also describes a separate physical-world path for new recordings of hands-on work, coordinated with businesses, where each opportunity has its own requirements. A request does not guarantee a match. For the full set of capture and annotation topics, see the workplace task demonstration hub and the humanoid training data guide.
Describe the workplace demonstration data your robotics project needs
Sources
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset — scene diversity at scale with a standardized rig replicated across institutions.
- BridgeData V2: A Dataset for Robot Learning at Scale — robot demonstrations across many environments.
- Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots — in-the-wild collection with a hand-held gripper and out-of-distribution results.
- CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities — scripted errors recorded in real kitchens with written consent.
- Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities — unscripted assembly with natural mistakes.
- IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting — procedural and execution errors concentrated in evaluation splits.
- Ego4D: Around the World in 3,000 Hours of Egocentric Video — unscripted daily-life recording across many wearers and locations.