Privacy and preparation
Physical AI dataset supplier checklist: what a video fact sheet discloses
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
A workplace video fact sheet should disclose who owns the footage and what workers were told and agreed to, then define the physical task from start to end, the camera setup and clocks, the operating variation captured, and how interventions, failures and outcomes can be found. It ends with known gaps. Fill it from documents, and let weak rights answers narrow the offer before footage leaves the company.
Video and sensors
Physical AI dataset supplier
Key takeaways
- Write the rights and collection conditions block first; most workplace footage fails review there, not on image quality.
- Record access, privacy permission, ownership and licensing rights as separate facts, because one never implies another.
- Define each task by a logged start and end event so footage can be cut into examples without guessing.
- State which clock stamps each stream and whether calibration exists; both are cheap to record and expensive to reconstruct.
- Count variation and recovery attempts, and take outcomes from systems of record rather than from someone watching the video.
- When notice named security only and no releases exist, narrow the offer or stop and ask counsel.
What should a workplace video dataset fact sheet disclose?
A workplace video fact sheet should disclose two things before anything else: what the people and the company actually permitted when the footage was captured, and what physical task the footage shows from start to finish. Everything else, from camera specs to outcome codes, only matters once those two answers hold up.
In practice the sheet has seven blocks: footage origin and ownership, people and permissions, task boundaries, capture setup, timing and synchronization, operating variation, and interventions, failures and outcomes. A final block lists known gaps in plain words. Each block is short, but each one answers a question that a physical AI team, a privacy reviewer or your own counsel will ask anyway.
Keep four ideas apart while you fill it in. Having access to the camera system is not the same as having privacy permission to reuse the recordings, owning the recordings is not the same as having a contractual right to license them, and blurring faces does not by itself create a right to license anything. The sheet records each of these separately so nobody has to infer one from another.
Why build the fact sheet before offering footage, not after a buyer asks?
Build it first because most workplace video fails review on questions the company could have answered in an afternoon: why the cameras were installed, who was told, and whether the footage shows a whole task or only fragments. Writing the sheet early surfaces those problems while you can still narrow the offer instead of withdrawing it.
The questions are not new. The Datasheets for Datasets template asks dataset creators whether the people in the data were notified about the collection, whether they consented, how consent was requested and whether it could be revoked. The Croissant Responsible AI vocabulary from MLCommons gives the same topics named fields, including a description of the collection process, the collection type, the collection timeframe and missing data. Google's Data Cards Playbook describes data cards as structured summaries of the facts stakeholders need across a dataset's lifecycle, including upstream sources and collection methods.
Those frameworks were written for published research datasets. A company holding years of warehouse, factory or field-service footage needs a narrower version that a plant manager can complete without a data science team, and that adds the physical details robotics and world-model teams care about: task boundaries, camera geometry, clocks and what happened at the end of each attempt. That is the sheet below.
Rights and collection conditions: which permissions does the footage really carry?
Fill these rows from documents, not memory. If a row cannot be answered from a written policy, notice, contract or log, write "unknown" and treat that row as an open issue rather than a yes.
Two legal regimes show why the rows exist. Illinois's Biometric Information Privacy Act lists a scan of face geometry as a biometric identifier and requires a private entity to inform the person in writing and obtain a written release before collecting one; the Act also excludes photographs from that definition, so the question is what processing will be run on the footage, not only what it shows. In the EU, the European Data Protection Board's video guidelines note that an employee at work usually does not expect to be monitored by the employer and that employers should generally not rely on consent, because consent in employment is unlikely to be freely given.
Neither regime is the whole picture, and state, national and works-council rules vary. The table records facts a reviewer needs; it does not decide whether a use is lawful.
Large research collections show the pattern a careful supplier can follow. The Ego4D project, for example, describes collecting explicit informed consent for identity capture in some indoor recordings and de-identifying the rest with redaction software and human review. The point for a business is not to copy that program but to know, row by row, which of your recordings sit in which group.
| Field | What to write | Illustrative entry | Red flag |
|---|---|---|---|
| Footage owner | The legal entity that operates the cameras and stores the recordings | Operating subsidiary that leases the building; recordings on its own video management system | Cameras or storage run by a landlord, client or security contractor |
| Purpose at capture | Why the cameras were installed, quoted from the policy or signage | Safety and incident review at packing stations, per the 2023 site camera policy | Purpose limited to theft prevention or security with no wider use stated |
| Worker notice | How workers were told, with the date and the exact wording | Handbook section and posted signs at each entrance; wording attached | No written notice, or notice that names only security use |
| Releases and consent records | Which people signed what, and whether any can withdraw | Volunteer operators signed recording releases for a 2025 pilot line only | Consent claimed but no signed records can be produced |
| Biometric processing | Whether anyone ran or plans face, gait or voice identification on the footage | None run; buyer must not run identification | Face matching used for timekeeping or access on the same cameras |
| Other people in frame | Visitors, drivers, contractors or customers who appear | Carrier drivers visible at dock doors 4 to 6 | Customer premises or public areas visible without any notice |
| Third-party contracts | Customer, client or site agreements that restrict recordings | Two retail clients prohibit sharing images of their branded packaging | Footage recorded on a client's site under that client's confidentiality terms |
| Labor agreements | Collective bargaining or works council terms on monitoring | None at this site; reviewed by HR | An agreement limits camera use and nobody has checked it |
| Retention | The written retention period and whether old footage was kept outside it | Recordings older than the retention period were exported for a safety study | Footage that should have been deleted under the company's own schedule |
| Excluded areas | Spaces never recorded or removed from the offer | Break room, restrooms and the first-aid room excluded | Any camera angle covering break, changing or medical areas |
Task and capture fields: does the footage show a task a model can learn from?
Footage is useful for physical AI when each clip can be tied to one bounded task with a known start, a known end and a known camera. If you cannot say where a task begins and ends, a buyer cannot cut the footage into training examples without guessing.
Camera geometry matters more than resolution for most robot and world-model work. The ROS CameraInfo message, which many robotics stacks use, carries calibration matrices and says an uncalibrated camera should leave them zeroed, so a missing calibration is visible rather than hidden. It also defines the header timestamp as the time the image was acquired, not the time it was saved.
Clocks are where workplace footage most often breaks. A video recorder, a barcode scanner and a machine controller each keep their own time, and a few seconds of drift can attach the wrong scan to the wrong motion. Record which clock each stream uses and how, if at all, they were aligned.
| Field | What to write | Illustrative entry | Why a reviewer asks |
|---|---|---|---|
| Task definition | The task in one sentence, with its start and end events | Pack one customer order: starts at tote scan, ends at carton label print | Defines where one training example begins and stops |
| Steps | The ordered steps a trained worker follows | Pick items, scan each, select carton, fill void, seal, label | Lets annotators mark step boundaries and skipped steps |
| Camera type and mount | Fixed, head-worn, body-worn or vehicle-mounted, and the angle | Fixed overhead camera above each of 12 stations | View angle decides which hand and object details are visible |
| Calibration | Whether intrinsics were measured and where they are stored | Not calibrated; camera model and lens recorded | Depth and 3D work need calibration or a way to estimate it |
| Frame rate and format | Recorded frame rate, resolution and codec, and whether variable | Variable frame rate from the recorder; nominal 15 frames per second | Variable frame rate shifts labels if timestamps are not kept |
| Clock source | Which clock stamps video and each other stream | Recorder synced to network time; scanners use their own clocks | Unaligned clocks attach events to the wrong moment |
| Linked event streams | Other systems that record the same task | Warehouse management scans, carton weight checks, label print log | Event logs supply outcomes without manual labeling |
| Episode boundaries | How the start and end of each attempt can be found | Derived from tote scan and label print events | Robot learning formats store data as episodes of steps |
What operating variation and recovery behavior is missing from the footage?
Most workplace footage over-represents the normal case: the same shift, the same stations, the same product mix, and the task going right. Count the variation you have, and write down what is absent, before a buyer finds the gap in a sample.
Use counts, not adjectives. "Varied lighting" tells a reviewer nothing; "night shift on 3 of 12 stations, overhead lights only" tells them what the model will and will not see. The grid below is a starting point; add the dimensions that change how your task is performed.
Recovery behavior deserves its own line. A model that only sees clean completions learns little about what to do after a dropped item, a jammed sealer or a mis-scan. If your footage includes those moments, say how many and how you can find them. If it does not, say so; that is a scope fact, not a weakness to hide.
| Dimension | What to count | Illustrative count | Gap signal |
|---|---|---|---|
| Shifts and hours | Hours of footage per shift | Day 70%, evening 25%, night 5% (illustrative) | One shift dominates the record |
| Operators | Distinct operators per station, counted not named | About 40 operators across 12 stations (illustrative) | A handful of people perform most attempts |
| Product or job mix | Attempts per product family or job type | 6 carton sizes; 2 rarely used (illustrative) | Rare variants have too few attempts to learn from |
| Environment | Lighting, season, temperature or clutter conditions | Seasonal peak with overflow totes in aisles | All footage from one calm period |
| Equipment state | Attempts with degraded or swapped equipment | Sealer replaced in week 30 (illustrative) | No record of equipment changes |
| Failures and recoveries | Attempts with an error and the recovery that followed | Mis-scans flagged in the warehouse system | Failures exist but cannot be located in the video |
How should interventions, failures and outcomes be recorded?
Record the outcome of every attempt from a system of record rather than from someone watching the video. A quality hold, a returned order, a reworked part or a closed maintenance work order is stronger evidence of what happened than a reviewer's impression.
At a glance
How should interventions, failures and outcomes be recorded
Robot learning formats already distinguish these cases. Google's RLDS format stores an episode as a sequence of steps, marks the first and last step, and separates a terminal step, where the episode genuinely ended, from a truncated episode that simply stopped being recorded. Your fact sheet should make the same distinction for workplace attempts, because a clip that cuts off before the label prints is not a failed pack.
- Completed as intended: the end event fired and no downstream system flagged the result.
- Completed with intervention: a supervisor, a second worker or a remote operator stepped in; note who intervened by role, never by name.
- Recovered error: something went wrong and the same worker corrected it within the attempt, such as a rescan or repack.
- Failed: the attempt ended without the intended result, linked to the record that shows it, such as a quality hold or a rework ticket.
- Truncated: the recording stopped before the end event, because of a camera gap, retention cut or export limit.
- Unknown: no system of record links to this attempt; keep these out of any outcome statistics.
Worked example: a fact sheet for a distribution center's packing stations
This example is illustrative; the company, figures and findings are invented to show how the sheet works. A regional distribution center with 12 packing stations has fixed overhead cameras installed for safety review and roughly two years of retained recordings. The operations lead completes the sheet over three working days with HR and the IT administrator who runs the video system.
The rights block finds a written camera policy that names safety and incident review, posted signage, and a handbook section, but no releases except for a 2025 pilot line where 8 volunteers signed recording releases. Two retail clients' agreements prohibit sharing images of their branded packaging, and one camera angle covers part of the break area. The sheet records all three facts and removes that angle from the offer.
The task block defines a pack as tote scan to label print, which the warehouse system logs for every order. Clocks differ: the recorder uses network time and the scanners drift by up to a few seconds, so the sheet notes that alignment must be rechecked on a sample. Outcome data comes from the carton weight check and returns log, giving completed, recovered and failed attempts that can be found by timestamp.
The exception changes the offer. Because only the pilot line has signed releases and the other stations rely on a safety-only notice, the company narrows its first offer to the pilot line's footage plus de-identified event logs from all stations, and asks counsel whether the remaining footage can be offered at all. The sheet states that narrower scope rather than claiming all 12 stations.
Which answers should stop the offer or shrink its scope?
Some answers on the sheet are reasons to stop, and others are reasons to offer less. Decide which before anyone outside the company sees the footage.
At a glance
Which answers should stop the offer or shrink its scope
- Stop and ask counsel when the cameras belong to a client, landlord or contractor, or when a client contract restricts recordings made on its premises.
- Stop when any angle covers restrooms, changing areas, medical rooms or similar spaces; remove the footage rather than masking it.
- Narrow the scope when notice named security or safety only and there are no releases: offer event logs, footage from consented lines, or angles where workers are not identifiable, subject to counsel's review.
- Narrow the scope when footage was kept past the company's own retention schedule; that footage may need to be deleted, not licensed.
- Bring in a privacy reviewer when any biometric processing has been run on the same cameras, when workers are in Illinois or another state with biometric rules, or when any footage was captured in the EU.
- Bring in a robotics or physical AI reviewer when the task definition, clocks or outcome links are uncertain; they can say whether the footage is still useful without them.
Taking the fact sheet into a SourceX assessment
For a company that decides to license workplace video, SourceX runs the rights review, contracting, preparation and delivery around the selected recordings. Qualification starts with a brief look at the company itself, with nothing uploaded, so keep the fact sheet ready instead of sending footage with it.
If the footage goes ahead, nothing is handed over until the company has approved the scope and terms and signed, and video archives that large stay in the company's storage or move on encrypted drives, since SourceX does not host them. A completed sheet makes each of those steps faster because the rights questions have written answers.
Frequently asked questions
Do workers have to consent before workplace video is licensed for AI?
It depends on the jurisdiction, the notice workers received and what processing will be run. Some rules, such as Illinois's biometric law, require a written release before biometric identifiers are collected, and EU guidance says employers should generally not rely on consent at all. Record what notice and releases exist and have counsel review the specific facts.
Is a workplace video fact sheet the same as a datasheet or data card?
It covers the same ground but is narrower and more physical. Datasheets and data cards were designed for published research datasets; this sheet adds task boundaries, camera geometry, clocks and outcome links that physical AI teams need, and it can be completed by an operations lead.
Should faces be blurred before the fact sheet is written?
No; the sheet describes the footage and its permissions, and redaction is a later preparation step agreed in scope. Blurring also does not settle the rights question, because de-identification alone does not create a right to license the recordings.
Can security camera footage be offered as physical AI training data?
Sometimes, but security footage often fails on purpose and notice: the policy names security only, retention is short, and angles rarely show a whole task. Record the stated purpose exactly and expect to narrow the offer or stop if the notice does not reach other uses.
Do I need to send sample footage to see whether my company qualifies?
No. Whether a company qualifies is checked first through a short assessment of the business, and no footage or other data is uploaded at that stage. Keep the fact sheet ready instead, since its answers decide whether any footage moves forward.
Sources
- The Datasheets for Datasets template asks whether the people in a dataset were notified, whether they consented, how consent was requested and whether it could be revoked. Source
- The Croissant Responsible AI vocabulary defines fields for the collection process, collection type, collection timeframe and missing data. Source
- Google's Data Cards Playbook describes data cards as structured summaries of essential facts about a dataset, including upstream sources and collection methods. Source
- The ROS CameraInfo message says an uncalibrated camera should leave its calibration matrices zeroed and that the header timestamp should be the image acquisition time. Source
- RLDS stores episodes as sequences of steps, marks first and last steps, and distinguishes terminal steps from truncated episodes. Source
- Illinois's Biometric Information Privacy Act lists a scan of face geometry as a biometric identifier and excludes photographs from that definition. Source
- Section 15 of the Act requires a private entity to inform the person in writing and receive a written release before collecting a biometric identifier. Source
- EDPB guidance says an employee at work is usually not expecting to be monitored by the employer and that employers should generally not rely on consent for video processing. Source
- The Ego4D paper describes collecting explicit informed consent for identity capture in some indoor footage and de-identifying the rest with redaction software and human review. Source
See if your company qualifies
A short company assessment. No data uploads are needed.