Skip to content

Privacy and preparation

Physical AI dataset supplier checklist: what a video fact sheet discloses

By SourceX Editorial · Reviewed by Noah Loul ·

9 sections

Short answer

A workplace video fact sheet should disclose who owns the footage and what workers were told and agreed to, then define the physical task from start to end, the camera setup and clocks, the operating variation captured, and how interventions, failures and outcomes can be found. It ends with known gaps. Fill it from documents, and let weak rights answers narrow the offer before footage leaves the company.

Video and sensors

Physical AI dataset supplier

This page covers physical AI dataset supplier checklist: what a video fact sheet discloses.Illustrative

Key takeaways

  • Write the rights and collection conditions block first; most workplace footage fails review there, not on image quality.
  • Record access, privacy permission, ownership and licensing rights as separate facts, because one never implies another.
  • Define each task by a logged start and end event so footage can be cut into examples without guessing.
  • State which clock stamps each stream and whether calibration exists; both are cheap to record and expensive to reconstruct.
  • Count variation and recovery attempts, and take outcomes from systems of record rather than from someone watching the video.
  • When notice named security only and no releases exist, narrow the offer or stop and ask counsel.

What should a workplace video dataset fact sheet disclose?

A workplace video fact sheet should disclose two things before anything else: what the people and the company actually permitted when the footage was captured, and what physical task the footage shows from start to finish. Everything else, from camera specs to outcome codes, only matters once those two answers hold up.

In practice the sheet has seven blocks: footage origin and ownership, people and permissions, task boundaries, capture setup, timing and synchronization, operating variation, and interventions, failures and outcomes. A final block lists known gaps in plain words. Each block is short, but each one answers a question that a physical AI team, a privacy reviewer or your own counsel will ask anyway.

Keep four ideas apart while you fill it in. Having access to the camera system is not the same as having privacy permission to reuse the recordings, owning the recordings is not the same as having a contractual right to license them, and blurring faces does not by itself create a right to license anything. The sheet records each of these separately so nobody has to infer one from another.

Why build the fact sheet before offering footage, not after a buyer asks?

Build it first because most workplace video fails review on questions the company could have answered in an afternoon: why the cameras were installed, who was told, and whether the footage shows a whole task or only fragments. Writing the sheet early surfaces those problems while you can still narrow the offer instead of withdrawing it.

The questions are not new. The Datasheets for Datasets template asks dataset creators whether the people in the data were notified about the collection, whether they consented, how consent was requested and whether it could be revoked. The Croissant Responsible AI vocabulary from MLCommons gives the same topics named fields, including a description of the collection process, the collection type, the collection timeframe and missing data. Google's Data Cards Playbook describes data cards as structured summaries of the facts stakeholders need across a dataset's lifecycle, including upstream sources and collection methods.

Those frameworks were written for published research datasets. A company holding years of warehouse, factory or field-service footage needs a narrower version that a plant manager can complete without a data science team, and that adds the physical details robotics and world-model teams care about: task boundaries, camera geometry, clocks and what happened at the end of each attempt. That is the sheet below.

Rights and collection conditions: which permissions does the footage really carry?

Fill these rows from documents, not memory. If a row cannot be answered from a written policy, notice, contract or log, write "unknown" and treat that row as an open issue rather than a yes.

Two legal regimes show why the rows exist. Illinois's Biometric Information Privacy Act lists a scan of face geometry as a biometric identifier and requires a private entity to inform the person in writing and obtain a written release before collecting one; the Act also excludes photographs from that definition, so the question is what processing will be run on the footage, not only what it shows. In the EU, the European Data Protection Board's video guidelines note that an employee at work usually does not expect to be monitored by the employer and that employers should generally not rely on consent, because consent in employment is unlikely to be freely given.

Neither regime is the whole picture, and state, national and works-council rules vary. The table records facts a reviewer needs; it does not decide whether a use is lawful.

Large research collections show the pattern a careful supplier can follow. The Ego4D project, for example, describes collecting explicit informed consent for identity capture in some indoor recordings and de-identifying the rest with redaction software and human review. The point for a business is not to copy that program but to know, row by row, which of your recordings sit in which group.

Rights and collection conditions: which permissions does the footage really carry?
FieldWhat to writeIllustrative entryRed flag
Footage ownerThe legal entity that operates the cameras and stores the recordingsOperating subsidiary that leases the building; recordings on its own video management systemCameras or storage run by a landlord, client or security contractor
Purpose at captureWhy the cameras were installed, quoted from the policy or signageSafety and incident review at packing stations, per the 2023 site camera policyPurpose limited to theft prevention or security with no wider use stated
Worker noticeHow workers were told, with the date and the exact wordingHandbook section and posted signs at each entrance; wording attachedNo written notice, or notice that names only security use
Releases and consent recordsWhich people signed what, and whether any can withdrawVolunteer operators signed recording releases for a 2025 pilot line onlyConsent claimed but no signed records can be produced
Biometric processingWhether anyone ran or plans face, gait or voice identification on the footageNone run; buyer must not run identificationFace matching used for timekeeping or access on the same cameras
Other people in frameVisitors, drivers, contractors or customers who appearCarrier drivers visible at dock doors 4 to 6Customer premises or public areas visible without any notice
Third-party contractsCustomer, client or site agreements that restrict recordingsTwo retail clients prohibit sharing images of their branded packagingFootage recorded on a client's site under that client's confidentiality terms
Labor agreementsCollective bargaining or works council terms on monitoringNone at this site; reviewed by HRAn agreement limits camera use and nobody has checked it
RetentionThe written retention period and whether old footage was kept outside itRecordings older than the retention period were exported for a safety studyFootage that should have been deleted under the company's own schedule
Excluded areasSpaces never recorded or removed from the offerBreak room, restrooms and the first-aid room excludedAny camera angle covering break, changing or medical areas

Task and capture fields: does the footage show a task a model can learn from?

Footage is useful for physical AI when each clip can be tied to one bounded task with a known start, a known end and a known camera. If you cannot say where a task begins and ends, a buyer cannot cut the footage into training examples without guessing.

Camera geometry matters more than resolution for most robot and world-model work. The ROS CameraInfo message, which many robotics stacks use, carries calibration matrices and says an uncalibrated camera should leave them zeroed, so a missing calibration is visible rather than hidden. It also defines the header timestamp as the time the image was acquired, not the time it was saved.

Clocks are where workplace footage most often breaks. A video recorder, a barcode scanner and a machine controller each keep their own time, and a few seconds of drift can attach the wrong scan to the wrong motion. Record which clock each stream uses and how, if at all, they were aligned.

Task and capture fields: does the footage show a task a model can learn from?
FieldWhat to writeIllustrative entryWhy a reviewer asks
Task definitionThe task in one sentence, with its start and end eventsPack one customer order: starts at tote scan, ends at carton label printDefines where one training example begins and stops
StepsThe ordered steps a trained worker followsPick items, scan each, select carton, fill void, seal, labelLets annotators mark step boundaries and skipped steps
Camera type and mountFixed, head-worn, body-worn or vehicle-mounted, and the angleFixed overhead camera above each of 12 stationsView angle decides which hand and object details are visible
CalibrationWhether intrinsics were measured and where they are storedNot calibrated; camera model and lens recordedDepth and 3D work need calibration or a way to estimate it
Frame rate and formatRecorded frame rate, resolution and codec, and whether variableVariable frame rate from the recorder; nominal 15 frames per secondVariable frame rate shifts labels if timestamps are not kept
Clock sourceWhich clock stamps video and each other streamRecorder synced to network time; scanners use their own clocksUnaligned clocks attach events to the wrong moment
Linked event streamsOther systems that record the same taskWarehouse management scans, carton weight checks, label print logEvent logs supply outcomes without manual labeling
Episode boundariesHow the start and end of each attempt can be foundDerived from tote scan and label print eventsRobot learning formats store data as episodes of steps

What operating variation and recovery behavior is missing from the footage?

Most workplace footage over-represents the normal case: the same shift, the same stations, the same product mix, and the task going right. Count the variation you have, and write down what is absent, before a buyer finds the gap in a sample.

Use counts, not adjectives. "Varied lighting" tells a reviewer nothing; "night shift on 3 of 12 stations, overhead lights only" tells them what the model will and will not see. The grid below is a starting point; add the dimensions that change how your task is performed.

Recovery behavior deserves its own line. A model that only sees clean completions learns little about what to do after a dropped item, a jammed sealer or a mis-scan. If your footage includes those moments, say how many and how you can find them. If it does not, say so; that is a scope fact, not a weakness to hide.

What operating variation and recovery behavior is missing from the footage?
DimensionWhat to countIllustrative countGap signal
Shifts and hoursHours of footage per shiftDay 70%, evening 25%, night 5% (illustrative)One shift dominates the record
OperatorsDistinct operators per station, counted not namedAbout 40 operators across 12 stations (illustrative)A handful of people perform most attempts
Product or job mixAttempts per product family or job type6 carton sizes; 2 rarely used (illustrative)Rare variants have too few attempts to learn from
EnvironmentLighting, season, temperature or clutter conditionsSeasonal peak with overflow totes in aislesAll footage from one calm period
Equipment stateAttempts with degraded or swapped equipmentSealer replaced in week 30 (illustrative)No record of equipment changes
Failures and recoveriesAttempts with an error and the recovery that followedMis-scans flagged in the warehouse systemFailures exist but cannot be located in the video

How should interventions, failures and outcomes be recorded?

Record the outcome of every attempt from a system of record rather than from someone watching the video. A quality hold, a returned order, a reworked part or a closed maintenance work order is stronger evidence of what happened than a reviewer's impression.

At a glance

How should interventions, failures and outcomes be recorded

Robot learning formats already distinguish these cases. Google's RLDS format stores an episode as a sequence of steps, marks the first and last step, and separates a terminal step, where the episode genuinely ended, from a truncated episode that simply stopped being recorded. Your fact sheet should make the same distinction for workplace attempts, because a clip that cuts off before the label prints is not a failed pack.

  • Completed as intended: the end event fired and no downstream system flagged the result.
  • Completed with intervention: a supervisor, a second worker or a remote operator stepped in; note who intervened by role, never by name.
  • Recovered error: something went wrong and the same worker corrected it within the attempt, such as a rescan or repack.
  • Failed: the attempt ended without the intended result, linked to the record that shows it, such as a quality hold or a rework ticket.
  • Truncated: the recording stopped before the end event, because of a camera gap, retention cut or export limit.
  • Unknown: no system of record links to this attempt; keep these out of any outcome statistics.

Worked example: a fact sheet for a distribution center's packing stations

This example is illustrative; the company, figures and findings are invented to show how the sheet works. A regional distribution center with 12 packing stations has fixed overhead cameras installed for safety review and roughly two years of retained recordings. The operations lead completes the sheet over three working days with HR and the IT administrator who runs the video system.

The rights block finds a written camera policy that names safety and incident review, posted signage, and a handbook section, but no releases except for a 2025 pilot line where 8 volunteers signed recording releases. Two retail clients' agreements prohibit sharing images of their branded packaging, and one camera angle covers part of the break area. The sheet records all three facts and removes that angle from the offer.

The task block defines a pack as tote scan to label print, which the warehouse system logs for every order. Clocks differ: the recorder uses network time and the scanners drift by up to a few seconds, so the sheet notes that alignment must be rechecked on a sample. Outcome data comes from the carton weight check and returns log, giving completed, recovered and failed attempts that can be found by timestamp.

The exception changes the offer. Because only the pilot line has signed releases and the other stations rely on a safety-only notice, the company narrows its first offer to the pilot line's footage plus de-identified event logs from all stations, and asks counsel whether the remaining footage can be offered at all. The sheet states that narrower scope rather than claiming all 12 stations.

Which answers should stop the offer or shrink its scope?

Some answers on the sheet are reasons to stop, and others are reasons to offer less. Decide which before anyone outside the company sees the footage.

At a glance

Which answers should stop the offer or shrink its scope

  • Stop and ask counsel when the cameras belong to a client, landlord or contractor, or when a client contract restricts recordings made on its premises.
  • Stop when any angle covers restrooms, changing areas, medical rooms or similar spaces; remove the footage rather than masking it.
  • Narrow the scope when notice named security or safety only and there are no releases: offer event logs, footage from consented lines, or angles where workers are not identifiable, subject to counsel's review.
  • Narrow the scope when footage was kept past the company's own retention schedule; that footage may need to be deleted, not licensed.
  • Bring in a privacy reviewer when any biometric processing has been run on the same cameras, when workers are in Illinois or another state with biometric rules, or when any footage was captured in the EU.
  • Bring in a robotics or physical AI reviewer when the task definition, clocks or outcome links are uncertain; they can say whether the footage is still useful without them.

Taking the fact sheet into a SourceX assessment

For a company that decides to license workplace video, SourceX runs the rights review, contracting, preparation and delivery around the selected recordings. Qualification starts with a brief look at the company itself, with nothing uploaded, so keep the fact sheet ready instead of sending footage with it.

If the footage goes ahead, nothing is handed over until the company has approved the scope and terms and signed, and video archives that large stay in the company's storage or move on encrypted drives, since SourceX does not host them. A completed sheet makes each of those steps faster because the rights questions have written answers.

Frequently asked questions

Do workers have to consent before workplace video is licensed for AI?

It depends on the jurisdiction, the notice workers received and what processing will be run. Some rules, such as Illinois's biometric law, require a written release before biometric identifiers are collected, and EU guidance says employers should generally not rely on consent at all. Record what notice and releases exist and have counsel review the specific facts.

Is a workplace video fact sheet the same as a datasheet or data card?

It covers the same ground but is narrower and more physical. Datasheets and data cards were designed for published research datasets; this sheet adds task boundaries, camera geometry, clocks and outcome links that physical AI teams need, and it can be completed by an operations lead.

Should faces be blurred before the fact sheet is written?

No; the sheet describes the footage and its permissions, and redaction is a later preparation step agreed in scope. Blurring also does not settle the rights question, because de-identification alone does not create a right to license the recordings.

Can security camera footage be offered as physical AI training data?

Sometimes, but security footage often fails on purpose and notice: the policy names security only, retention is short, and angles rarely show a whole task. Record the stated purpose exactly and expect to narrow the offer or stop if the notice does not reach other uses.

Do I need to send sample footage to see whether my company qualifies?

No. Whether a company qualifies is checked first through a short assessment of the business, and no footage or other data is uploaded at that stage. Keep the fact sheet ready instead, since its answers decide whether any footage moves forward.

Sources

  • The Datasheets for Datasets template asks whether the people in a dataset were notified, whether they consented, how consent was requested and whether it could be revoked. Source
  • The Croissant Responsible AI vocabulary defines fields for the collection process, collection type, collection timeframe and missing data. Source
  • Google's Data Cards Playbook describes data cards as structured summaries of essential facts about a dataset, including upstream sources and collection methods. Source
  • The ROS CameraInfo message says an uncalibrated camera should leave its calibration matrices zeroed and that the header timestamp should be the image acquisition time. Source
  • RLDS stores episodes as sequences of steps, marks first and last steps, and distinguishes terminal steps from truncated episodes. Source
  • Illinois's Biometric Information Privacy Act lists a scan of face geometry as a biometric identifier and excludes photographs from that definition. Source
  • Section 15 of the Act requires a private entity to inform the person in writing and receive a written release before collecting a biometric identifier. Source
  • EDPB guidance says an employee at work is usually not expecting to be monitored by the employer and that employers should generally not rely on consent for video processing. Source
  • The Ego4D paper describes collecting explicit informed consent for identity capture in some indoor footage and de-identifying the rest with redaction software and human review. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies