Privacy and preparation
Robotics dataset lighting conditions: why one well-lit demo is not a factory
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
To assess workplace video beyond one well-lit demonstration, list the lighting and environment states the real site goes through (shift, light source, daylight, glare, clutter, occupancy, camera placement), then count usable episodes in each state with logged camera settings, failures and outcomes. A coverage matrix shows which conditions are evidenced, which are thin and which were never recorded, so you know whether to offer the footage, narrow it or record more.
Video and sensors
Robotics dataset lighting
Key takeaways
- A demonstration filmed under one set of lights shows one condition of the site, not the site; the condition has to be named and counted like any other variable.
- Robot learning studies treat lighting as its own factor of variation, and a benchmark built on 14 perturbation factors lists lighting among the changes that hurt policies most.
- The coverage matrix logs each condition from evidence (timestamps, fixture records, camera settings), not from memory of when filming happened.
- Camera auto exposure and auto white balance can hide lighting differences in the pixels while changing frame timing, so the settings belong in the metadata.
- Cells with no logged condition are reported as unknown, never folded into the best-lit cell.
- Each shift that adds coverage also adds people on camera, so consent and visibility checks scale with the matrix.
How do you assess workplace video coverage beyond one well-lit demonstration?
You assess it by treating lighting and environment as recorded variables, then counting usable episodes per condition instead of per task. A single clean demonstration under daytime fixtures proves the task can be filmed; it says nothing about the night shift, the bay next to the roll-up door or the week the overhead bank was half out.
Start from the site, not the footage. Walk the floor with whoever runs the shift schedule and write down every state the work area actually passes through: which light sources are on, when daylight enters, where glare lands, how cluttered the bench gets by late afternoon and how many people are usually in frame. Only then open the archive and see which of those states the video really holds.
The answer you want at the end is a short sentence a robotics buyer can check: for example, that a library covers two shifts and both seasons at three of four stations, with the fourth station evidenced only in daylight. That sentence is worth more than a clip total.
The site walk also tells you which conditions do not vary. A windowless clean room with fixed fixtures may need no daylight or season rows at all, while a loading dock or a greenhouse packing line may need little else. Writing down what stays constant is part of the evidence, because it explains why a short matrix is still complete.
What robot learning research says about lighting as a separate factor
Research on visual robot policies isolates lighting as one of several environment factors and finds that changes in it can cut performance on their own. Xie, Lee, Xiao and Finn split a robot's surroundings into separable factors (background, lighting, distractor objects, table texture, object texture, table position and camera position) so they could measure how much each one contributes to the generalization gap, in simulation and on a real robot.
The Colosseum benchmark extends 20 manipulation tasks with 14 variation factors that randomize parts of the scene. Its authors report that changing the number of distractor objects, the target object's color or the lighting are among the perturbations that reduce model performance the most.
A separate large study by Lin and colleagues, built on more than 40,000 demonstrations, reports that generalization to new settings grows roughly as a power law in the number of distinct environments and objects, and that this diversity matters more than piling more demonstrations into one environment. For a business holding video, the practical reading is plain: extra hours filmed under the same lights add less than fewer hours filmed under conditions the library does not yet contain.
Which conditions belong on the axes of the matrix?
Put a condition on the matrix only when it changes what the camera sees or what the worker has to do, and when you can log it from evidence. Nine axes cover most indoor industrial and service sites; drop the ones that do not vary at your location and say so in the summary rather than leaving them blank.
- Light source: which fixture banks, task lamps or emergency lighting are on; record from the building management log or a shift checklist.
- Daylight ingress: windows, skylights and open dock doors, which change by hour and by season.
- Shift and time of day: the timestamp, mapped to the shift calendar, so night and weekend work show up as their own rows.
- Glare and reflections: polished stock, stainless benches, wet floors and safety glass that throw highlights into the lens.
- Smoke, dust, mist or steam: welding fume, coolant mist and kitchen steam scatter light and soften edges.
- Clutter and surface state: clean start of shift versus end-of-shift benches, oily or wet surfaces, packaging debris.
- Occupancy: how many other people pass through or stand in frame, which also drives the consent work.
- Camera placement: mount position, height and angle per episode, plus any change after a bump or a re-mount.
- Camera settings: exposure and white balance mode, frame rate and calibration state at recording time.
The lighting and environment coverage matrix
The matrix pairs each condition with the field you log, the minimum evidence that lets a cell count and the red flag that stops it from counting. Fill one row per condition value (for example, night shift or dock door open), then count usable episodes in each row for every station or task you plan to offer.
The thresholds below are editorial rules of thumb for a first pass, not findings from a study; agree the real minimums with the buyer, because a hard manipulation task under poor light may need far more examples than an easy one.
Once the counts are in, read the matrix station by station and apply one decision per station rather than one verdict for the whole library. The rules below keep the decision honest when a few rows look strong and others look empty.
- Offer as is: every condition row the site walk found has usable episodes above the agreed floor, with logged evidence and linked outcomes.
- Offer narrowed: some rows are empty or thin, so the offer names only the covered conditions (for example day and evening shift, door closed) and states the gaps.
- Record more first: a row that the buyer's deployment will certainly meet, such as night operation, is empty, and a few weeks of targeted filming would fill it.
- Hold: occupancy rows lack consent records, or camera settings and timestamps are missing for most episodes, so no condition claim can yet be verified.
- Concentration check: if one condition holds more than about half of all episodes (an illustrative ceiling), say so in the summary even when other rows clear the floor.
| Condition axis | Field to log per episode | Minimum evidence for a cell to count | Red flag that blocks the cell |
|---|---|---|---|
| Light source | Fixture banks on, task lamp on or off, emergency lighting | Shift checklist or building log matching the episode date | Condition inferred only from how bright the frame looks |
| Daylight ingress | Window or door state, hour, month | Episodes in at least two seasons or both open and closed door states | All daylight footage from one month |
| Shift and time of day | Acquisition timestamp mapped to the shift calendar | Roughly 30 usable episodes per shift per station (illustrative floor) | File-creation time used instead of acquisition time |
| Glare and reflections | Glare zone tag, material on the bench | Episodes where the hand or tool passes through the glare zone | Glare episodes cut because they looked bad |
| Airborne haze | Process running (welding, coolant, steam) yes or no | Episodes recorded while the process actually runs | Haze staged with a fan or filmed only during breaks |
| Clutter and surface state | Start, middle or end of shift; wet or dry surface | End-of-shift and messy-bench episodes present | Bench tidied for filming every time |
| Occupancy | Count of other people in frame, consent record ID | Every visible person matched to a release covering AI training | Bystanders with no record, or faces blurred inconsistently |
| Camera placement | Mount ID, height, angle, calibration file version | Calibration file present for each mount period | Mount moved with no new calibration or note |
| Camera settings | Auto or manual exposure and white balance, frame rate, anti-flicker mode | Settings captured from the device at recording time | Settings unknown and frame rate varies without explanation |
Camera settings that make a lighting claim provable
Camera settings belong in the episode record because automatic modes can mask lighting changes in the image while altering timing. Intel's RealSense SDK, for instance, exposes options for exposure, gain and white balance, notes that setting a manual exposure or white balance value disables the automatic mode, offers a power line frequency setting for anti-flicker (off, 50 Hz, 60 Hz or auto) and has an auto exposure priority option that lets the sensor adjust frame rate depending on lighting conditions.
That last point matters for robotics buyers who align video with joint states or tool signals. If dim scenes quietly drop the frame rate, the night-shift rows of your matrix may also be the rows with the worst synchronization, and the buyer will find it before you do unless the settings travel with the footage.
Calibration needs the same care. The ROS 2 CameraInfo message keeps intrinsic calibration values fixed until the camera is recalibrated, stamps each message with the image acquisition time and treats a zeroed camera matrix as a sign of an uncalibrated camera. Record which calibration file applied to each mount period, and treat a re-mounted camera as a new placement until it has one.
Worked example (illustrative): a machining cell over three shifts
This example is invented to show how the matrix reads; it does not describe a real site or dataset. A contract machine shop holds 1,200 episodes of an operator loading and deburring parts at four stations, recorded by fixed overhead cameras over eight months.
Mapped to the shift calendar, 74% of episodes come from day shift, 21% from evening shift and 5% from night shift. Station 4 sits beside a dock door, and every one of its 160 episodes was recorded between May and July with the door open, so its winter state with the door closed and the high bays dimmed has no evidence at all. Glare tags show that polished stainless stock appears in 90 episodes, but only 12 of those come from the station with the strongest overhead reflections, and 30 glare episodes had been trimmed out of the archive because they looked poor.
The readout: stations 1 to 3 can be offered for day and evening conditions; night shift is evidenced at roughly 15 episodes per station, under the illustrative floor, so it is offered only as a labeled thin slice or recorded further; station 4 is offered as a summer, door-open subset only. The trimmed glare episodes are restored from backup and tagged rather than discarded, because those are the scenes a policy most needs to see.
Which recoveries and interventions vanish when filming favors good light?
Lighting gaps usually travel with behavior gaps, so check both at once. Operators under poor light slow down, re-grip, lean in, switch on a task lamp or ask a colleague for help; these recoveries are exactly what a robot policy has to learn and exactly what disappears when filming is scheduled for the best-lit hours.
At a glance
Which recoveries and interventions vanish when filming favors good light
For each condition row, tag whether the episode contains a failed attempt, a recovery, a human intervention or a quality outcome such as a rejected part. A row with plenty of clean successes but no recoveries is thin even if its count looks healthy. Ask the shift lead which mistakes are more common at night or in winter, and look for them specifically.
- Missing recovery: the operator re-positions a part after glare hides an edge, but those episodes were trimmed.
- Missing intervention: a supervisor steps in during night shift, yet the night rows hold only solo work.
- Missing outcome: inspection results exist in the quality system but are not linked to the episodes.
- Missing state change: the lamp is switched on mid-task, and the episode is split so the transition is lost.
Can augmentation stand in for a missing lighting condition?
Augmentation can widen a condition you already recorded; it cannot prove a condition you never saw. Training tools make lighting effects easy to simulate: the LeRobot dataset tooling, for instance, offers brightness, contrast and color jitter along with shadows, gamma correction and patches of uneven brightness described as camera and lighting effects seen on real robots.
Those transforms do not model the operator leaning in, the coolant mist, the shadow of a passing forklift or the slower pace of a tired night crew. Treat augmentation as the buyer's training choice, not as part of your coverage claim, and never present augmented frames as recorded conditions.
The assumption that most often flips the decision is the one about unlogged episodes. If 300 episodes have no reliable timestamp, do not assign them to day shift because most footage is from day shift. Report them as a separate unknown row, show the matrix with and without them, and let the buyer see that the night-shift claim does not depend on a guess.
Consent and visibility grow with every shift you add
Extending coverage to new shifts and zones brings new people into frame, so the consent check has to be repeated per row, not once for the library. Night and weekend crews may include temporary staff, contractors or cleaning teams who never signed the release that day-shift operators did.
Match each visible person to a record that covers recording for AI training and licensing to third parties, and note whether faces, badges or screens in the background need masking. This is general information rather than legal advice; counsel should review the notice and consent records for each place and period the footage was recorded.
Bringing a coverage-mapped video library to SourceX
A filled matrix makes a first conversation about workplace video much shorter, because it shows which stations, shifts and seasons are evidenced before anyone looks at a frame. A company with a mapped video library can learn whether it qualifies with SourceX through a short company assessment, without sending a single clip.
As a broad guide, buyers favor long, consistent histories and large volumes, though rarer work can qualify with less, and each library is judged on its own. Footage is delivered only after the company signs off on scope and the agreement, and because SourceX does not host multi-terabyte datasets, a large library stays in the company's storage or ships on encrypted drives.
Frequently asked questions
Do I need light meter readings for every episode?
No. A light meter reading per station and condition, taken once and stored with the matrix, is useful context, but per-episode readings are rarely practical. What matters per episode is evidence of the condition: the timestamp, the fixture or door state and the camera settings.
Should I cut footage that is too dark or full of glare?
Keep it and tag it. Poorly lit and glare-heavy episodes are often the scenes a robot policy fails on, so a buyer may value them more than clean footage. Exclude an episode only when it cannot be used at all, and record why it was excluded.
Is fixed camera footage treated differently from head-mounted video?
The matrix works for both, but the axes weigh differently. Fixed cameras make placement stable and lighting the main variable, while head-mounted cameras add motion blur, changing angles and the wearer's own shadow, so camera placement and settings need more rows.
How many seasons of footage are enough?
It depends on whether daylight reaches the work area. A windowless cell may need no seasonal split at all, while a station by a dock door or skylight needs both its bright and dark states. Let the site walk decide which axes vary before you count.
Can I describe a library as covering all lighting conditions?
Only if every condition row your site walk found has usable episodes with logged evidence. A more accurate claim names the conditions covered and the ones that are thin or missing, which is also the claim a buyer can verify.
Sources
- Xie, Lee, Xiao and Finn treat lighting, background, distractors, textures, table position and camera position as separable factors and measure each one's contribution to the generalization gap of visual robot policies. Source
- The Colosseum authors report that changes in distractor count, target object color or lighting are among the perturbations that reduce robot manipulation performance the most. Source
- Colosseum implements 20 RLBench manipulation tasks and supports 14 variation factors that randomize parts of the simulation. Source
- Lin et al., using over 40,000 demonstrations, report that generalization scales roughly as a power law in the number of environments and objects, with diversity mattering more than more demonstrations per environment. Source
- RealSense options include exposure and white balance values that disable the automatic modes when set, an anti-flicker power line frequency setting and an auto exposure priority that lets frame rate change with lighting. Source
- The ROS 2 CameraInfo message keeps calibration values fixed until recalibration, stamps acquisition time and treats a zeroed camera matrix as uncalibrated. Source
- LeRobot dataset v3 stores schema and frame rate in meta/info.json and offers image transforms that imitate camera and lighting effects such as brightness jitter, shadows and gamma correction. Source
See if your company qualifies
A short company assessment. No data uploads are needed.