Privacy and preparation
Robotics video camera synchronization for workplace demonstrations
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Synchronize multi-camera workplace demonstrations by putting every camera on one time reference, recording a sync event visible in all views at the start and end of each take, and writing a per-take timecode manifest with first and last frame times, frame counts, offsets, drift and recording segments. Run a scripted acceptance test right after each take so a restart or drift is caught while a re-shoot is still possible.
Video and sensors
Robotics video camera
Key takeaways
- Cameras set to the same frame rate drift apart because each counts time with its own oscillator.
- Pick the lightest time reference that meets the buyer's tolerance: trigger, timecode, PTP, NTP or events alone.
- Record start and end sync events in every take, even with a shared clock, to measure offset and drift.
- A per-take manifest with recording segments makes a mid-take restart visible instead of a silent misalignment.
- Run the acceptance test on site, right after recording, so failed takes can be re-shot.
How do you synchronize multi-camera workplace demonstrations?
Put every camera on one time reference, mark a sync event that all cameras can see and hear at the start and end of every take, and write a timecode manifest for the take that records each stream's first and last frame time, frame count, measured offset and drift. Then run a short acceptance test before the take counts as recorded. If it fails, fix or re-shoot while the worker and the setup are still there.
The reference can be a hardware trigger, a shared timecode generator or a network clock; which one depends on how tight the alignment must be and what the cameras support. The sync event and the manifest are not optional with any of them, because they are what lets someone who was not in the room confirm that a frame from the overhead camera and a frame from the head-mounted camera show the same instant.
This page is written for the business hosting or recording the demonstrations, such as a manufacturer, a 3PL or a field service company, whose team will run the cameras day to day. It gives the manifest fields, the acceptance test and one passing and one failing take.
Why do cameras set to the same frame rate still drift apart?
Each camera counts time with its own oscillator, so two cameras both set to 30 frames per second produce slightly different real rates, and the gap grows with every minute of a take. Phones and action cameras may also vary their frame rate with lighting or heat, drop frames when storage is slow, and start recording after different delays when someone presses record on each one.
Dataset tooling can hide the problem. The LeRobot documentation warns that a frame's timestamp is derived from its index, so a session that actually ran at 25 Hz still produces a dataset that claims 30, and the recorded motion is simply faster than reality. The same library checks on load that consecutive timestamps are separated by one frame interval plus or minus a tolerance, which defaults to 0.0001 seconds; that check catches gaps in a stream, but it cannot tell you whether two streams agree with each other.
Cross-stream alignment is a separate step. In ROS 2, for example, the message_filters package offers an ExactTime policy that matches messages only when their header timestamps are identical and an ApproximateTime policy that pairs messages with an adaptive algorithm. Both rely on the timestamps being right in the first place, which is what the rig and the manifest have to establish.
Which time reference should the camera rig follow?
Choose the lightest method that meets the tolerance the buyer sets, and record the choice in the manifest. A slow assembly step can tolerate more offset than a fast pick where the gripper and the hand move within a single frame interval.
Mixed rigs are common in workplaces: a pair of machine-vision cameras that accept a trigger, plus a head-mounted camera that does not. In that case, trigger what you can, put everything on the same network or timecode reference, and rely on the sync events to measure the camera that sits outside the trigger chain. Record in the manifest which cameras were triggered and which were only referenced, because a buyer will weigh their offsets differently.
| Method | How it works | What it fixes | What it leaves open |
|---|---|---|---|
| Hardware trigger | One camera or a generator drives the others through a sync cable; Intel's RealSense multi-camera guide sets each camera to master or slave and wires sync and ground pins | Exposure start on every frame, so offset and drift are removed at the source | Cabling on a shop floor; only cameras with a sync input can join |
| Shared timecode | A generator feeds SMPTE timecode, which labels frames as hours:minutes:seconds:frames, to every camera | A common frame address across cameras and audio recorders | Cameras without a timecode input; sub-frame alignment |
| Precision Time Protocol | Devices discipline their clocks over Ethernet using IEEE 1588, which IEEE describes as capable of sub-microsecond accuracy | Wall-clock agreement across wired sensors, robot controllers and recorders | Needs PTP-capable network gear; cameras may still timestamp on arrival rather than at exposure |
| Network Time Protocol | Devices sync to a time server; RFC 5905 puts potential accuracy in the tens of microseconds on fast LANs | Coarse agreement between laptops, phones and loggers | Accuracy degrades on Wi-Fi and busy networks; not frame-level on its own |
| Sync event only | A clap, light flash or slate visible and audible in every view at start and end | Offset and linear drift, measured after the fact | Anything that changes mid-take, such as a restart or dropped frames |
What should the timecode manifest record for each take?
One manifest per take, written as the take is recorded and closed when it passes the acceptance test. Store times in a single unit with a declared epoch. The MCAP container specification is a useful model: its timestamps are unsigned 64-bit nanoseconds since a user-understood epoch such as Unix time or robot boot time, and each message carries a log_time for when it was recorded and a publish_time for when it was published.
At a glance
What should the timecode manifest record for each take
Whatever container you use, write the manifest as a small text file next to the media, in JSON or CSV, so it can be read without special tools and checked by a script. The fields below are the minimum a robotics team needs to align streams without asking the site team questions.
- take_id, task_id, site_id and session_id, so every stream from the same attempt shares one identifier.
- rig_version and calibration_id, pointing to the camera layout and the calibration captured for this session.
- time_reference: hardware trigger, timecode, PTP, NTP or event only, plus the epoch used for all times in the manifest.
- Per camera: camera_id, role (overhead, side, wrist, head-mounted), device model, serial, firmware, resolution and nominal frame rate.
- Per camera: first_frame_time, last_frame_time, frame_count, measured frame rate and dropped frame count.
- Per camera: timecode_start, if the camera records timecode.
- sync_events: the time each camera saw the start event and the end event, as frame numbers and times.
- Per camera: offset at start and offset at end relative to the reference camera, in milliseconds, and drift per minute derived from them.
- segments: a list of continuous recording spans per camera, so a restart is visible as a second segment rather than a silent jump.
- intervention_log: times when an operator paused, corrected or took over the task, on the same clock.
- consent_ref: a pointer to the consent record for the people in view, never the personal details themselves.
- result: pass or fail for each acceptance check, the checker, and the date.
Which checks make up the synchronization acceptance test?
Run these checks on every take, ideally by script right after recording. Tolerances are set by the buyer for the task; the pass rules below show the form a rule should take, and any figures in them are illustrative.
| Check | How it is measured | Illustrative pass rule | On failure |
|---|---|---|---|
| Continuous timestamps | Frame times per camera are strictly increasing | No repeats or backward steps | Split into segments; flag the camera |
| Frame interval | Gap between consecutive frames versus one nominal frame interval | Every gap within the buyer's tolerance | Count and locate dropped frames |
| Frame count | frame_count against duration times measured frame rate | Differs by no more than the dropped frame count | Recheck nominal versus measured rate |
| Start offset | Time each camera saw the start event versus the reference camera | Within the buyer's offset tolerance, for example under half a frame interval | Apply a measured offset or re-shoot |
| End offset and drift | End-event offset minus start-event offset, divided by take length | Drift per minute within tolerance | Resample or shorten takes |
| Event in every view | Start and end events visible or audible in each stream | Present in all streams | Take cannot be verified; re-shoot |
| Segment integrity | Number of recording segments per camera | One segment per camera per take | Treat each segment as its own take or re-sync |
| Intervention alignment | Operator log times fall inside the take and match visible events | Each logged intervention visible in at least one view | Correct the log before release |
What separates a passing take from a failing one? (illustrative)
Both records below are illustrative, not taken from any real site or customer. A packaging cell is recorded with three cameras: an overhead camera chosen as the reference, a side camera at bench height and a head-mounted camera on the operator. The rig uses a shared timecode generator and a light flash at start and end.
In the passing take, a four-minute carton assembly, every camera records one segment. The side camera sees the start flash two milliseconds after the overhead camera and the end flash three milliseconds after, so drift is a quarter of a millisecond per minute. The head-mounted camera reports four dropped frames, all logged, and the frame count matches duration and measured rate once those are subtracted. Every check passes and the manifest is closed.
In the failing take, the head-mounted camera's battery runs low at minute three and the operator swaps it. The camera resumes recording with a new file and a restarted internal clock. Its frame times jump backward, the segment check shows two segments, and the end flash appears eleven seconds earlier on its timeline than on the reference camera's. Without the manifest, the two files would be concatenated and every later frame would be misaligned. With it, the team marks the take as failed for the head-mounted view, keeps the overhead and side views, and either re-shoots or treats the second segment as a separate take with its own start event, which in this case it lacks.
Is the footage showing the variation and recovery a buyer needs?
Synchronized views earn their cost on the moments a single camera misses: a dropped part, a regrasp, a jam cleared by hand, a supervisor stepping in. A rig that is perfectly aligned but only ever records clean, repeated runs answers the wrong question.
Plan camera roles around those moments. The side view should see hands and parts at the point where errors happen, the head-mounted view should show what the operator looks at before recovering, and the intervention log should mark each recovery on the same clock so it can be found across all views. Record the conditions that vary across takes, such as part supplier, lighting and shift, in the take header so variation is measurable later rather than assumed.
What does a synchronized recording day look like on site, consent included?
Treat synchronization as a routine with three checkpoints: before the first take, after every take and at the end of the day. Most failures come from small changes between takes, such as a swapped battery, a camera knocked off its mount or a laptop that slept and lost its network time, rather than from the rig design.
Before the first take, power every device, confirm each one is locked to the time reference, and record a short test take with a sync event. Run the acceptance test on it. Capture calibration for the session and write its id into the manifest template so every take inherits it.
After every take, close the manifest, run the checks and decide on the spot: accept, re-shoot or split. Note anything that changed, from lighting to a different operator, in the take header. If a device restarted, start a new take rather than continuing.
At the end of the day, export manifests with the media, compute a checksum for every file, and confirm that the count of files matches the count of camera segments the manifests list. A file without a manifest entry, or an entry without a file, is resolved before anything leaves the site.
Keep consent outside the manifest and link to it. The manifest travels with the footage and may be shared with a robotics team evaluating the data, so it should identify takes, cameras and times, not people. A consent_ref field pointing to the record your HR or legal team holds lets a reviewer confirm that each person in view agreed to the recording and its intended use.
Camera placement affects consent too. A wide overhead view can capture coworkers who never joined the program, so note in the take header which areas each camera covers and whether anyone outside the task appeared. Those takes can then be masked, trimmed or excluded according to the plan your counsel approves.
When does the routine need a camera or robotics engineer?
The manifest and the acceptance test handle most workplace rigs, but some setups need someone who can work at the sensor level. Bring one in before recording, not after a batch has failed.
- The buyer's tolerance is smaller than one frame interval, which usually means a hardware trigger and cameras with exposure-time stamps.
- Fast motion with rolling-shutter cameras, where different rows of one frame are captured at different times.
- Wireless cameras or phones on Wi-Fi, where network time and arrival timestamps vary from take to take.
- Robot controller logs, force sensors or depth cameras must join the video on the same clock.
- Sessions run for hours, so drift and storage limits interact and takes must be split by design.
- Workers outside the task appear in view; the consent and notice plan, and counsel where needed, decide what may be kept.
How do synchronized recordings become a licensable dataset?
A business with a clean manifest per take, consent records for the people in view and a clear task list has the core of a dataset a robotics team can evaluate. Before any footage is discussed, SourceX checks whether the company qualifies through a brief assessment that needs no uploads.
No take leaves the site before the company approves scope and terms and signs the agreement. Multi-camera video adds up quickly; SourceX does not host it, so large deliveries stay on the company's own servers or go out on encrypted drives.
Frequently asked questions
Do I need hardware sync for multi-camera robot demonstrations?
Not always. If the buyer's tolerance is larger than a frame interval and takes are short, shared timecode or network time plus start and end sync events can be enough. If alignment must be tighter than one frame, or motion is fast, a hardware trigger is the safer choice.
Why record a sync event if the cameras share a clock?
The shared clock tells you what each device believes the time is; the sync event shows whether the frames actually agree. It also catches problems a clock cannot, such as a camera that restarted mid-take or one that stamps frames on arrival rather than at exposure.
What is the difference between offset and drift?
Offset is how far apart two cameras are at one moment, such as the start event. Drift is how that gap changes over time, measured by comparing the offset at the start and end of a take. A constant offset can be corrected by shifting one stream; drift needs resampling or shorter takes.
How long should one take be?
Long enough to capture the whole task attempt, including recovery, and short enough that drift stays within tolerance without correction. Many teams record one attempt per take. Restarting a device always ends the take for that camera.
What should never go into the timecode manifest?
Names, faces, badge numbers or any other personal details of people in view. The manifest should point to a consent record by reference, so it can be shared with a buyer without exposing who was filmed.
Sources
- The LeRobot documentation says a frame's timestamp is derived from its index, so a session that ran at 25 Hz still produces a dataset that claims 30 and the recorded motion is faster than reality. Source
- LeRobotDataset checks that each timestamp is separated from the next by 1/fps plus or minus tolerance_s, which defaults to 0.0001 seconds, including for frames decoded from video. Source
- ROS 2 message_filters offers an ExactTime policy that matches only identical header timestamps and an ApproximateTime policy that matches messages with an adaptive algorithm. Source
- Intel's RealSense multi-camera configuration guide sets cameras to default, master or slave sync mode, wired through sync and ground pins with one camera acting as master. Source
- SMPTE ST 12-1 time and control code labels frames with an hours:minutes:seconds:frames address and offers user bits for identifiers or metadata. Source
- IEEE 1588-2019 defines the Precision Time Protocol for networked measurement and control systems, which IEEE describes as achieving sub-microsecond synchronization accuracy. Source
- RFC 5905 (NTPv4) says its algorithms extend potential accuracy to the tens of microseconds with modern workstations and fast LANs. Source
- The MCAP specification defines timestamps as uint64 nanoseconds since a user-understood epoch and gives each message a log_time and a publish_time. Source
See if your company qualifies
A short company assessment. No data uploads are needed.