Short answer
RLDS and LeRobot are training-oriented dataset layouts: episodes of steps, or Parquet tables plus per-camera MP4 video. MCAP and ROS 2 bags are recording containers for timestamped multi-channel logs. HDF5 and Zarr are general array containers. Choose the training layout you consume, require raw logs as an archive, and make the contract specify schema, rates, timestamps and a conversion test.
Why format belongs in the contract
A robot-data supplier records on a robot stack, a handheld rig or a teleoperation station. Your training code reads something else. Between the two sits a conversion step, and that step is where timestamps drift, camera streams get dropped, action columns lose their meaning and episode boundaries shift by a frame.
Agree the format before collection starts, not at delivery. The container is the smaller decision. The larger one is what each field means, at what rate, in which frame. For those fields, see the robot trajectory dataset specification and action space and control frequency.
Two different jobs: recording and training
Formats fall into two groups that are often compared as if they were rivals.
Recording containers hold what the robot published, in time order. MCAP is an open-source container for multimodal log data with multiple channels of timestamped, pre-serialized messages; it does not care whether the payload is ROS 1, ROS 2, Protobuf, JSON or FlatBuffers, and it offers optional indexes and compression (Foxglove, n.d.). The ROS 2 ros2 bag tool records data published on topics and saves it for replay, with mcap as one example storage ID (Open Robotics, n.d.). So "MCAP vs rosbag" is a partly false contrast: an MCAP file can be what a ROS 2 bag is stored in.
Training layouts hold episodes shaped for a dataloader. RLDS stores sequential decision data as episodes, each with a dataset of steps carrying observation, action, reward, discount and first/last/terminal flags, in a standardized lossless format integrated with TensorFlow Datasets (Ramos et al., 2021). LeRobotDataset keeps tabular data in Parquet, one MP4 video per camera, and a meta/info.json declaring features, frame rate, codebase version and path templates, plus statistics, a task list and episode metadata; version 3.0 packs many episodes into each file (Hugging Face, n.d.).
General array containers sit underneath either group. HDF5 is a hierarchical, self-describing file format whose objects resemble directories and files, with direct partial access to data (The HDF Group, n.d.). Zarr is an open specification for chunked, compressed N-dimensional arrays, with v2 and v3 encodings (Zarr community, n.d.). Neither prescribes what an episode, an action or a camera is; the supplier's schema does.
Illustrative comparison
| Format | Primary role | Strength for a buyer | Watch for |
|---|---|---|---|
| MCAP / ROS 2 bag | Raw multi-channel log | Preserves per-message timestamps and all topics | Not training-ready; needs episode cutting and decoding |
| RLDS | Episode-and-step training data | Explicit step flags, shared tooling | Video handling and frame rates depend on how the supplier built it |
| LeRobotDataset | Training data with per-camera MP4 | Readable metadata, task list, statistics | Lossy video compression; check codebase version |
| HDF5 | Array container | Self-describing, partial reads | Layout is a convention each supplier invents |
| Zarr | Chunked array container | Chunked, compressed arrays, open spec | Same: schema is the supplier's |
The "watch for" column is a diligence prompt, not a ruling. Whether MP4 compression matters depends on whether your model consumes decoded frames at the original quality, which is a buyer decision.
What to require whichever format you pick
Illustrative delivery clauses a buyer might set:
# Illustrative format requirements (synthetic values)
training_layout: lerobot_v3 # buyer chooses
raw_archive: mcap # unmodified recordings retained
per_episode_fields:
- observation.images.cam_wrist # name, resolution, fps declared
- observation.state # joint order and units listed
- action # type: joint_delta | ee_pose_abs
timestamps: monotonic, source clock declared, per-stream offset reported
calibration_files: intrinsics + extrinsics per session
episode_index: success flag, operator_id, site_id, task string
conversion_test: supplier delivers script; buyer re-runs on 3 sample episodes
Add a schema document and a datasheet-style description (Gebru et al., 2018 proposed the pattern) so the meaning of every column survives staff changes. See datasheets for robotics datasets.
What the format cannot fix
A tidy container does not make poor data good. It cannot supply calibration that was never recorded, repair unsynchronized clocks, or recover actions that were never logged. Check sensor synchronization requirements separately.
Converting a raw log to a training layout is also lossy by design: topics are dropped, streams are resampled, video is re-encoded. Keep the raw archive so a later conversion with different choices stays possible, and note that retaining it raises storage and transfer planning questions.
Questions to ask suppliers
- Which robot stack, middleware and logger produced the raw recording, and can you deliver the unmodified log?
- What is the exact feature list, with units, joint order and coordinate frames, for observation and action?
- How were episode boundaries decided, and are failed or truncated episodes kept and flagged?
- At what rate was each stream recorded, and at what rate is it delivered after conversion?
- How are timestamps defined, and which clock does each sensor use?
- Which codec and quality settings were used for video, and was any frame dropped or duplicated?
- Is the conversion script part of the delivery, and can we re-run it on sample episodes?
- Which library and codebase versions does the delivered layout require?
- Where do calibration files live relative to episodes?
- How do you version a corrected re-delivery?
Acceptance tests
Use acceptance testing for deliveries: load a sample with your own dataloader, compare frame counts and durations between raw and converted files, plot action against state to confirm units, and verify stream offsets. Fail the delivery if the schema document and the files disagree.
Discuss your requirements
If you are scoping robot-log data and know the layout you train on, describe it in your request to SourceX. SourceX reviews the requirement and follows up about potential sources; specialist robot capture is scoped case by case, and a request does not guarantee a match. For the wider buying picture, see the procurement hub and the humanoid data guide.
Discuss your humanoid training-data requirements with SourceX.
Sources
- RLDS: an Ecosystem to Generate, Share and Use Datasets in Reinforcement Learning — episode and step structure.
- LeRobot — LeRobotDataset layout and metadata.
- MCAP — log container properties.
- Recording and playing back data (ROS 2) — ros2 bag recording and MCAP storage.
- The HDF5 Library & File Format — hierarchical self-describing format.
- Zarr — chunked array specification.
- Datasheets for Datasets — documentation pattern for datasets.