Delivery and Transfer of Large Robotics Datasets

The buyer's problem

Multi-camera video, depth, force and robot logs add up quickly. A collection that fits on a laptop for a sample may need terabytes at full scale, and the delivery step becomes a project of its own. Failures here are mundane and expensive: a transfer that stalls at 90%, a drive that arrives with corrupted files, a bucket opened too widely, or an archive that cannot be verified against what was licensed.

This page covers the mechanics and is not legal advice: for terms that govern use, see licensing terms for robotics training data. For what to do after delivery, see acceptance testing for deliveries.

Delivery routes

Large datasets at the supplier's side may be too big to upload anywhere, which is common when recordings are kept on site storage. In that case, ask whether the supplier can hand over from where the data lives, or whether the drive route is the realistic one.

Do the arithmetic before agreeing

Transfer time is size divided by sustained throughput, and sustained throughput is usually well below the nominal link rate. An illustrative calculation: moving 10 TB at an effective 500 Mbit/s takes about 44 hours of uninterrupted transfer (synthetic numbers; use your own). Real transfers add pauses, retries and verification time. Agree which party pays egress, whether it is metered by region, and whether partial deliveries are billed or counted.

For physical drives, count the total handling time too: copy to drives, shipping, ingest and verification at the buyer. Ask how many drives, what file system and which encryption tool, so the buyer can mount them without surprises. Ask also about format; see robot learning data formats, which affects whether many small files or a few large containers are delivered. LeRobot, for example, stores tabular data in Parquet alongside per-camera MP4 video, with newer versions packing many episodes per file (LeRobot); file layout changes both transfer behavior and verification.

Integrity and completeness

Every delivery should carry a manifest and a way to prove it matches. Ask for:

• a manifest listing each file, size, checksum algorithm and checksum value;
• a count of episodes, hours or items per modality, matching the license schedule;
• a version identifier for the delivery and for the schema;
• a procedure for verifying, such as a script or documented command;
• a procedure for reporting discrepancies, and the supplier's duty to re-send.

Verify by recomputing checksums after transfer and comparing the manifest to the license schedule. Checksums prove bytes arrived intact; they do not prove the content is what you ordered. Content checks belong in acceptance testing.

Security during transit and storage

Encrypt data in transit and at rest, and decide who holds the keys. For drives, send the key by a separate channel from the shipment. For buckets, use time-limited access, least-privilege permissions and logs of who accessed what. Confidential business information in footage raises the stakes; see confidential business information in footage.

ISO/IEC 27001:2022 defines requirements for an information security management system, for organizations of any size and sector, with ISO/IEC 27002:2022 providing the detailed controls, and certification is voluntary (ISO/IEC 27001:2022). If a buyer's own policy requires suppliers to operate such a system, ask what scope applies to the delivery environment. A supplier's claim of certification is a claim to verify with the certificate and its scope, and the absence of certification is not itself a finding about delivery security. Do not infer anything about any specific counterparty from the standard's existence.

Media sanitization and end of life

Drives and buckets outlive the project unless someone closes them. NIST Special Publication 800-88 Revision 2, Guidelines for Media Sanitization, was published in September 2025 and supersedes Revision 1 (NIST, 2025). Reference the current revision in contracts when specifying how returned, replaced or retired media will be sanitized, and ask for written confirmation of the method used. Tie this to deletion duties; see retention, deletion and audit rights.

Illustrative delivery plan

# Illustrative only - synthetic values; the parties set every term
delivery_id: dlv_2026_03_A (synthetic)
volume: 18 TB total in 4 tranches (tranche sizes stated in manifest)
route: encrypted drives for tranche 1; cloud copy for tranches 2-4
manifest:
  files: path, bytes, sha256, modality, episode_id
  schedule_match: counts per modality equal licensed schedule
encryption:
  at_rest: full-disk encryption on drives; bucket-side encryption
  keys: delivered separately; rotated after acceptance window
access:
  bucket: time-limited, read-only, named identities, access logging
verification_window: 30 days from receipt (buyer sets)
discrepancy_process: written report, supplier re-sends within agreed period
close_out:
  copies_returned_or_sanitized: method referenced to current NIST SP 800-88 revision
  written_confirmation: yes

Limitations

Delivery mechanics do not fix content problems; a perfectly transferred dataset can still be mis-synchronized or mislabeled. Hosting costs, egress charges and storage retention are commercial terms that vary by provider and region. Sovereignty or residency rules can limit which storage locations are allowed; see EU AI Act and GDPR considerations. Derived copies on buyer systems (caches, thumbnails, training shards) are part of deletion scope and often forgotten.

Questions to ask suppliers

1. How large is the total delivery in files, hours and terabytes, per modality?
2. Which delivery routes can you support, and which do you recommend for this volume?
3. Can you provide a manifest with checksums before the transfer?
4. Who pays egress, shipping and media costs?
5. What encryption is used in transit and at rest, and how are keys delivered?
6. Can delivery be staged so early tranches can be accepted first?
7. What happens if files arrive corrupted or incomplete?
8. How long will you keep your copy, and how is it sanitized afterward?
9. Does the delivery environment fall within a documented information security management scope?

Discuss your requirements

If you expect a large delivery, describe the volume, formats and security requirements to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, agrees allowed uses in a license, and coordinates delivery and payment. A request does not guarantee a match; SourceX follows up about potential sources. See the humanoid training data guide and the procurement hub.

Discuss your humanoid training-data requirements with SourceX

Sources

• NIST SP 800-88 Rev. 2 Guidelines for Media Sanitization — NIST SP 800-88 Rev. 2 (September 2025) media sanitization, supersedes Rev. 1.
• ISO/IEC 27001:2022 Information security, cybersecurity and privacy protection - Information security management systems - Requirements — ISO/IEC 27001:2022 ISMS requirements; certification voluntary.
• LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch — LeRobot dataset layout: Parquet plus per-camera MP4 video.