Training data for robotics and embodied AI from real work
Robots and embodied AI models that must do skilled physical work learn from first-person video of trained people doing it on real sites, paired with records of each task and its result. SourceX runs new, consented egocentric video collection with partner businesses in trades such as maintenance, assembly and construction — each program has conditions, for example a minimum team size and a region — and sources existing field service, manufacturing quality, construction and CAD records for the same kinds of work.
Dataset types to start with
- First-person video of skilled manual work
Head-mounted video of trained workers shows hands, tools and parts from the viewpoint a robot or video model has to act from, including grasps, step order and recovery from mistakes. It is recorded new for each program with partner businesses, not pulled from an archive.
- Field service and maintenance work orders
Work orders trace symptom, diagnosis, parts and fix, with technician notes and photos. They give task structure, failure modes and success labels for repair and maintenance work, and can define the task list for a video collection.
- Manufacturing quality and inspection records
Inspection results, defect images, nonconformance reports and corrective actions define what correct work looks like on a line: labeled examples for visual inspection models and acceptance criteria for assembly and handling tasks.
- Construction project records
Daily logs, site photos, inspection reports and punch lists record how work is sequenced and how a site changes week to week, which helps site-monitoring models and task planning in unstructured environments.
- CAD and PCB engineering files with revision history
Native CAD assemblies and PCB layouts with revision history give exact geometry, tolerances and part relationships for the objects a robot handles. They become simulation assets, grasp targets and assembly sequences, and show how a part changed between revisions.
Why this data is hard to get
Skilled work on real sites is rarely filmed
Open first-person video collections mostly show everyday activities or tasks performed for the recording. Trained workers doing their actual jobs at working pace, with real tools, parts and site conditions, are scarce on film, so this footage has to be recorded new, with partner businesses and with consent.
Consent reaches beyond the camera wearer
Each recorded worker has to agree freely, which needs care inside an employment relationship and, in some countries, consultation with works councils or worker representatives. Co-workers, customers and members of the public in frame need notice, blurring or exclusion.
Worksites carry confidentiality and safety rules
A technician filming at a customer's plant needs that site's permission as well as the employer's, and proprietary processes, screens and documents in view must be excluded or masked. Cameras must not add hazards, for example where equipment has to be rated for explosive atmospheres.
Footage without task context teaches little
Unlabeled video shows motion but not intent. Learning a policy or a success detector needs to know which task was attempted, which steps it involved and whether it worked, and that information lives in work orders, procedures and inspection results, not in the pixels.
Engineering files come with other people's rights
CAD models and drawings are often made for, and owned by, a manufacturer's customers, and some fall under export controls such as ITAR or the EAR. Each file needs ownership review and screening before it can be licensed.
Two supply paths: new recordings and existing records
Robotics data through SourceX comes from two different supply paths. First-person video of skilled manual work is a new collection program, not an archive: SourceX coordinates recording with partner businesses in manual-work industries, such as maintenance, installation, assembly and the construction trades. Every program sets requirements for its partners, such as a minimum team size and a region, so that each site has enough people doing the work to capture real variety without disrupting the job. Recording happens on the partner's own premises, under consent, site and safety arrangements agreed before the first session.
The other recommended types are existing records, sourced to spec from company archives: work orders and technician notes from field service systems, inspections and nonconformance reports from quality systems, daily logs and site photos from construction project tools, and CAD and PCB files with revision history. They need no new recording, and they often come from the same industries as the video, so the two can be planned together: the records say which tasks matter and how they end, and the video shows how they are done.
What different robot models take from this data
Real-work data serves several kinds of robot and embodied AI model, and each uses a different part of it:
- Manipulation policies. Human first-person video helps pretrain visual representations and teaches object affordances and grasp choice, but it contains no robot actions. It complements demonstrations recorded on your own hardware rather than replacing them.
- Dexterous hands. Retargeting human hand motion to a robot hand needs the hands in frame, calibrated cameras and, ideally, synchronized IMU data.
- Task planning. Planners for long, multi-step work learn from step-segmented video together with procedures and work orders, including the symptom, diagnosis and fix chains in maintenance records.
- Inspection and site perception. Defect images with dispositions and site photos with inspection outcomes train models that judge whether work was done right.
- Simulation. CAD geometry becomes object and scene assets for training and testing in simulation before anything runs on a real robot.
Recordings from workers and sites a model has never seen are also the most direct test of whether it generalizes, so plan a held-out slice from the start; see private evaluation sets.
Planning a program
An egocentric collection program is planned backward from the tasks your model must eventually perform: which industries do them, in which regions, with which tools and variations. Settle the hardware, annotation scheme and privacy rules before recording, and write acceptance criteria for the pilot — share of time with hands in frame, sensor sync, blur quality, label agreement. The pilot is where camera mounts and task lists can still change cheaply. Ownership of the footage, permitted use and any exclusivity should be settled in the license before the first session, because they shape what workers are asked to consent to.
What good data looks like
- Camera intrinsics, mount position, frame rate and exposure settings documented for every session, with IMU or other sensor streams time-synchronized to the video where agreed.
- Hands and the work surface in frame for most of each task, checked on pilot recordings before collection scales up.
- Each recording linked to a task definition, such as a work order, procedure or job type, with step segments and a success, failure or rework outcome.
- Variation planned across workers, sites, tools, materials, lighting and clutter, and reported per program.
- Written consent from each recorded worker, permission from each site owner, and a documented process for blurring faces and masking screens, documents and customer property.
- Existing records with photos tied to the work-order step they document, and CAD files cleared for ownership and export controls.
Questions buyers ask
Can I license existing first-person video of skilled trades?
Rarely. Few businesses keep head-mounted video of their own work, and general CCTV footage or arbitrary photos do not qualify. SourceX treats egocentric video as a new collection program: footage is recorded for your order at partner businesses whose workers agree to take part. The other robotics data on this page, from work orders and quality records to site photos and CAD files, is licensed from existing company archives.
How does an egocentric video collection program work?
It runs in stages. Your spec sets the tasks, regions, hardware, annotation scheme and permitted use. SourceX then looks for partner businesses able to meet the program's conditions, for example on team size and region, and settles consent, site access and safety with each one. A short pilot follows, reviewed against acceptance criteria, before full recording and annotation. Whether a program goes ahead depends on businesses agreeing to take part.
Can human video train a robot without robot demonstrations?
Not on its own. Human video contains no robot joint states, gripper commands or forces, and human hands move differently from most grippers. Teams use it to pretrain visual representations, learn task structure and object affordances, and retarget hand motion, then fine-tune on demonstrations or rollouts recorded on their own robots. Calibrated cameras, synchronized IMU data and step labels make the footage more useful for each of these.
Why combine video with maintenance or quality records?
Video shows how a task was physically done; records show what the task was, why it was needed and whether it worked. Work orders, procedures and inspection results supply goal descriptions, step structure and outcome labels for the footage, and give planning and language models the task context that video alone does not carry.
Can CAD files be used to build simulation assets?
Yes, where the license permits it. Native CAD gives exact geometry, assembly constraints and materials for the parts and products a robot handles, which can be converted into meshes and collision models for simulation and grasp planning. Because designs often belong to the manufacturer's customers and some are export-controlled, CAD is provided on request, once ownership has been reviewed and each file has been screened.
Tell us what you are building
Describe the model or agent, the tasks it must handle, and the volume, format and permitted use you need. SourceX will match it to partner data.
Updated 3 October 2026.