Short answer
Reward and progress models can be pretrained on human task video without robot action labels, as published work on value-implicit pretraining shows. Useful video shows complete tasks from start to finish, with clear goal states and enough variety in objects and scenes. It gives a visual notion of progress, but a robot still needs its own data to act on that signal.
The buyer's problem
Reward is the bottleneck in many manipulation programs. Hand-written rewards are brittle, success detectors need labeled failures, and robot trials to collect those labels are slow. Human video offers a different supply: many people completing many tasks, with the goal state visible at the end.
A reward or progress model asks a particular question of the footage: given a frame, how close is this to done? That is not the same question as "what actions did the person take" or "what are the steps". It changes what you curate. For the broader trade-off between using video to pretrain and using it to imitate, see demonstrations for imitation vs pretraining.
What published work used
VIP pretrained on Ego4D human video with an action-free, self-supervised objective that learns a goal-conditioned value function. The authors report that it provides a dense visual reward without task-specific fine-tuning and supports few-shot offline reinforcement learning on a real robot with as few as 20 trajectories (Ma et al., 2022). The key property is that the pretraining needed no robot actions.
R3M, from the same era, pretrained a visual representation on Ego4D with time-contrastive and video-language alignment objectives. It reported more than 20% higher success than training from scratch across 12 simulated tasks, and a Franka Panda learned tasks in a real apartment from 20 demonstrations (Nair et al., 2022). It is a representation rather than a reward model, but its time-contrastive objective uses temporal structure that progress estimation also depends on.
Success and progress signals matter beyond reinforcement learning. Inner Monologue shows language-model planners replanning from closed-loop feedback, with success detection and scene description among its sources (Huang et al., 2022). A model that recognizes completion supports that kind of loop.
These studies used large general-purpose human video. They do not show how much workplace video a particular architecture needs, and they were evaluated on their own robot setups and tasks.
What makes video suitable
| Property | Why a reward or progress model cares | How to check a sample |
|---|---|---|
| Complete task episodes | Progress needs a start, middle and end | Count clips that begin before the first step and end after completion |
| Visible goal state | The model learns what "done" looks like | Check that the final frames show the result clearly |
| Temporal continuity | Time-contrastive and value objectives use ordering | Verify timestamps and absence of unmarked cuts |
| Multiple ways to finish | Avoids learning one path as the only path | Look for variation in order and technique |
| Failures and partial attempts | Teaches non-completion | Ask for outcome labels; see success and outcome labels |
| Variety in scene and object | Supports generalization of the visual signal | Counts by site, object and operator |
| Camera consistency | Reduces nuisance variation | Camera type and mount per clip |
| Language | Enables goal-conditioning from text | Captions, narration or matching procedures |
Edited footage is often the weak point. Training videos cut away from slow steps, jump between scenes and show only successful outcomes, which can teach a model that progress is quick and always ends well.
Supplier types and routes
Corporate training libraries. Learning and development teams hold step-by-step videos of tasks, often with scripts and captions. They are usually clean and well lit, and cover goal states. They are also staged, edited and sometimes vendor-produced or featuring actors; see corporate training video archives.
Operating businesses with recorded work. Process recordings can include real variation and failures but tend to lack clean boundaries and labels.
Commissioned recording. If you need controlled start and end states with failures on purpose, commission recording. That belongs to a different sourcing route; compare in existing footage vs commissioned recording.
Reward models versus latent-action pretraining
Both can learn from video without robot actions, but they ask for different curation. Latent-action pretraining favors volume, diversity and clean frame-to-frame continuity; see sourcing unlabeled video for latent-action pretraining. Reward and progress models favor complete episodes with identifiable endings and, ideally, outcome labels. A single archive can serve both, but a purchase scoped for one may leave gaps for the other.
Preparation the footage will need
- Selection. Inventory the library, then keep clips that show full tasks with visible outcomes.
- Segmentation and captions. Mark task starts and ends, and, if you will use step-level progress, step boundaries; see step segmentation annotation.
- Outcome labels. Add completed, failed, abandoned or corrected flags.
- Redaction. Remove faces, screens and logos that are not needed; check that redaction does not hide the goal state.
- Rights check per video. Training libraries mix employees, actors and third-party material, so rights must be confirmed clip by clip or by collection.
What this data cannot do
Progress and reward models trained from human video describe human-visible progress. They do not provide robot actions, forces or joint states, so they cannot train control on their own. The visual gap between a human hand and a robot hand can make a learned reward unreliable on robot footage; ask whether your evaluation includes robot observations. Rewards learned from video can be exploited by a policy that produces goal-looking frames without completing the task, so many teams combine them with success detectors or human checks. Held-out evaluation should be by site, operator or task, not by clip. See workplace video for task planning for the adjacent planning use.
Illustrative package for reward and progress training
# Illustrative only - synthetic values
package: reward_progress_video_v0
clips: complete task episodes, 20 s to 6 min, original frame rate kept
labels:
task_name: from supplier procedure titles
outcome: [completed, failed, abandoned, corrected]
progress_markers: optional step boundaries at 3 granularity levels
goal_frame_index: annotator-marked first frame where goal state is reached
language: one instruction sentence per episode
metadata: [site_id, camera_type, mount, edited_flag, actor_flag]
excluded: [robot actions, forces]
splits: held out by site_id for evaluation
Questions to ask suppliers
- Are clips complete task episodes, or edited highlights with cuts?
- Does each clip show the goal state clearly at the end?
- Are failed or partial attempts included, and are they labeled?
- Were any actors used, and were scenes staged for the camera?
- How many tasks, sites, camera types and people are represented?
- Are captions, scripts or procedure IDs available, and who wrote them?
- Is any third-party or vendor-produced content included, and who holds those rights?
- What have you redacted, and can I review samples at full resolution?
- Can you identify which clips are edited and where cuts occur?
Discuss your requirements
If you are assembling video for reward or progress models, describe the tasks, outcome labels and permitted uses to SourceX. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates license and delivery. SourceX reviews the requirement and follows up about potential sources; a request does not guarantee a match. For wider navigation, see the workplace task demonstration hub and the humanoid training data guide. The catalog entry on training materials and LMS content is related.
Discuss your humanoid training-data requirements with SourceX
Sources
- VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training — action-free pretraining on human video for dense visual reward.
- R3M: A Universal Visual Representation for Robot Manipulation — human-video representation pretraining with temporal objectives.
- Inner Monologue: Embodied Reasoning through Planning with Language Models — success detection as feedback to language-model planners.