Short answer
An action space defines what each recorded action means: joint targets or end-effector poses, absolute values or deltas, positions, velocities or torques, in a stated reference frame. Control frequency is the rate at which actions are issued and logged. Specify both for your target robot and policy before collecting data, because they decide whether the data can be used, converted or resampled later.
The decision you cannot postpone
Action representation is fixed when the data is recorded. One teleoperation session can yield joint targets, end-effector poses or the operator's device pose, but if the provider logs only one of them, the others may be lossy or impossible to derive. A policy trained in one representation cannot be handed data in another without a conversion, and conversions have costs.
So the specification belongs in the first request, before a provider builds a rig. These are robot-specific choices. They come from robots, teleoperation rigs, handheld capture devices or instrumented setups, not from business archives or workplace video, which contain no actions at all; see what human demonstration video can and cannot do.
Six dimensions of an action space
| Dimension | Common options | What it means for the data |
|---|---|---|
| Space | Joint targets; end-effector (Cartesian) pose targets; whole-body kinematic pose targets | Joint actions are tied to one robot's joint layout. End-effector actions transfer more easily between arms but need inverse kinematics at deployment. OmniH2O used kinematic pose as a universal control interface for whole-body humanoid teleoperation (He et al., 2024) |
| Reference | Absolute or delta; frame: robot base, world, tool, camera | A delta needs its reference defined (previous command or measured state); every pose needs its frame |
| Quantity | Position, velocity, torque or current | Determines what the low-level controller must do with the value |
| Hand and gripper | Open/close, width, joint targets for a multi-DoF hand, fingertip targets | Dexterous hands multiply the dimensionality and the calibration burden |
| Base and legs | Base velocity, footstep or contact targets, leg joint targets | A lower-level whole-body controller may sit beneath the policy, in which case the policy's action is a target for that controller |
| Model-side form | Continuous vectors, discretized tokens, action chunks, sampled sequences | Decides which rates and smoothness the data must have |
The model-side form varies across published work. RT-2 expresses robot actions as text tokens (Brohan et al., 2023). ACT introduced Action Chunking with Transformers (Zhao et al., 2023), π0 uses a flow-matching action expert that predicts 50-step action chunks (Black et al., 2024), Diffusion Policy models the policy as conditional denoising diffusion (Chi et al., 2023), and GR00T N1 pairs a vision-language model with a diffusion-transformer action module (NVIDIA, 2025). Octo can be finetuned to new observation and action spaces (Octo Model Team, 2024), which shows that adapting is possible; it does not remove the need for data that is consistently labeled in the space you adapt to.
What can be converted later
| Conversion | Works if the dataset records | What is lost or at risk |
|---|---|---|
| Joint targets to end-effector pose | A kinematic model and calibration; forward kinematics | Pose accuracy is limited by model and calibration error. A URDF-style description gives links connected by joints, tree structures only (ROS, n.d.) |
| End-effector pose to joint targets | The kinematic model and joint limits; inverse kinematics | For redundant arms the answer is not unique and depends on the solver; logged joint states are safer |
| Absolute to delta | Consistent frames and a stated reference | Little; the reference must be defined |
| Delta to absolute | A starting pose and every step | Errors accumulate; anchor to measured state |
| Commanded to measured, or the reverse | Both logged | If only one is logged, tracking error, latency and contact effects cannot be recovered exactly |
| Higher rate to lower rate | Timestamps | Straightforward if filtered first; keep discrete events such as gripper closing aligned |
| Lower rate to higher rate | Nothing; the missing steps were never recorded | Interpolation adds no information and blurs events |
| One embodiment to another | Retargeting or a shared representation | Different joints, limits and hands; see cross-embodiment robot datasets |
Control frequency
Three rates are often conflated: the sensors' rates, the rate at which actions are issued and logged, and the rate of the low-level controller, which is usually higher. Specify each. Published systems cover a wide span. DROID teleoperated a Franka Panda arm with a Meta Quest 2 controller at 15 Hz (Khazatsky et al., 2024), and RT-1 ran a 35-million-parameter model at 3 Hz (Brohan et al., 2022). Those are what those systems used, not recommended values.
Delivery containers store the rate in their schema; see robot learning data formats. The rate sets the time horizon of everything built on steps. Illustrative arithmetic: a 50-step action chunk spans 1 second at 50 Hz and about 3.3 seconds at 15 Hz. If you plan chunked policies, the logged rate and chunk length together set how far ahead each prediction reaches.
What the right rate is depends on your architecture, task dynamics, robot bandwidth, inference time and latency budget. Slow transfers and contact-rich insertions place different demands on it. Higher is not automatically better: a rate you cannot run at deployment is only useful if you can decimate to it cleanly, for example by logging at an integer multiple of the policy rate. Log cameras and other sensors at their native rates with timestamps, and resample in training; the timing requirements are in sensor synchronization requirements.
Humanoid-specific points
- Which joints are in the action vector. Arms, hands, torso, neck and legs may be commanded together or by separate controllers. State which are included and which a lower-level controller handles.
- Hands. A grasp primitive, a gripper width and a full dexterous-hand joint vector are three different data products.
- Whole-body teleoperation. Operator pose is retargeted to the robot. Log both the operator's device pose and the command that reached the robot, so you can revisit the retargeting; see teleoperation data collection requirements.
- Mixed data. GR00T N1 describes a data pyramid with web data and human egocentric video at the base, synthetic data in the middle and real-robot teleoperation at the top, and it labels action-less video with a latent-action codebook and inverse-dynamics pseudo-actions (NVIDIA, 2025). Those labels are inferred. The real-robot tier is where actions are defined, so its action space needs to be exact.
Illustrative starting points
These are examples of what a buyer might ask for given a model family. They are not requirements.
| If your policy | Ask the collector to log |
|---|---|
| Predicts action chunks | Absolute targets and measured state at the rate you will run, with low timestamp jitter, so deltas and chunks can be rebuilt |
| Tokenizes actions | Continuous commanded values with ranges and units, so bin choices stay open |
| Samples action sequences by diffusion or flow | Smooth, uniformly sampled sequences with no dropped steps |
| Commands a whole-body controller | Targets in that controller's interface, measured joint and base state, and the controller version |
| Pretrains across embodiments | End-effector actions with an embodiment descriptor, plus native joint-space data for your target robot |
The general rule is to log the superset and derive later: commanded targets in the operator-interface space, commanded targets in the robot's space, and measured state, all timestamped.
# Illustrative only — synthetic values; every item is a decision the buyer sets
action_spec: ACT_SPEC_2026_001
log_rate_hz: 30 # integer multiple of the planned policy rate
commanded:
operator_device_pose: {frame: operator_world, unit: [m, quat]}
robot_command: {space: joint_position_absolute, unit: rad, dims: 14, order: [l_shoulder_1, "…", r_wrist_3]}
hand_command: {space: joint_position_absolute, unit: rad, dims: 12}
measured:
joint_state: {position: rad, velocity: rad_per_s}
end_effector_pose: {frame: robot_base}
conventions:
pairing: "command[t] issued after observation[t]"
delta_definition: "command[t] minus measured_state[t]"
timestamps: single shared clock; jitter reported per episode
robot_model: humanoid_X_r3.urdf # kinematic description supplied
What this guidance cannot settle
- A universal action space or rate. None exists; requirements depend on architecture, task, embodiment and intended use.
- Missing actions. Actions cannot be recovered from video that never recorded them; inferred labels are a modeling choice, not a recorded signal.
- Conversion accuracy. Every conversion inherits the calibration and kinematic model error of the rig that recorded the data.
- Quality. A well-specified action space says nothing about whether the demonstrations were good; see robot trajectory dataset specification for the full episode fields, including operator IDs and outcomes.
Questions to ask providers
- Which action spaces can the rig log natively, and can it log more than one at once?
- Is the command logged, the measured state, or both, at the same step?
- At what rate are actions logged, what is the measured rate, and how is jitter reported?
- How is a delta defined: against the previous command or the measured state?
- Which reference frames are used, and are calibration and the kinematic description delivered with the data?
- What is the retargeting method between the operator interface and the robot, and can the operator pose be logged too?
- Which joints are in the action vector, and what controller handles the rest?
- What changes between robot revisions or controller versions, and are they recorded per episode?
Discuss your requirements
If you already know your action space and rate, put them in the first message; if you do not, say which model family you plan to train. Describe the robot data you need to SourceX and it will investigate whether US businesses hold suitable material. If no existing asset fits, SourceX can scope a commissioned collection of new recordings. SourceX reviews the requirement and follows up about potential sources; a request does not guarantee a match, and nothing is contracted until a supplier agrees. Related reading: the procurement hub, the humanoid training data guide and the catalog's robotics and embodied AI use case.
Discuss your humanoid training-data requirements with SourceX
Sources
- GR00T N1: An Open Foundation Model for Generalist Humanoid Robots — diffusion-transformer action module, pseudo-actions and the data pyramid.
- π0: A Vision-Language-Action Flow Model for General Robot Control — flow-matching action expert and 50-step action chunks.
- Diffusion Policy: Visuomotor Policy Learning via Action Diffusion — policy as conditional denoising diffusion.
- Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware — action chunking with transformers.
- DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset — Quest 2 teleoperation at 15 Hz.
- RT-1: Robotics Transformer for Real-World Control at Scale — a model running at 3 Hz.
- RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control — actions expressed as text tokens.
- Octo: An Open-Source Generalist Robot Policy — finetuning to new observation and action spaces.
- OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning — kinematic pose as a control interface.
- URDF (Unified Robot Description Format) — links-and-joints robot description (partial verification).