Object Data Requirements for Multi-Finger Hands vs Parallel Grippers

Why the end effector changes the data order

Buyers often write one object-data requirement for "the robot" and discover later that the labels fit only one kind of hand. A parallel-jaw gripper makes a narrow set of decisions: where to place two pads, at what angle, with what opening. A humanoid hand with several fingers makes many more: which grasp family to use, which fingers touch which surface, and how to re-orient the object once it is held.

Object recognition data is the shared starting point for both. The differences begin with grasp-relevant labels, which is where object recognition data and manipulation data part ways. This page compares what each end effector needs from the object side.

What published datasets show for each case

ACRONYM is the clearest gripper example. It provides 17.7M simulated parallel-jaw grasps across 8,872 objects in 262 categories, with each grasp outcome labeled by physics simulation (Eppner et al., 2020). The labels are poses and outcomes for two-finger closing; they say nothing about how a five-finger hand would wrap the same object.

DexYCB sits on the hand side. It captured 582K RGB-D frames from 1,000 sequences in which 10 subjects grasped 20 YCB-Video objects, recorded from 8 views, and it benchmarks 2D object and keypoint detection, 6D object pose estimation and 3D hand pose estimation; it also evaluates robot grasp generation for human-to-robot handover (Chao et al., 2021). That is human hand data on a small object set, useful for hand-object understanding, not a ready-made library of grasps for a robot hand.

For object parts, HANDAL provides category-level pose and scale plus handle affordance segmentation for manipulable hardware and kitchen tools (Guo et al., 2023). Part labels like "handle" matter much more to a hand than to a gripper that can pinch almost anywhere.

Comparison: what to specify

Using a grasp taxonomy as a label vocabulary

Hand grasps need a shared vocabulary, or annotators will describe the same grasp three ways. The GRASP taxonomy merges earlier taxonomies into 33 grasp types, reducible to 17 general grasps if object shape and size are ignored, organized by opposition type, virtual finger assignment, power/precision/intermediate character and thumb position (Feix et al., 2016). It covers only static, stable, one-handed grasps.

Those limits matter. A taxonomy of human grasps describes human hands; a robot hand with fewer fingers or different joint limits cannot execute many of them. Treat the taxonomy as a labeling vocabulary and a starting point for mapping, not a promise that each class is available to your hand. The embodiment questions are covered in the human-to-humanoid embodiment gap.

Illustrative label specification for a hand-focused object set

# Illustrative only - values are synthetic examples a buyer would replace
object_record:
  object_id: obj_0412
  identity: {sku: SKU-000412, category: spray_bottle}
  geometry: {mesh: obj_0412.glb, scale_m: [0.07, 0.07, 0.24]}
  physical: {mass_g: 410, fill_state: half, friction_note: "measured on pad_A, dry"}
  parts: [trigger, neck, body, base]
grasp_label:
  source: human_demo | teleop | sim
  grasp_type: "tripod"            # from the buyer's chosen taxonomy
  contacts: {thumb: neck, index: trigger, palm: none}
  hand_model: hand_spec_X         # which hand the label is valid for
  outcome: held_5s | slipped | dropped
  task_context: "lift and pour"
validity_note: "labels apply only to hand_spec_X; retargeting required for other hands"

For gripper work the same record shrinks to an approach pose, width and outcome. For product-level labels, read grasp annotations for products and affordance and part labels. Hand tools raise their own questions; see tool affordances and grasp annotations.

Limitations: what object data cannot do for either hand

Simulated grasp labels depend on the physics model. Simulation outcome labels for rigid objects do not tell you how a soft, filled or slippery item behaves in a real hand, so real-world trials still decide success. Human grasp annotations record what a person's hand did; they do not provide robot joint commands or the force a robot would apply. Neither dataset type contains tactile feedback unless it was recorded.

Object data also cannot replace interaction data for a hand. Whether a particular grasp is stable on your hardware needs robot or teleoperated trials, with calibrated sensors and hand state recorded. Object catalogs and scans help the perception and planning layers only. Plan the physical measurements separately; see physical property annotations.

Deciding what to buy

1. Fix the end effector, including finger count, joint limits and any tactile sensing.
2. If you use a gripper, prioritize meshes, scale accuracy and simulated grasps, then validate on real objects.
3. If you use a hand, add part labels, a grasp vocabulary and a plan to collect hand-state data, whether by teleoperation, handheld capture or instrumented rigs.
4. Keep an evaluation set of real objects the model never saw; see novel object generalization evaluation.

A worked example: one object, two end effectors

Take a half-full spray bottle. For a parallel gripper, a useful label is a handful of antipodal pad poses around the body with an opening width, plus a note of which regions to avoid. For a multi-finger hand, the same bottle needs a different record: a tripod or power grasp on the body for lifting, a different grasp that places a finger on the trigger for use, and a flag that the fluid level shifts the center of mass. Neither record tells you how the bottle behaves when squeezed. That has to come from handling data, and it is why object labels are an input to grasp planning, not a substitute for interaction data.

Questions to ask suppliers

1. For which specific end effector are the grasp labels valid, and how was that checked?
2. Were grasps generated in simulation, annotated by humans, or recorded on a robot?
3. Which grasp taxonomy or vocabulary was used, and how consistent were annotators?
4. Do labels include per-finger contact regions, or only a hand pose?
5. How are object scale and the size of the hand represented?
6. Which physical properties were measured on the real objects, and with what instruments?
7. What is the grasp success definition (held for how long, under what disturbance)?
8. Which objects overlap with a public benchmark object set, if any, and which are new?
9. Are failed and unstable grasps included with labels?
10. What are the redistribution and derivative-work terms on the 3D models and images?

Discuss your requirements

If you can name the hand type and the objects it must handle, discuss your humanoid training-data requirements with SourceX. SourceX reviews the requirement and follows up about potential sources among US businesses. Annotation and specialist capture partners can be arranged case by case, and a request does not guarantee a match.

See the humanoid training data guide, the product and object data hub, and the robotics and embodied AI overview. For labeling work, see expert annotations and labels.

Discuss your humanoid training-data requirements with SourceX

Sources

• DexYCB: A Benchmark for Capturing Hand Grasping of Objects — human hand grasp capture and benchmark tasks.
• ACRONYM: A Large-Scale Grasp Dataset Based on Simulation — simulated parallel-jaw grasp labels.
• The GRASP Taxonomy of Human Grasp Types — grasp vocabulary and its limits.
• HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions — handle affordance and category-level pose labels.