Short answer
Hand tool use demonstrations show people selecting, gripping and working screwdrivers, wrenches, pliers and hammers on real fasteners and parts. Video, hand pose and tool pose can support tool-use perception and step understanding. They record neither the torque a wrench applied nor how a fastener resisted, so contact-rich tool control needs instrumented tools or robot data. Repair shops, training providers and commissioned recording are the main routes.
The buyer's problem
Maintenance, assembly and installation all run on hand tools, which makes tool use a core skill family for humanoid programs aimed at those jobs. Business footage of this work is usually recorded for another purpose: the tool tip disappears behind the hand, nobody noted which fastener it was, and the tightening force was never measured. A buyer has to separate footage that shows the skill from footage that merely shows someone nearby.
This page covers hand-operated tools. Drills, drivers and grinders are in power tool use demonstrations, whole repair jobs in repair video demonstrations, and finding tools in a drawer in tool identification image data.
Task anatomy
| Phase | What happens | Common failures | Data that helps |
|---|---|---|---|
| Select | Match tool type and size to the fastener | Wrong size; wrong drive type | Tool and fastener IDs on every clip |
| Acquire and grip | Pick up the tool and take a grip suited to the force needed | Poor grip; forced regrip | Egocentric and wrist views; grasp-type labels |
| Engage | Seat the tip or jaws on the fastener or part | Driver slipping out of the head (cam-out); misalignment; rounded head | Close view of tip and fastener; tool and hand pose |
| Apply | Turn, lever, squeeze or strike | Slip; over- or under-tightening; stripped thread | Instrumented-tool torque; otherwise an outcome label only |
| Reposition | Ratchet back, regrip, brace with the other hand | Losing engagement | Continuous video; bimanual labels |
| Verify and stow | Check seating or tightness; put the tool away | Skipped check; dropped tool | Step end markers and outcome labels |
Environment changes the task as much as the tool does. A fastener on a bench, on a vertical panel, overhead or in a recess calls for different postures, and access constraints are covered in tool use in tight and awkward spaces.
Tool families differ in what carries the skill
| Tool family | What carries the skill | What video shows | What it hides |
|---|---|---|---|
| Screwdrivers and hex keys | Axial pressure plus rotation | Rotation, hand posture | Pressure; early cam-out |
| Wrenches, sockets and ratchets | Torque through a lever arm, braced by the other hand | Arm and hand motion, stroke count | Torque applied; fastener resistance |
| Pliers and cutters | Grip force and leverage | Jaw closure, hand posture | Grip force; material resistance |
| Hammers and mallets | Swing path and impact | Trajectory, strike position | Impact force |
In each row the visible part is the motion, while the part that decides success is largely a force.
What published work shows
TACO is the closest purpose-built reference. It contains 2.5K bimanual motion sequences across tool-action-object combinations, recorded with 12 third-person views plus one egocentric view by combining multi-view sensing with optical motion capture, and annotates hand-object 3D meshes, 2D segmentation and behavior triplets (Liu et al., 2024). It shows what richly annotated tool use looks like when capture is designed around a motion-capture setup; it is not workplace footage.
HANDAL annotates tools as objects, with category-level pose and scale and handle affordance segmentation (Guo et al., 2023). It tells a model where the handle is, not how to use it.
Ego-Exo4D recorded skilled activities, bike repair among them, from egocentric and several exocentric cameras with expert commentary, and includes proficiency estimation among its benchmarks (Grauman et al., 2023). The lesson for buyers is the combination of viewpoints and spoken expertise.
Force is where video stops; when video is not enough lists the tasks affected. DexCap's portable hand capture tracks motion but records no force (Wang et al., 2024), whereas RH20T logs force with each of its contact-rich robot sequences and pairs each with a human demonstration video (Fang et al., 2023). Buyers typically need that pairing: human demonstrations for task structure and robot data for contact. For the general argument, see what human demonstration video can and cannot do.
Labeling grips
A published taxonomy gives annotators a shared vocabulary: it identifies 33 grasp types, reducible to 17 general grasps when object shape and size are ignored (Feix et al., 2016). It covers static, stable one-handed grasps, so the regrips, support hands and sliding grips of real tool use need extra labels. A humanoid hand may also realize only a subset of human grasps, which is why tool affordances and grasp annotations matters as a separate layer.
Where demonstrations come from
| Source | What may exist | Strength | Watch for |
|---|---|---|---|
| Repair shops and dealer service departments | Shop cameras, technician body cameras, training clips | Real fasteners and real failures | Tool tip hidden; customers' property in frame; uneven framing |
| Training and vocational providers | Instructional demonstrations and practical-assessment recordings | Clean, repeated, often narrated tasks | Staged happy paths; course copyright; see vocational training and practical-assessment recordings |
| Commissioned recording | Sessions designed around chosen tools, fasteners and cameras | Coverage, labels and consent designed in | Cost, site access, demonstrator recruitment |
| Instrumented sessions | Torque or force logged during human work | The missing signal | Specialist capture; see instrumented tools |
Existing footage is cheaper but was framed for another purpose; existing footage vs commissioned recording covers the trade-off. Camera placement guidance is in the head-mounted camera capture specification.
Illustrative capture specification
# Illustrative only: synthetic ids; every parameter is the buyer's to set
session: handtool_bench_017
operators: operator_A12, operator_B07, operator_C31 # consented; skill levels recorded
workpieces: test_board_T4 (hex, Phillips, slotted fasteners in several sizes), panel_P2
tools: tool_inst_0417 (13 mm ring wrench), tool_inst_0088 (ratchet, 3/8 drive)
postures: [bench_level, vertical_panel, overhead, recessed]
cameras:
head: egocentric, headband mount
wrist: left and right
side: one fixed view focused on the tool tip
sync: shared clock; offset measured at start and end of each session
scripted_variants: [correct_size, deliberately_wrong_size, seized_fastener, dropped_tool_recovery]
annotations: tool_id, fastener_id, step_segments, grasp_type, hand_visibility,
outcome (seated | tightened | slipped | rounded | abandoned), regrip_count
optional: torque-logging tool, units and sampling rate stated, clock-synced to video
excluded: robot actions, joint states, grip force (unless instrumented)
Requirements depend on architecture, tool family and embodiment; there is no universal volume that suffices. How many demonstrations you need summarizes what published studies used.
What this data cannot support
- Applied torque and grip force. Neither is recorded in video; both need an instrumented tool or robot data.
- Feel. Detecting that a thread is cross-threading or a fastener is about to strip is largely tactile.
- Gloved or occluded hands. Work gloves and the part itself often hide the grip; see gloved hands and PPE in workplace video.
- Human-to-humanoid hand transfer. Human hands and a humanoid's hands differ in degrees of freedom, strength and reach; see the human-to-humanoid embodiment gap.
- Rare cases. Seized, corroded or damaged fasteners are unusual in routine footage; see difficult and failed repairs.
Expect preparation work: synchronization and calibration, tool pose tracking, 3D hand pose and hand-object interaction labels.
Quality and acceptance criteria
- Visibility. The share of engage and apply frames in which tool tip and fastener are both visible is reported per clip; the threshold is yours.
- Identification. Every clip carries tool and fastener IDs.
- Failures retained. Slips, corrections and abandoned attempts are kept and labeled.
- Condition coverage. Counts by posture, access, fastener size, tool family and finish.
- Operator and part splits. Held-out sets are separated by operator and by fastener or part type, per training vs evaluation datasets.
- Synchronization. Measured offsets between cameras, and between video and any torque log, are documented.
- Torque data. If present, units, calibration and sampling rate are stated.
- Annotation agreement. Step boundaries and outcomes are double-labeled on a sample.
Questions to ask suppliers
- Can you show clips in which the tool tip and fastener stay visible through engagement?
- Which tool families, sizes and fastener types appear, and are they identified per clip?
- Were any recordings staged, re-enacted or edited for training or marketing?
- Are slips, stripped fasteners and corrections kept in?
- Which camera positions and mounts were used, and do they vary within the archive?
- Is any torque, force or tightening data logged, and how is it synchronized?
- Who appears in frame or owns the work shown: employees, customers, vehicles?
- Who owns the footage: you, a training vendor or a tool manufacturer?
Permissions and provenance
Hands-on work involves people, customers' property and often a manufacturer's branded tools or course content. Consent and bystander questions are covered in worker consent and identifiable people, and content owned by others in third-party content in workplace recordings. Ask whether participants were employees, trainees or students, and who gave notice to each. Redaction does not settle whether footage may be licensed. This is not legal advice.
Discuss your requirements
If you need hand tool footage, describe the tools, tasks, camera views and permitted uses to SourceX. SourceX investigates whether US businesses hold suitable recordings, checks the data and the supplier's licensing permissions, and coordinates the license and delivery. If no existing footage fits, SourceX can scope a commissioned collection. SourceX's role is sourcing, rights review, licensing and delivery coordination; annotation and specialist capture partners can be arranged case by case, and a request does not guarantee a match.
Start from the maintenance, repair and tool-use hub or the humanoid training data guide. The catalog entry for first-person video of skilled manual work and the page on egocentric video of manual work describe the related record type.
Describe the workplace demonstration data your robotics project needs
Sources
- TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understanding — purpose-built, motion-captured tool-use sequences.
- HANDAL: A Dataset of Real-World Manipulable Object Categories with Pose Annotations, Affordances, and Reconstructions — tool pose and handle affordance annotations.
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives — multi-view skilled activity, including repair, with expert commentary.
- RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot — force logged with contact-rich robot sequences.
- DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation — hand capture without force sensing.
- The GRASP Taxonomy of Human Grasp Types — grasp vocabulary and its limits.