Skip to content

Dexterous Hands

Overview: Why Dexterous Hands Are Needed

The human hand possesses approximately 27 bones, 20 degrees of freedom, and thousands of tactile receptors, making it nature's most sophisticated manipulation tool. Dexterous hands attempt to replicate this capability, enabling robots to perform fine manipulation — tightening screws, rotating objects, using tools, folding fabric, and more.

Compared to simple parallel grippers, dexterous hands offer:

  • In-hand manipulation: Readjust grasp pose without putting the object down
  • Multi-finger coordination: Simultaneously manipulate multiple objects or execute complex multi-contact tasks
  • Tool use: Hold and use various tools like a human hand
  • Versatility: One hand adapts to all grasping and manipulation scenarios

Core Challenges

  • High-dimensional control: 16-20 DOF continuous control space
  • Contact modeling: Multi-finger, multi-point contact mechanics are extremely complex
  • Sensing: High-density tactile feedback required
  • Underactuation: Most hands are underactuated (DOF > number of actuators), requiring clever transmission design
  • Hardware reliability: Precision transmission mechanisms are prone to wear

Dexterous Hand Platform Comparison

Platform DOF Drive Method Features Price Typical Use
Shadow Dexterous Hand 20 Pneumatic/electric Closest to human hand, OpenAI Rubik's cube experiment >$100K High-end research
Allegro Hand (Wonik) 16 Electric torque control Widely used in research, direct torque control ~$15K RL research
LEAP Hand (CMU) 16 Electric (Dynamixel) 3D printed open-source design, ~$2K ~$2K Low-cost RL
Ability Hand (PSYONIC) 6 Electric Prosthetic-grade, tactile feedback Prosthetic pricing Prosthetics/research
Inspire Hand 6-12 Electric Low cost, multiple models, commonly used in Chinese humanoids ~$1-5K Humanoid integration
Agilex DexHand 12 Electric Paired with Agilex mobile platforms ~$5K Mobile manipulation
DEX-EE (Sanctuary AI) 20+ Electric For Phoenix humanoid Not for sale Internal
Tesla Optimus Hand 11/hand Electric Mass-production oriented, matches human hand size Not for sale Humanoid integration

Shadow Dexterous Hand

The Shadow Hand is the "gold standard" of dexterous hand research:

  • 24 joints, 20 degrees of freedom
  • Thumb has opposition movement
  • Pneumatic version has fast response but requires external air supply
  • Electric version is more suitable for lab environments
  • BioTac tactile sensor integration (optional)
  • OpenAI used it to solve a Rubik's cube in 2019

LEAP Hand

LEAP Hand from CMU dramatically lowers the barrier to dexterous hand research:

  • Design: Fully 3D printed structure + Dynamixel XC330 servos
  • Cost: ~$2,000 (compared to Allegro $15K, Shadow $100K+)
  • Open-source: Hardware design, URDF, control code all open-sourced
  • Compatible: Works with Isaac Gym / MuJoCo simulation
  • Performance: Validated in multiple in-hand manipulation RL experiments

Grasp Taxonomy

Cutkosky Grasp Classification

The Cutkosky (1989) grasp taxonomy is the foundational framework for dexterous manipulation:

graph TD
    A[Human Grasps] --> B[Power Grasp]
    A --> C[Precision Grasp]

    B --> B1[Cylindrical]
    B --> B2[Spherical]
    B --> B3[Hook]
    B --> B4[Palmar]

    C --> C1[Tip Pinch]
    C --> C2[Lateral Pinch]
    C --> C3[Tripod]
    C --> C4[Disk]

    B1 --> D[Tool Holding]
    C1 --> E[Precision Assembly]
    C3 --> F[Writing/Drawing]

Power grasp: Wraps the object using the palm and multiple fingers, providing maximum grip force and stability.

Precision grasp: Uses only fingertips for contact, providing higher dexterity but lower stability.

Contact Models

Point contact (with friction): The most commonly used contact model, assuming infinitesimal contact area:

\[ \mathbf{f}_c = [f_x, f_y, f_z]^T, \quad \sqrt{f_x^2 + f_y^2} \leq \mu f_z \]

Surface contact: Considers finite contact area, contact wrench includes torque components:

\[ \mathbf{w}_c = [f_x, f_y, f_z, \tau_x, \tau_y, \tau_z]^T \]

Soft finger contact: Allows torque about the normal direction, common in finger pad contacts:

\[ \mathbf{w}_c = [f_x, f_y, f_z, 0, 0, \tau_z]^T, \quad f_t^2 + \frac{\tau_z^2}{e^2} \leq \mu^2 f_z^2 \]

where \(e\) is the ellipsoidal friction parameter.

In-Hand Manipulation

Manipulation Primitives

In-hand manipulation can be decomposed into basic motions:

  • Finger gaiting: Fingers alternately make contact to achieve large-range rotation
  • Finger walking: Fingers slide on the object surface to change grasp position
  • Pivoting: Using gravity to rotate the object about a contact point
  • Rolling: Fingertips roll on the object surface for fine pose adjustment

In-Hand Reorientation Problem

Given an object's initial pose \(\mathbf{T}_0\) and goal pose \(\mathbf{T}_g\), manipulate the object from \(\mathbf{T}_0\) to \(\mathbf{T}_g\) through finger movements without putting it down.

Sources of difficulty: 1. Contact states are discrete (which fingers are in contact) 2. Mode switching between sliding/rolling/separation 3. Object may fall (stability constraints) 4. Planning in high-dimensional space

Tactile Sensing

Why Tactile Sensing Is Needed

Vision fails when occluded (fingers blocking the object), while tactile sensing can provide:

  • Contact force: Whether grip force is sufficient
  • Slip detection: Whether the object is about to fall
  • Surface features: Texture, hardness, temperature
  • Contact location: Precise contact region

Mainstream Tactile Sensors

Sensor Principle Resolution Features
GelSight (MIT) Elastomer deformation + camera ~30 um Ultra-high spatial resolution, geometry reconstruction
DIGIT (Meta) GelSight variant, compact design ~40 um Mountable on robot fingers
BioTac (SynTouch) Hydraulic + electrode array 19 taxels Multimodal (force, vibration, temperature)
ReSkin (Meta) Magnetoresistive sensing ~1 mm Film-type, low cost, replaceable
Taxim Simulation + GelSight - Tactile simulation framework

GelSight Working Principle

GelSight uses a transparent elastomer coated with reflective material. When an object presses the elastomer:

  1. Elastomer deformation reflects contact surface geometry
  2. Multi-color LEDs illuminate the inner surface of the elastomer
  3. Camera captures the deformed image
  4. 3D surface reconstructed via Photometric Stereo
\[ \mathbf{n}(x,y) = f(I_R(x,y), I_G(x,y), I_B(x,y)) \]

where \(I_R, I_G, I_B\) are the red, green, blue channel images, and \(\mathbf{n}\) is the surface normal.

Reinforcement Learning for Dexterous Manipulation

OpenAI Rubik's Cube Experiment (2019)

This is a milestone work in dexterous manipulation RL:

  • Goal: Use Shadow Hand to rotate a Rubik's cube to a target state
  • Training: PPO in simulation, large-scale domain randomization
  • Domain randomization parameters: Friction, object size, finger length, gravity, sensor noise, etc.
  • Automatic Domain Randomization (ADR): Automatically increases randomization range based on policy performance
  • Results: ~60% success rate, demonstrating the power of extreme domain randomization

Recent Work

timeline
    title Dexterous Manipulation RL Key Milestones
    2018 : DAPG<br/>Demonstration-accelerated RL
    2019 : OpenAI Rubik's Cube<br/>Shadow Hand + Large-scale DR
    2021 : DexMV<br/>Learning Dexterous Manipulation from Human Videos
    2022 : DexPoint<br/>Point Cloud Input Dexterous Manipulation
         : Hora<br/>In-hand Reorientation RL
    2023 : DexGraspNet<br/>Large-scale Dexterous Grasping Dataset
         : AnyTeleop<br/>Universal Dexterous Hand Teleoperation
    2024 : RotateIt<br/>Universal In-hand Rotation
         : Bunny-VisionPro<br/>VR Teleoperation Dexterous Hand

Key design patterns:

# Typical dexterous manipulation reward
reward = (
    w_pose * exp(-k * ||q_obj - q_target||)   # Object pose approaches target
    + w_reach * exp(-k * d(fingers, obj))       # Fingers approach object
    - w_drop * (obj_z < threshold)              # Object drop penalty
    - w_energy * sum(tau^2)                      # Energy penalty
    + w_bonus * success                          # Success bonus
)

Transmission and Drive Design

Drive Method Comparison

Direct drive: Motor directly drives the joint - Pros: Good backdrivability, precise force control - Cons: Low torque density, large size

Tendon-driven: Motor drives distal joints through cables (tendons) - Pros: Motor can be far from joints, finger segments can be thin - Cons: Tendon friction, elastic deformation, complex routing - Representatives: Shadow Hand, Inspire Hand

Linkage transmission: Transmits motion through linkage mechanisms - Pros: Good structural rigidity, no cable slack issues - Cons: Complex design, poor compactness - Representative: LEAP Hand

Underactuated vs fully actuated: - Fully actuated: Each DOF has an independent actuator, fully controllable - Underactuated: Fewer actuators than DOF, relies on mechanical coupling and elastic elements for adaptive grasping - Underactuation advantage: Simpler, more robust, cheaper

Hand-Eye Coordination

Dexterous manipulation requires tight coordination between vision (eye) and hand:

Eye-in-hand: Camera mounted on the hand - Precise observation of manipulation area - Field of view moves with the hand

Eye-to-hand: Camera fixed externally - Global field of view - Unaffected by hand motion - May be occluded by the hand

Practical systems typically combine both: global camera for localization + wrist camera for fine alignment + tactile sensors for contact feedback.

References

  • Bicchi, "Hands for Dexterous Manipulation and Robust Grasping", IEEE Trans. Robotics, 2000
  • OpenAI, "Solving Rubik's Cube with a Robot Hand", arXiv, 2019
  • Shaw et al., "LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning", RSS, 2023
  • Cutkosky, "On Grasp Choice, Grasp Models, and the Design of Hands for Manufacturing Tasks", IEEE Trans. Robotics, 1989

Related Notes:

评论 #