Skip to content

Sensors

Sensors are the foundation of robot perception. This article covers six major categories of robot sensors — vision, depth, LiDAR, inertial, force/torque, and tactile — discussing their principles, selection criteria, and technical specifications.

Sensor Classification Overview

Category Sensed Information Typical Frequency Typical Use
RGB Camera Color/texture 30-120 Hz Object detection, visual servoing
Depth Camera 3D depth map 30-90 Hz Obstacle avoidance, grasping
Event Camera Pixel-level brightness changes ~1 MHz equivalent High-speed motion, HDR
LiDAR 3D point cloud 10-20 Hz SLAM, navigation
IMU Acceleration/angular velocity 200-1000 Hz Attitude estimation, VIO
Force/Torque Sensor 6-axis wrench 100-8000 Hz Force control, assembly
Tactile Sensor Contact force/deformation 30-100 Hz Dexterous manipulation, slip detection

Vision Sensors

RGB Cameras

Commonly used industrial/research RGB cameras:

Camera Resolution Frame Rate Interface Features Price Range
Intel RealSense D435i (RGB) 1920x1080 30fps USB3 Built-in IMU ~$300
FLIR Blackfly S Up to 5MP 75fps (1.6MP) GigE/USB3 Industrial-grade, global shutter ~$500-800
Basler ace 2 Up to 24MP Variable GigE/USB3 Industrial automation standard ~$300-1000
Orbbec Femto Bolt (RGB) 3840x2160 30fps USB3 4K RGB ~$500

Selection Criteria:

  • Global shutter vs rolling shutter: Global shutter avoids image distortion during robot motion
  • Frame rate: Visual servoing requires >=60fps; object detection typically needs only 30fps
  • Lens: Choose FOV based on working distance (wide-angle for close range, narrow for far)
  • Synchronization: Multi-camera setups require hardware trigger synchronization

Stereo Camera

Stereo cameras compute depth through binocular disparity, fundamentally a passive depth measurement method.

Product Baseline Depth Range Features
Stereolabs ZED 2 120mm 0.3-20m Built-in VIO + AI
ZED Mini 63mm 0.1-15m Compact, suitable for drones
ZED X 120mm 0.3-20m IP67, industrial-grade
Multisense S27 270mm 0.5-10m Long baseline, high precision

Event Camera

Event cameras (also known as DVS, Dynamic Vision Sensor) asynchronously output pixel-level brightness change events.

Feature Value
Temporal resolution Microsecond-level (equivalent >1000fps)
Dynamic range >120dB (vs standard camera ~60dB)
Power consumption Extremely low (only changed pixels output)
Data volume Sparse event stream
Typical products iniVation DAVIS346, Prophesee EVK4

Suitable scenarios: High-speed grasping, drone obstacle avoidance, extreme lighting (welding, tunnels).

Depth Sensors

Mainstream Depth Camera Comparison

Product Measurement Principle Depth Range Resolution Frame Rate Outdoor Price
Intel RealSense D435i Active IR stereo 0.1-10m 1280x720 90fps Fair ~$300
Intel RealSense D455 Active IR stereo 0.6-6m 1280x720 90fps Good ~$350
Intel RealSense L515 LiDAR (ToF) 0.25-9m 1024x768 30fps Poor Discontinued
Stereolabs ZED 2 Passive stereo 0.3-20m 2208x1242 15fps Good ~$450
Orbbec Femto Bolt ToF (iToF) 0.25-5.46m 640x576 30fps Poor ~$500
Orbbec Gemini 2 Active IR stereo 0.15-10m 1280x800 30fps Fair ~$200
Azure Kinect DK ToF (iToF) 0.5-5.46m 640x576 30fps Poor Discontinued

Depth Measurement Principles

Principle Description Advantages Disadvantages
Structured light Project known patterns, analyze deformation High indoor accuracy Outdoor interference
Active IR stereo Project IR texture to assist matching Good balance Accuracy degrades at distance
ToF (Time-of-Flight) Measure light travel time High frame rate, good consistency Multi-path interference, low resolution
Passive stereo Pure binocular matching Works outdoors Fails in textureless areas

Selection Recommendations:

  • Indoor tabletop manipulation: RealSense D435i (classic choice) or Orbbec Gemini 2 (value)
  • Indoor navigation: RealSense D455 (wider baseline)
  • Outdoor: ZED 2 (passive stereo unaffected by sunlight)
  • High-precision short-range: Structured light solutions

LiDAR

Common LiDAR Comparison

Product Channels Range Accuracy Scanning Method Frequency Price
Velodyne VLP-16 16 100m +-3cm Mechanical rotation 5-20Hz ~$4,000
Ouster OS1-64 64 120m +-1.5cm Digital rotation 10-20Hz ~$6,000
Livox Mid-360 Equiv. N/A 40m +-2cm Non-repetitive scanning 10Hz ~$500
Livox HAP Equiv. N/A 150m +-3cm Non-repetitive scanning 10Hz ~$500
RPLIDAR A1 1 (2D) 12m <1% Mechanical rotation 5.5Hz ~$100
RPLIDAR S2 1 (2D) 30m +-3cm Mechanical rotation 10Hz ~$200
Intel RealSense L515 — 9m 5mm@1m Solid-state 30Hz Discontinued

Scanning Methods

Method Representative Advantages Disadvantages
Mechanical rotation Velodyne, Ouster 360-degree FOV, mature Limited lifespan, bulky
Solid-state (MEMS) Livox No moving parts, reliable Limited FOV
Non-repetitive scanning Livox Coverage increases with integration time Requires integration time
Flash — Ultra-fast, compact Short range

Selection Recommendations:

  • 2D navigation (AGV/service robots): RPLIDAR series (low cost)
  • 3D mapping/autonomous driving: Ouster OS1 or Velodyne VLP-16
  • Low-cost 3D SLAM: Livox Mid-360 (excellent value)

For more SLAM-related content, see SLAM.

IMU (Inertial Measurement Unit)

IMU Basics

An IMU contains a 3-axis accelerometer + 3-axis gyroscope (6-axis). High-end IMUs also include a 3-axis magnetometer (9-axis).

Parameter Description Typical Value (MEMS)
Gyroscope bias stability Drift at zero input 1-10 deg/hr
Accelerometer bias Offset at zero input 0.1-1 mg
Noise density (ARW) Angular random walk 0.1-0.5 deg/sqrt(hr)
Sampling rate Output frequency 200-8000 Hz
Range Measurement range +-250~2000 deg/s

Common IMUs

Product Grade Interface Features Price
MPU-6050 Consumer I2C Low cost, Arduino entry-level ~$3
BMI270 Consumer SPI/I2C Low power, smartphone-grade ~$5
ICM-42688-P Mid-range SPI Low noise, commonly used in drones ~$10
VN-100 Industrial UART/SPI Built-in AHRS filtering ~$500
ADIS16470 Industrial SPI High accuracy, low drift ~$300
KVH 1775 Tactical UART Fiber-optic gyroscope, extremely low drift ~$10,000

VIO (Visual-Inertial Odometry)

Fusing IMU with visual sensors forms VIO, one of the mainstream approaches for robot localization:

  • IMU provides high-frequency short-term pose (200-1000Hz) but drifts over time
  • Vision provides low-frequency absolute pose (30Hz) but cannot track in the short term
  • Complementary fusion achieves high-frequency, low-drift localization

Common VIO solutions: VINS-Mono/Fusion, ORB-SLAM3, Basalt, cuVSLAM (NVIDIA).

Force/Torque Sensors

6-Axis Force/Torque Sensors

6-axis F/T sensors measure forces in three directions (\(F_x, F_y, F_z\)) and torques in three directions (\(\tau_x, \tau_y, \tau_z\)).

Product Force Range Torque Range Resolution Frequency Price
ATI Gamma 65N 5Nm 1/64N 7kHz ~$5,000
ATI Mini45 145N 5Nm 1/16N 7kHz ~$4,000
ATI Nano17 12N 0.12Nm 1/160N 7kHz ~$6,000
OnRobot HEX-E 200N 6Nm 0.2N 1kHz ~$3,000
Robotiq FT 300 300N 30Nm 0.5N 100Hz ~$3,000
Bota SensONE 1500N 40Nm 0.3N 800Hz ~$2,000

Mounting Position: Typically installed between the robot arm end-effector flange and the gripper.

Typical Applications:

  • Force-controlled assembly: Pin insertion, screw tightening (force/torque threshold detection)
  • Collision detection: Trigger stop when unexpected forces are exceeded
  • Impedance control: Adjust position based on force feedback
  • Teleoperation force feedback: Transmit follower-side force information back to the operator

Tactile Sensors

Tactile sensors are key to achieving dexterous manipulation and have developed rapidly in recent years due to robot learning demands.

Mainstream Tactile Sensors

Product Principle Information Type Resolution Features Price
GelSight Elastomer + camera High-resolution contact geometry ~25um Academic classic DIY ~$200
DIGIT (Meta) Elastomer + camera Contact geometry + force estimation Medium Compact, open design ~$50 (DIY)
GelSight Mini Elastomer + camera 3D contact deformation ~25um Commercialized version ~$500
BioTac Multimodal Force + vibration + temperature 19 electrodes Biomimetic fingertip ~$5,000
ReSkin Magnetic 3-axis force Medium Thin film, replaceable ~$5
XELA uSkin Capacitive array 3-axis force distribution 4x4 taxel Commercialized ~$1,000

Vision-Tactile Sensors (GelSight Series)

Working principle of GelSight-type sensors:

  1. Elastomer (transparent silicone) surface coated with reflective material
  2. Built-in RGB LEDs illuminate from different angles
  3. Miniature camera captures the inner surface of the elastomer
  4. Object contact causes elastomer deformation
  5. 3D contact geometry reconstructed via Photometric Stereo

Applications in robot learning:

  • Slip detection: Detecting whether an object is about to slip from the gripper
  • Material recognition: Different materials produce different textures in tactile images
  • Contact pose estimation: Inferring the precise pose of grasped objects
  • Dexterous manipulation: Multimodal policies combining vision and tactile
# GelSight data processing example
import cv2
import numpy as np

# Read tactile image
tactile_img = cv2.imread("gelsight_contact.png")

# Compute contact area
diff = cv2.absdiff(tactile_img, reference_img)
gray = cv2.cvtColor(diff, cv2.COLOR_BGR2GRAY)
_, contact_mask = cv2.threshold(gray, 20, 255, cv2.THRESH_BINARY)

# Estimate contact force (simplified model)
contact_area = np.sum(contact_mask > 0)
estimated_force = contact_area * force_per_pixel  # Requires calibration

Sensor Fusion

Real robot systems require fusing information from multiple sensors:

Fusion Approach Sensor Combination Output Typical Framework
VIO Camera + IMU 6-DoF pose VINS-Fusion, ORB-SLAM3
LiDAR-Inertial LiDAR + IMU 6-DoF pose + point cloud map LIO-SAM, FAST-LIO2
Visual-Tactile Camera + tactile Grasping policy Custom (research frontier)
Multimodal Perception RGB + Depth + F/T Manipulation policy input LeRobot, robomimic

For more on calibration and integration, see Calibration and System Integration.

评论 #