Real-Time Systems

Overview

A robot system is a multi-frequency, multi-tier real-time control system. From 1kHz servo control to 10Hz perception processing, different tiers have vastly different real-time requirements. This article covers control frequency tiers, real-time operating systems (RTOS), ROS2 real-time features, and communication buses.

Real-time != Fast

The core of a real-time system is determinism — guaranteeing task completion within a specified time, not pursuing maximum speed. A system that completes in 1ms but occasionally takes 10ms is less reliable than one that consistently takes 2ms.

I. Control Frequency Tiers

1.1 Multi-Frequency Architecture

Tier	Frequency	Period	Function	Real-time Requirement
Current loop	10-40 kHz	25-100 us	Motor current control	Hard real-time
Servo control	1 kHz	1 ms	Joint position/velocity/torque control	Hard real-time
Cartesian control	200-500 Hz	2-5 ms	End-effector trajectory interpolation, impedance control	Hard real-time
Motion planning	10-100 Hz	10-100 ms	Path planning, obstacle avoidance	Soft real-time
Perception processing	5-30 Hz	33-200 ms	Vision, point cloud processing	Soft real-time
Task planning	0.1-1 Hz	1-10 s	High-level decisions, VLA inference	Non-real-time

Hard vs Soft Real-time

Hard real-time: Exceeding the deadline constitutes system failure (may cause safety incidents)
Soft real-time: Occasional timeouts are acceptable but degrade performance
Non-real-time: Latency-insensitive, can be queued

1.2 Jitter Requirements

Tier	Allowed Jitter	Notes
Current loop	< 1 us	Guaranteed by driver FPGA
Servo control	< 50 us	Requires RTOS
Cartesian control	< 200 us	Requires RTOS
Motion planning	< 5 ms	Standard Linux acceptable
Perception processing	< 30 ms	Standard Linux acceptable

II. Real-Time Operating Systems (RTOS)

2.1 PREEMPT_RT vs Xenomai

Feature	PREEMPT_RT	Xenomai (Cobalt)
Architecture	Linux kernel patch	Dual-kernel (microkernel + Linux)
Worst-case latency	50-100 us	10-30 us
Average latency	10-30 us	1-5 us
Linux compatibility	Fully compatible	Requires dedicated API (POSIX skin)
Maintenance status	Being merged into mainline	Community maintained, declining
Suitable for	1kHz control is sufficient	Need >1kHz or extremely low jitter
Typical users	Universal Robots, Franka	KUKA (earlier), some CNC

Selection Recommendation

The 2024-2025 trend favors PREEMPT_RT. As it gradually merges into the Linux mainline kernel, its ecosystem advantage grows. Unless <20us extreme real-time is needed, PREEMPT_RT is recommended.

2.2 Real-Time Linux Configuration

# 1. Install PREEMPT_RT kernel
sudo apt install linux-image-rt-amd64

# 2. CPU isolation (isolcpus) - prevent normal processes from interfering
# Add to GRUB:
GRUB_CMDLINE_LINUX="isolcpus=2,3 nohz_full=2,3 rcu_nocbs=2,3"

# 3. Set real-time priority
sudo chrt -f 99 ./robot_controller

# 4. Lock memory (prevent page swapping)
mlockall(MCL_CURRENT | MCL_FUTURE);

# 5. Disable frequency scaling
sudo cpupower frequency-set -g performance

2.3 Latency Testing

Use cyclictest to evaluate real-time performance:

# Test for 10 minutes on isolated core
sudo cyclictest -t1 -p99 -n -i1000 -l600000 -a2 -m

# Key metrics:
# Min: minimum latency
# Avg: average latency (target < 20us)
# Max: maximum latency (target < 100us)

III. ROS2 Real-Time Features

3.1 DDS QoS Configuration

QoS Policy	Real-time Recommendation	Description
Reliability	BEST_EFFORT	Avoid retransmission delay
Durability	VOLATILE	Don't cache historical data
History	KEEP_LAST(1)	Keep only latest message
Deadline	Set to control period	Detect communication timeout
Liveliness	MANUAL_BY_TOPIC	Detect node liveness

3.2 ros2_control Framework

+------------------------------------------+
|           Controller Manager              |
|  +----------+  +----------+  +--------+  |
|  | Position  |  | Velocity |  | Force  |  |
|  |Controller|  |Controller|  |Control |  |
|  +----+-----+  +----+-----+  +---+----+  |
|       |              |            |        |
|  +----+--------------+------------+----+   |
|  |       Hardware Interface Layer      |   |
|  +----+--------------+------------+----+   |
+-------+              |            +--------+
        |              |            |
   +----+----+   +----+----+  +---+-----+
   |EtherCAT |   |  CAN    |  |  Serial |
   | Driver  |   | Driver  |  | Driver  |
   +---------+   +---------+  +---------+

IV. Communication Buses

4.1 Industrial Communication Bus Comparison

Bus	Bandwidth	Typical Latency	Jitter	Topology	Typical Application
EtherCAT	100 Mbps / 1 Gbps	< 100 us	< 1 us	Daisy chain/star	High-performance servo
PROFINET IRT	100 Mbps	< 1 ms	< 1 us	Star	Siemens ecosystem
CAN/CANopen	1 Mbps	1-10 ms	100 us	Bus	Mobile robots
CAN FD	8 Mbps	0.5-5 ms	50 us	Bus	Next-gen joints
SPI	10-50 MHz	< 10 us	< 1 us	Master-slave	On-board sensors
RS-485/Modbus	115.2 kbps	5-50 ms	1 ms	Bus	Simple devices
USB	480 Mbps	1-10 ms	1 ms	Star	Cameras, peripherals

4.2 EtherCAT Details

EtherCAT is the most mainstream real-time bus in robotics:

Working Principle: - Master sends Ethernet frames; slaves read/write data "on the fly" as frames pass through - Frames need only traverse all slaves once, providing extremely low latency - Typical: 16 servo drives, 1ms cycle, jitter < 1us

Open-source Masters:

Master	License	Features
IgH EtherCAT Master	GPL	Linux kernel module, full-featured
SOEM (Simple Open EtherCAT Master)	LGPL	User-space, cross-platform
EtherLab	GPL	MATLAB/Simulink integration

V. Real-Time Performance Optimization

5.1 Code Level

Avoid dynamic memory allocation: Pre-allocate all buffers
Avoid lock contention: Use lock-free queues
Avoid system calls: Reduce context switches
Avoid I/O blocking: Use asynchronous I/O
Avoid exception handling: Disable C++ exceptions (-fno-exceptions)

5.2 Typical Latency Budget

For a 1kHz control loop (1ms period):

Stage	Allocated Time	Description
Sensor readout	50 us	EtherCAT process data
State estimation	50 us	Filtering, forward kinematics
Control computation	200 us	Inverse dynamics, torque computation
Safety check	50 us	Limits, collision detection
Command output	50 us	EtherCAT write
Margin	600 us	For worst-case scenarios
Total	1000 us

60% Margin Principle

In real-time systems, CPU utilization under normal conditions should not exceed 40%, leaving sufficient margin for worst-case scenarios.