Shared Autonomy

Shared Autonomy is the core design paradigm for fusing BCIs with robots: the user supplies intent, the robot supplies execution. It resolves the fundamental mismatch between a low-bandwidth signal source and a robot that requires high-bandwidth commands, and is the engineering default for today's assistive-robot BCIs.

1. Why Shared Autonomy Is Necessary

The BCI signal bottleneck

even the best BCIs (Neuralink, Willett 2023) deliver only ~8–30 bps
humans speak at ~40 bps and type at ~10 bps
a 7-DoF robot arm needs about 200 bps of control signal per second

Conclusion: BCI bandwidth will never be enough to directly drive a complex robot — the robot must "fill in the gaps" itself.

Cognitive load

Even if the bandwidth were sufficient, having the user control 7 DoF simultaneously is exhausting — fatigue, errors, and frustration follow.

Execution precision

Human neural noise produces control jitter; robots are precise and stable.

The promise of shared autonomy: the user is the intent layer and the robot is the execution layer.

2. The Javdani-Srinivasa Framework

Javdani, Srinivasa & Bagnell (2018, IJRR) introduced the standard POMDP framework for shared autonomy:

States

robot state \(s\) (known)
user goal \(g\) (unknown, partially observable)
user input \(u_t\) (a noisy version of the intent)

POMDP structure

\[P(g | u_{1:t}) \propto P(g) \prod_{\tau=1}^t P(u_\tau | g, s_\tau)\]

The robot maintains a belief over goals and picks the action that maximises expected utility:

\[a^* = \arg\max_a \mathbb{E}_{g \sim P(g|u)} [V(s', g)]\]

Hindsight Optimization

Exact POMDP solution is intractable — Javdani's Hindsight Optimization approximates it by assuming "the future belief equals the current posterior":

\[a_{\text{HO}}^* = \arg\max_a \sum_g P(g|u) \cdot Q^{\text{full-obs}}(s, a, g)\]

This reduces the POMDP to an MDP weighted by the belief — computationally tractable and close in performance.

3. Classic Shared-Autonomy Behaviours

Assisted grasping

The user nudges the joystick → the robot identifies "toward which object" → it automatically adjusts the arm trajectory and hand pose.

Assisted mobility

The user gives a rough direction → the robot handles obstacle avoidance and follow-through.

Assisted feeding

The user picks "what to eat" → the robot identifies the location, plans a grasp, and delivers food to the mouth.

Assisted typing

The user's BCI intent is low-bandwidth → LLM guessing completes the rest (see LLM post-processing fusion).

4. Arbitration (Allocation of Authority)

The central design question for shared autonomy: who gets the final word, the user or the robot?

Several strategies

Fixed blending: \(a = \alpha \cdot a_{\text{user}} + (1-\alpha) \cdot a_{\text{robot}}\) with \(\alpha\) fixed
Confidence weighting: when the robot is confident it takes more autonomy; when the user signal is strong the robot yields
User override: an unambiguous user intent interrupts the robot
Layered: the robot plans at the high level and the user fine-tunes at the low level

Best practice (empirical)

novices / fatigued users: robot-dominant
experts / fine-grained phases: user-dominant
Arbitration should be tunable — let the user choose the degree of autonomy

"User-in-the-loop" vs "Human-in-the-loop": the former is a real-time closed loop; the latter is triggered by events. BCIs are typically user-in-the-loop.

5. LLMs as the New Brain for Shared Autonomy

After 2023, LLMs have become the "brain of the robot" in shared autonomy:

BrainBody-LLM architecture

BCI intent (low bandwidth)
  ↓
LLM (GPT-4 / Claude) — expand to a full plan
  ↓
robot planning (ROS2 / MoveIt)
  ↓
execution

BCI conveys ~100 bps of "semantics"
the LLM expands this into a 1000+ step action sequence
the robot executes

Representative work

HiCRISP (Chen 2023): LLM + BCI hierarchical planning
PaLM-E (Google 2023): multimodal LLM for robot reasoning
RT-2 (Google 2023): end-to-end vision-language-action

The BCI + LLM + robot stack is the hottest direction of 2024–2026. See Hierarchical Planning: BCI + LLM + Robot.

6. Evaluating Shared Autonomy

Beyond "task success rate", we need to evaluate:

task completion time (shorter is better)
user intervention frequency (fewer is better)
user cognitive load (NASA-TLX scale)
error recovery (can the user interrupt / correct?)
satisfaction (subjective user rating)

The ideal curve: as autonomy grows, completion time ↓, cognitive load ↓, but "sense of user control" ↓ as well. The sweet spot lies somewhere in the middle.

7. Philosophical Issues of Shared Autonomy

Ownership of intent

When the robot makes 90% of the decisions on the user's behalf, "is this still the user's action?" — a question of embodied philosophy.

Responsibility for errors

When an accident is caused by the robot's autonomous decision, who bears responsibility? The user, the BCI company, or the robot manufacturer?

Neurorights

The robot can continuously infer the user's intent from the BCI — does such inference count as a violation of the user's "mental privacy"? (See Chapter 13 Neurorights.)

8. Landmark Systems

Pitt robot arm (2013)

Collinger's team: a 7-DoF arm + low-level shared autonomy. The user controls 3D position + gripper and the robot handles joint redundancy automatically.

Walk Again exoskeleton (2016)

Nicolelis's team: a paralysed user initiates gait via BCI and the exoskeleton generates the specific stepping pattern. This is event-triggered shared autonomy.

BrainGate Coffee (2012)

Hochberg's team: tetraplegic patient Cathy Hutchinson picks up and drinks coffee — the first human demonstration of shared autonomy.

Neuralink PRIME (2024)

Noland Arbaugh uses N1 to control the mouse + OS-level assistance (autocomplete, AI assistant) — consumer-grade shared autonomy.

9. Overlap with HRI (Human-Robot Interaction)

Shared autonomy is not only a BCI question, it is also a central topic of HRI (Human-Robot Interaction):

How does the robot infer user intent?
How does the robot communicate its plan back to the user?
How do we recover when errors occur?

BCI adds one extra dimension — the user conveys intent directly through neural signals — but all HRI principles still apply.

10. Chain of Reasoning

BCI bandwidth will never be enough to directly drive a complex robot — shared autonomy is the only solution.
POMDP + Hindsight Optimization is the mathematical standard for shared autonomy.
The arbitration mechanism decides the balance of authority between user and robot, and must be tunable.
LLMs push shared autonomy to the "semantic level" — BCI intent → LLM expansion → robot execution.
Shared autonomy raises philosophical, ethical, and legal issues — all of which are key discussions in BCI commercialization.

References

Javdani, Srinivasa & Bagnell (2018). Shared autonomy via hindsight optimization for teleoperation and teaming. IJRR. https://journals.sagepub.com/doi/10.1177/0278364918776480
Jain & Argall (2019). A probabilistic framework for collaborative human-robot problem-solving. Auton Robot.
Collinger et al. (2013). High-performance neuroprosthetic control by an individual with tetraplegia. Lancet.
Hochberg et al. (2012). Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature. https://www.nature.com/articles/nature11076
Chen et al. (2023). HiCRISP: An LLM-driven hierarchical closed-loop robotic intelligent self-correction planner. arXiv.