Skip to content

Visual Cortex Prosthesis

A cortical visual prosthesis is the write-in side of BCI — letting the blind "see" via cortical stimulation. It sits at the three-way intersection of medicine, neuroscience, and AI. Fernández 2021 Science Advances is the modern milestone for this direction.

1. Visual Cortex Organization (Recap)

  • V1: retinotopy — each location on the cortex corresponds to a region in the visual field
  • Central vision (fovea) occupies a disproportionate share of V1
  • Stimulating V1 evokes "phosphenes" — the perception of a spot of light at a specific location

This is the anatomical basis of visual cortical prostheses.

2. Three Visual Prosthesis Paths

Path Location Resolution Limits
Retinal prosthesis Retina 60 pixels (Argus II) Retina must still be healthy
Optic nerve Optic nerve ~40 pixels Not applicable after optic-nerve degeneration
V1 cortex Occipital V1 96 channels (Fernández) Bypasses all downstream — broadly applicable

V1 is the "last chance" for blindness BCI — after retinal degeneration or optic-nerve loss, the cortical path is the only option.

3. History: Dobelle to Fernández

Dobelle 1968 (first generation)

William Dobelle made the first attempt in 1968: - 64 electrodes on the cortical surface - Evoked phosphenes - Patients could "see" simple shapes (letters, crosses) - But extremely low resolution and poor stability

2000s stagnation

The Dobelle system never went commercial. Research shifted toward retinal prostheses (Argus II approved in 2013).

Fernández 2021 (second generation)

Fernández et al. (2021, Sci Adv) delivered the modern breakthrough.

4. Fernández 2021 Science Advances

Subject

  • Bernardeta Gómez (blind for 16 years, with prior residual visual concepts)
  • 96-channel Utah Array implanted in V1

Experiment

  1. Stimulation threshold: 70 μA sufficed to evoke phosphenes
  2. Spatial resolution tests: adjacent electrodes evoke phosphenes at different spatial positions
  3. Identification task: user could recognize letters (E, O, H)
  4. Vision + camera: wear a camera + run computer vision → image features → electrode stimulation patterns

Key findings

  • V1 cortical stimulation evokes stable, repeatable visual percepts
  • Low threshold (70 μA, far below retinal prosthesis requirements)
  • First cortical prosthesis to achieve "letter reading"

Significance

  • Proves the V1 cortical visual prosthesis path is viable
  • 96 channels is not enough for a clear image, but the proof-of-principle is achieved
  • Opens the path to the next generation with 1000+ channels

5. Phosphene Engineering

Controllability

By tuning current amplitude, frequency, and pulse width, phosphene properties can be modulated:

Parameter Effect
Current amplitude Brightness
Frequency Flicker
Pulse width Size
Electrode selection Spatial position

Multi-electrode stimulation

Simultaneously stimulating multiple electrodes produces a mosaic pattern — akin to a low-resolution display.

But there is nonlinear cross-interaction: co-stimulating adjacent electrodes does not equal the superposition of two single phosphenes.

Phosphene shapes

Empirically, phosphenes are usually dots, but also include: - Line segments - Color patches - Even simple shapes

6. Differentiable Phosphene Simulation

Latest 2024 progress: differentiable phosphene simulation.

Problem

Directly optimizing "electrode pattern → user percept" requires extensive patient feedback — slow and subjective.

Solution

  1. Build a differentiable phosphene generation model: input electrode pattern, output predicted phosphene image
  2. Train the generative model with user feedback
  3. Given a target image, use gradients to optimize the electrode pattern

Advantages

  • End-to-end optimizable: target image → best electrode pattern
  • No need to poll the user each time
  • Supports dynamic vision

Representatives

  • Beauchamp 2020 and related work at the Moran Eye Institute
  • Duret 2024 Differentiable phosphene simulation

7. Next-Generation High-Channel-Count Prostheses

Moran Eye / BLV (Baylor-Illinois-NIH-VA)

  • 1000+ channel silicon-based
  • Multiple regions (V1 + V2 + LGN)
  • Human trials expected 2026+

Second Sight (restructured after bankruptcy)

  • Orion project (V1 prosthesis)
  • 60+ channels
  • Preclinical

Chinese visual-prosthesis projects

  • Related research at SJTU, Tsinghua Neuracle
  • Human trials expected 2025–2027

8. Integration with Computer Vision

A visual prosthesis doesn't work in isolation — it requires camera + computer vision processing:

Camera image
  ↓
Object recognition + salient region extraction
  ↓
Low-resolution phosphene mapping
  ↓
Electrode stimulation pattern
  ↓
User percept

AI optimization

  • Saliency awareness: stimulate only key objects (YOLO + attention)
  • Edge enhancement: sharpen boundaries
  • Motion emphasis: moving objects brighter
  • Personalization: optimize per user

BrainGate also needs AI for vision processing — making 96 "pixels" as useful as possible.

9. Brain-to-Video Decoding vs Visual Prosthesis

These two paths are dual:

  • Brain-to-video decoding: brain → visual content (read)
  • Visual prosthesis: visual content → brain (write)

They share: - Visual-cortex anatomy - CLIP / visual representations - Generative-model priors

Future fused systems might include: - Blind user wears a camera → AI processes → V1 stimulation (write) - Simultaneously record V1 response → decode "what the user sees" (read) - Closed-loop optimization

This is the direction of bidirectional visual BCI.

10. Ethics and Patient Experience

Visual prosthesis ≠ restoring vision

Users do not see "normal images", but rather low-resolution phosphene mosaics — they have to learn to interpret them.

Adaptation process

The brain needs months to integrate phosphenes into meaningful vision — neural plasticity is key.

Risks

  • Electrode infection
  • Long-term stability
  • Psychological adjustment (vision vs expectation)

Complex expectation management — patients must understand "it won't look like normal seeing."

11. Logical Chain

  1. V1 retinotopy is the anatomical basis of cortical visual prostheses.
  2. Fernández 2021 used a 96-channel Utah to let a 16-year-blind patient see letters.
  3. Phosphenes are the fundamental visual unit of cortical stimulation; multiple electrodes form low-resolution images.
  4. Differentiable phosphene simulation lets electrode patterns be optimized by AI.
  5. Next-generation 1000+ channel prostheses are the goal — approaching practical vision.
  6. AI + camera + visual prosthesis is the complete system: CV processing + cortical write-in.
  7. Read-write fusion is the future direction of visual BCI.

References

  • Fernández et al. (2021). Visual percepts evoked with an intracortical 96-channel microelectrode array in a blind patient. Science Advances. https://www.science.org/doi/10.1126/sciadv.abf8986
  • Dobelle et al. (1974). Phosphenes produced by electrical stimulation of human occipital cortex, and their application to the development of a prosthesis for the blind. J Physiol.
  • Beauchamp et al. (2020). Dynamic stimulation of visual cortex produces form vision in sighted and blind humans. Cell. https://www.cell.com/cell/fulltext/S0092-8674(20)30437-2
  • Chen et al. (2020). Shape perception via a high-channel-count neuroprosthesis in monkey visual cortex. Science.
  • Duret et al. (2024). End-to-end optimization of phosphene patterns via differentiable simulation. bioRxiv.

评论 #