In DevelopmentV0.4

SOMA Care

Medical-care intent-to-action simulation

The medical-care product line of the SOMA family. Powered by the ANIMA cognition framework, it turns intent (text, voice, and future BCI signals) into auditable robot actions inside a simulated hospital ward. v0.4 focuses on showcasing the full intent-to-care loop via MuJoCo simulation and offline video.

The care product line of the SOMA family, running in parallel with soma-arm and sharing the ANIMA cognition framework.

SOMA Care
Prototype view of the Stretch RE3 in the simulated ward — bed, nightstand, bedside table, and care robot in one frame.
Why simulation

Why start in simulation

We start with a simulated ward + video delivery rather than real hardware or clinical partners. Reasons: no medical license required, no patient data touched, fully reproducible — yet rich enough to exercise the full L0–L5 intent-to-action loop, five-factor assessment, and failure-negotiation narratives. Hardware and clinical partnerships are deferred to v0.5+.

Architecture

ANIMA cognition stack + five-factor assessment

Intents enter at L0, flow through Parser / Planner / Skill / Adapter, and are rated by the five-factor assessor at L5. Screenshots are from the live UI of the research prototype.

ANIMA six-layer cognitive planning panel
Live visualization of the full L0–L5 stack: Signal / Parser / Planner / Skill / Adapter / Assessment.
Five-factor assessment dashboard
Real-time values and trends for ITA / MQA / SQA / GOA / PEA.
Intent -> Action

Three steps from intent to action

Natural-language intent input panel
L0 Signal: simulated-intent / text / voice input layer.
Natural language to structured TaskSpec
L1 Parser: LLM-as-Parser with forced tool-calling producing a TaskSpec.
Action orchestration view
L2–L4: py_trees behavior tree + skill registry + simulation adapter.
Prototype screenshots

Multi-camera views of the simulated ward

These screenshots are from the v0.1–v0.3 research prototype and mark the baseline for v0.4 video delivery. They will be superseded by local Metal high-frame-rate captures.

Ward top-down camera
top_down camera: global layout and path-planning verification.
Bedside camera view
bedside_view camera: close-up for contact-rich tasks.
Robot fetch-water task
DRINK_WATER skill executing: locate → navigate → grasp → deliver → assess.
Hardware
Robot (simulated)
Stretch RE3 mobile manipulator, running in MuJoCo 3.7 via the hello-robot/stretch_mujoco Python API
Cameras (virtual)
Five virtual cameras: demo_view, grasp_view, bedside_view, top_down, and first_person_nurse
Scene
Simulated ward: bed, nightstand, bedside table, and a physical E-stop button (clickable inside the scene)
Software
Cognition
ANIMA v0.1.0 (pip-installed) + LLM-as-Parser + py_trees
Backend
FastAPI + WebSocket + MJPEG multi-camera streams + MuJoCo passive viewer
Frontend
Next.js 16 + Tailwind v4 + zustand + ReactFlow (live L0–L5 dashboard)
LLM provider
DeepSeek / OpenAI, swappable via the LLMToolCaller protocol defined in anima
Current scope
  • PBR hospital assets + local high-frame-rate rendering on Mac Metal
  • 4–6 contact-rich skills (TurnOver / Feed / HandMedicine / WipeMouth / StraightenBlanket / CompanionChat)
  • Three non-signal-path bypasses: physical E-stop button, voice STOP intercept, eye-closure / head-tilt visual cue
Working now
  • Importing Hospital_assets textures + three-tier lighting (directional sun, fluorescent ceiling, bedside lamp)
  • Wiring MQA sub-indicators (contact force envelope, grasp margin, trajectory smoothness)
  • Scripting, storyboarding, and recording 5–8 thematic videos of 3–5 minutes each
Future scope
  • Non-invasive BCI signal integration (EEG / fNIRS) in simulation (v0.5 candidate)
  • Real-hardware validation on a physical Stretch RE3 (requires hardware access and additional runway)
  • Clinical partner pilot (requires external funding and compliance lift; not in current commitments)
Milestones

Version Roadmap

2026-04 (historical)

v0.1 – v0.3 — Prototype era

Lineage from the human-brain-interface-demo research prototype: FastAPI + Next.js scaffold, six-intent loop, MuJoCo Stretch sim, L0–L5 UI panels, multi-camera MJPEG. Migrated out and frozen as a historical snapshot on 2026-04-21.

2026-04-21 ~

V0.4 — Quality Pass: Simulation, Ops, Content, Video

PBR hospital assets + local Metal high-frame-rate rendering, 4–6 contact-rich skills, 8–10 success narratives + 3–5 failure-negotiation paths, 5–8 thematic videos.

Planned

V0.5P+ — To be scoped

To be decided after v0.4 videos ship and feedback is gathered: non-invasive BCI integration in simulation, real-hardware migration, or clinical pilot. None are committed yet.

Demo / Media
v0.4 thematic videos (recording)
5–8 videos of 3–5 minutes each are being rendered locally on Metal. They will ship to GitHub Releases, YouTube, and soma.jeffliulab.com.
Research-prototype screenshots
See the 'Prototype screenshots' section on this page — material from the v0.1–v0.3 era.
FAQ

Frequently Asked Questions

How does SOMA Care relate to SOMA Arm?

They are parallel product lines sharing the ANIMA cognition framework. SOMA Arm is tabletop chess manipulation on real hardware. SOMA Care is a simulated-ward care loop exercising multi-skill orchestration, failure negotiation, and five-factor assessment. Same L0–L5 stack, different application layers.

Why does SOMA Care start in simulation rather than on real hardware or with a clinical partner?

Simulation lets us exercise the full intent-to-action loop, five-factor assessment, and failure-negotiation narratives without medical licensure, without touching patient data, and with full reproducibility. That keeps the intent-to-care story auditable end-to-end. Real hardware and clinical pilots are v0.5+ candidates pending external hardware, funding, and compliance channels.

What are the active product lines in the SOMA family today?

Two: SOMA Arm (tabletop chess manipulation, the first reference for ANIMA on the home track) and SOMA Care (medical-care intent-to-action loop inside a simulated ward). Both share the same ANIMA cognition framework. soma-watch remains a long-term brand slot and is not part of current commitments.