SOMA & ANIMA
← Back to version list

V0.3 — From "tech dashboard" to "product interface"

v0.2 was optimized for "does the tech work," with a four-column layout of ANIMA dashboards + a simulation window. v0.3 inverts the information hierarchy: first-time viewers should see the product (a care robot doing things in a ward), not the tech stack. ANIMA's L0–L5 details move into collapsible drawers — available on demand for anyone who wants to dig in.

UI: from dark tech theme to light clinical theme

  • Theme shifts from zinc-950 dark to white + zinc-200 borders, matching the aesthetic of a real clinical product surface
  • New bci/ component group: SessionHeader / ChannelHealth / NeuralActivity / DecoderOutput / FiringRateBars — all labeled decorative, consistent with the honesty line from v0.1 ("do not fake neural decoding")
  • New shell/ component group: MissionRibbon (top mission bar) / BciDrawer (left drawer) / TaskDrawer (right drawer) — drawers collapse by default so the MJPEG feed owns the visual field
  • SimulationView upgrades to a product-grade component with multi-camera switching, E-stop, and status polling

Backend: five-camera render pipeline

From a single demo_view to five: demo_view / grasp_view / bedside_view / top_down / tv_view.

Implementation detail:

  • _CameraFeed class with one mujoco.Renderer per feed plus its own latest_jpeg and condvar
  • A single render thread syncs all feeds from one shared MjData
  • /api/sim/cameras advertises the list; /api/sim/mjpeg?camera=X picks one by name
  • Disable each renderer's mjVIS_RANGEFINDER flag (otherwise the floor fills with yellow rangefinder beams)

E-stop bypass channel

The first emergency-stop path independent of the LLM and behavior tree:

  • SimManager.estop_active exposes a global threading.Event
  • POST /api/sim/estop sets the flag and zeros base velocity; POST /api/sim/estop/clear lifts it
  • SimSkillBehaviour.update checks the flag as its first action each tick → immediate FAILURE, bypassing LLM / BT rebuild
  • Reset Sim also clears E-stop
  • Frontend: red button top-right; when active it becomes an amber "Clear E-stop", with a red warning band across the top

This is the prototype of what v0.4 formalizes as "three non-signal-path bypass channels."

All six intents running end-to-end

Each intent has a concrete demonstrable action:

  • DRINK_WATER — five-step grasp-and-deliver (inherited from v0.2)
  • ADJUST_TV — TV screen emission material brightens to sky blue
  • CALL_HELP — robot arm raises as a beacon; a nurse mocap appears at bedside and holds for 6 s before retreating; a help.called event is pushed over WebSocket
  • GOTO_BED — base drives to the bedside head pose (4.5, 1.3, -π/2)
  • TURN_OFF_LIGHT / TURN_ON_LIGHT — five light sources zero or restore diffuse; ceiling lamp emission toggles dark / bright. Key detail: the subtask name (turn_off_light / turn_on_light) determines the target state, not a pure toggle — otherwise "say 'turn off light' twice" would surprise the user by turning the light back on
  • E-stop mid-run → outcome=cancel

Lessons worth recording

Bedside-pose obstacle avoidance: the first choice of BEDSIDE_POSE was (4.2, 1.0, -π/2). The unicycle PID got stuck around (3.7, 1.0) oscillating for 50+ seconds until timeout. Root cause: the nightstand's collision box has a y-edge at 0.79; the Stretch base radius is ~0.35, so any robot center with y < 1.14 collides with the nightstand's north edge. Changing to y=1.3 leaves a 0.16 m safety margin, and the robot reaches the pose on the first try.

Rendering ceiling: Hetzner CPX31 has no GPU. Software rendering five cameras via osmesa tops out at ~0.4 fps. The direct consequence: any BT-driven "slide-in" animation (like the original design where the nurse walks in through the door) only samples 1–2 frames and visually reads as teleportation. The fix is to rewrite animations as discrete keyframes that hold each target state ≥ one render period — the 6-second hold in CallHelpSkill exists for exactly this reason. This hard constraint directly drove v0.4's decision to abandon the Hetzner live demo and move to local Mac Metal rendering + offline video delivery.

Canonical subtask safety net

l1_parser.py gains a _CANONICAL_PLANS table: every intent carries a deterministic BT fallback plan. Even if the LLM parse fails, the demo doesn't stall. In a demonstration setting, "never get stuck" beats "always take the optimal path."

Acceptance (passed)

  • /api/sim/cameras returns five cameras
  • tv_view MJPEG: 200 OK, ~50 KB per frame
  • Two consecutive /api/sim/reset calls no longer SEGV (an earlier version had a lifetime race between the render thread and _model — fix lives in SimManager.stop())
  • All six intents run outcome=success end-to-end in the frontend; E-stop gives outcome=cancel

Handoff from v0.3 → v0.4

v0.3 is the last version from the research-prototype era. Delivery was still the Hetzner live demo. v0.4 onwards happens in the standalone jeffliulab/soma-care repo and shifts toward local high-frame-rate rendering, offline video delivery, and finer contact-rich operations. The original Hetzner demo (89.167.35.145) stays accessible but is no longer the primary showcase.