How abhay-dronavalli/clench-muse-als works
Clench is an assistive communication and room-control system: jaw clenches and blinks drive a menu, while gaze or head pointing selects tiles; the Core serves the board and caregiver console, and actions can speak, message, call, or control a room. A separate sensor process reads Muse input and reports commands to the Core. The web client also supports camera-based pointing and sensor supervision; an Android tablet implementation provides another gaze and Muse-input surface. README content and most implementation excerpts were unavailable, so uncertain internal call wiring is omitted.
Main entry points: session.py Menu session · main.py Core server · BoardPage.tsx Patient board · ConsolePage.tsx Caregiver console
Components 26
Input and pointing
- Muse sensor process
sensor/main.py - Gesture detection
sensor/detect - Muse input
sensor/sources/muse.py - Web pointing
web/src/facetrack/usePointing.ts - Camera tracking
web/src/facetrack/tracker.ts - Tablet app
kushagra/tablet/app/src/main/kotlin/com/clench/eyetrack
Core runtime
- Core server
core/main.py - Menu session
core/session.py - Communication menu
core/menu.py - Choice ranking
core/rank/ranker.py - Phrase suggestions
core/suggest/service.py
User interfaces
- Web application
web/src/main.tsx - Patient board
web/src/board/BoardPage.tsx - Caregiver console
web/src/console/ConsolePage.tsx - WebSocket client
web/src/lib/useSocket.ts
Communication and actions
- Action handlers
core/actions - Speech output
core/voice.py
State and integrations
- Interaction history
core/rank/history.py - SQLite state
core/db.py - Sensor supervisor
core/sensor_service.py - Gemini provider
core/suggest/gemini.py
Other components
- Patient
- Caregiver
- Family and contacts
- Room systems
- Audio playback
Connections 28
- Patient → Patient board: uses
- Caregiver → Caregiver console: configures
- Web application → Patient board: routes
- Web application → Caregiver console: routes
- Patient board → WebSocket client: exchanges messages
- Caregiver console → WebSocket client: exchanges messages
- WebSocket client → Core server: WebSocket traffic
- Muse sensor process → Muse input: reads samples
- Muse sensor process → Gesture detection: detects gestures
- Muse sensor process → Core server: sends sensor events
- Core server → Menu session: dispatches input
- Core server → Sensor supervisor: supervises process
- Menu session → Phrase suggestions: requests suggestions
- Menu session → Communication menu: uses menu
- Menu session → Choice ranking: ranks choices
- Menu session → Action handlers: dispatches actions
- Menu session → Speech output: requests speech
- Phrase suggestions → Interaction history: reads summaries
- Interaction history → SQLite state: reads and writes
- Phrase suggestions → Gemini provider: requests completions
- Action handlers → Family and contacts: messages or calls
- Action handlers → Room systems: controls
- Speech output → Audio playback: provides audio
- Patient board → Web pointing: uses pointing
- Web pointing → Camera tracking: subscribes samples
- Web pointing → WebSocket client: sends tile picks
- Tablet app → Patient: provides tablet interface
- Caregiver console → Sensor supervisor: controls sensor