# abhay-dronavalli/clench-muse-als architecture

> Architecture diagram of the GitHub repository abhay-dronavalli/clench-muse-als, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abhay-dronavalli/clench-muse-als
- Repository: https://github.com/abhay-dronavalli/clench-muse-als

## Overview

Clench is an assistive communication and room-control system: jaw clenches and blinks drive a menu, while gaze or head pointing selects tiles; the Core serves the board and caregiver console, and actions can speak, message, call, or control a room. A separate sensor process reads Muse input and reports commands to the Core. The web client also supports camera-based pointing and sensor supervision; an Android tablet implementation provides another gaze and Muse-input surface. README content and most implementation excerpts were unavailable, so uncertain internal call wiring is omitted.

Main entry points: [`core/session.py`](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/session.py) (Menu session), [`core/main.py`](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/main.py) (Core server), [`web/src/board/BoardPage.tsx`](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/board/BoardPage.tsx) (Patient board), [`web/src/console/ConsolePage.tsx`](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/console/ConsolePage.tsx) (Caregiver console)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inputs["Input and pointing"]
  node_sensorprocess["Muse sensor process<br/>[main.py]"]
  node_detections["Gesture detection"]
  node_musesource["Muse input<br/>[muse.py]"]
  node_webpointer["Web pointing<br/>[usePointing.ts]"]
  node_webtracking["Camera tracking<br/>[tracker.ts]"]
  node_tablet["Tablet app"]
end

subgraph group_core["Core runtime"]
  node_coreapp["Core server<br/>[main.py]"]
  node_session["Menu session<br/>[session.py]"]
  node_menu["Communication menu<br/>[menu.py]"]
  node_ranking["Choice ranking<br/>[ranker.py]"]
  node_suggest["Phrase suggestions<br/>[service.py]"]
end

subgraph group_interfaces["User interfaces"]
  node_webapp["Web application<br/>[main.tsx]"]
  node_board["Patient board<br/>[BoardPage.tsx]"]
  node_console["Caregiver console<br/>[ConsolePage.tsx]"]
  node_socket["WebSocket client<br/>[useSocket.ts]"]
end

subgraph group_outcomes["Communication and actions"]
  node_actions["Action handlers"]
  node_voice["Speech output<br/>[voice.py]"]
end

subgraph group_state["State and integrations"]
  node_history["Interaction history<br/>[history.py]"]
  node_database[("SQLite state<br/>[db.py]")]
  node_sensorcontrol["Sensor supervisor<br/>[sensor_service.py]"]
  node_llm["Gemini provider<br/>[gemini.py]"]
end

node_patient(("Patient"))
node_caregiver(("Caregiver"))
node_family(("Family and contacts"))
node_room(("Room systems"))
node_speech(("Audio playback"))

node_patient -->|"uses"| node_board
node_caregiver -->|"configures"| node_console
node_webapp -->|"routes"| node_board
node_webapp -->|"routes"| node_console
node_board -->|"exchanges messages"| node_socket
node_console -->|"exchanges messages"| node_socket
node_socket -->|"WebSocket traffic"| node_coreapp
node_sensorprocess -->|"reads samples"| node_musesource
node_sensorprocess -->|"detects gestures"| node_detections
node_sensorprocess -->|"sends sensor events"| node_coreapp
node_coreapp -->|"dispatches input"| node_session
node_coreapp -->|"supervises process"| node_sensorcontrol
node_session -->|"requests suggestions"| node_suggest
node_session -->|"uses menu"| node_menu
node_session -->|"ranks choices"| node_ranking
node_session -->|"dispatches actions"| node_actions
node_session -->|"requests speech"| node_voice
node_suggest -->|"reads summaries"| node_history
node_history -->|"reads and writes"| node_database
node_suggest -.->|"requests completions"| node_llm
node_actions -.->|"messages or calls"| node_family
node_actions -.->|"controls"| node_room
node_voice -->|"provides audio"| node_speech
node_board -->|"uses pointing"| node_webpointer
node_webpointer -->|"subscribes samples"| node_webtracking
node_webpointer -->|"sends tile picks"| node_socket
node_tablet -->|"provides tablet interface"| node_patient
node_console -->|"controls sensor"| node_sensorcontrol

click node_sensorprocess "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/sensor/main.py"
click node_detections "https://github.com/abhay-dronavalli/clench-muse-als/tree/main/sensor/detect"
click node_musesource "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/sensor/sources/muse.py"
click node_webpointer "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/facetrack/usePointing.ts"
click node_webtracking "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/facetrack/tracker.ts"
click node_tablet "https://github.com/abhay-dronavalli/clench-muse-als/tree/main/kushagra/tablet/app/src/main/kotlin/com/clench/eyetrack"
click node_webapp "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/main.tsx"
click node_board "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/board/BoardPage.tsx"
click node_console "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/console/ConsolePage.tsx"
click node_socket "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/lib/useSocket.ts"
click node_coreapp "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/main.py"
click node_session "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/session.py"
click node_menu "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/menu.py"
click node_ranking "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/rank/ranker.py"
click node_suggest "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/suggest/service.py"
click node_actions "https://github.com/abhay-dronavalli/clench-muse-als/tree/main/core/actions"
click node_voice "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/voice.py"
click node_history "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/rank/history.py"
click node_database "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/db.py"
click node_sensorcontrol "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/sensor_service.py"
click node_llm "https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/suggest/gemini.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_sensorprocess,node_detections,node_musesource,node_webpointer,node_webtracking,node_tablet toneBlue
class node_coreapp,node_session,node_menu,node_ranking,node_suggest toneAmber
class node_webapp,node_board,node_console,node_socket toneMint
class node_actions,node_voice toneRose
class node_history,node_database,node_sensorcontrol,node_llm,node_patient,node_caregiver,node_family,node_room,node_speech toneIndigo
```

## Components

### Input and pointing

- [Muse sensor process](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/sensor/main.py): `sensor/main.py`
- [Gesture detection](https://github.com/abhay-dronavalli/clench-muse-als/tree/main/sensor/detect): `sensor/detect`
- [Muse input](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/sensor/sources/muse.py): `sensor/sources/muse.py`
- [Web pointing](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/facetrack/usePointing.ts): `web/src/facetrack/usePointing.ts`
- [Camera tracking](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/facetrack/tracker.ts): `web/src/facetrack/tracker.ts`
- [Tablet app](https://github.com/abhay-dronavalli/clench-muse-als/tree/main/kushagra/tablet/app/src/main/kotlin/com/clench/eyetrack): `kushagra/tablet/app/src/main/kotlin/com/clench/eyetrack`

### Core runtime

- [Core server](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/main.py): `core/main.py`
- [Menu session](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/session.py): `core/session.py`
- [Communication menu](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/menu.py): `core/menu.py`
- [Choice ranking](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/rank/ranker.py): `core/rank/ranker.py`
- [Phrase suggestions](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/suggest/service.py): `core/suggest/service.py`

### User interfaces

- [Web application](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/main.tsx): `web/src/main.tsx`
- [Patient board](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/board/BoardPage.tsx): `web/src/board/BoardPage.tsx`
- [Caregiver console](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/console/ConsolePage.tsx): `web/src/console/ConsolePage.tsx`
- [WebSocket client](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/web/src/lib/useSocket.ts): `web/src/lib/useSocket.ts`

### Communication and actions

- [Action handlers](https://github.com/abhay-dronavalli/clench-muse-als/tree/main/core/actions): `core/actions`
- [Speech output](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/voice.py): `core/voice.py`

### State and integrations

- [Interaction history](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/rank/history.py): `core/rank/history.py`
- [SQLite state](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/db.py): `core/db.py`
- [Sensor supervisor](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/sensor_service.py): `core/sensor_service.py`
- [Gemini provider](https://github.com/abhay-dronavalli/clench-muse-als/blob/main/core/suggest/gemini.py): `core/suggest/gemini.py`

### Other components

- **Patient**
- **Caregiver**
- **Family and contacts**
- **Room systems**
- **Audio playback**

## Connections

- Patient → Patient board: uses
- Caregiver → Caregiver console: configures
- Web application → Patient board: routes
- Web application → Caregiver console: routes
- Patient board → WebSocket client: exchanges messages
- Caregiver console → WebSocket client: exchanges messages
- WebSocket client → Core server: WebSocket traffic
- Muse sensor process → Muse input: reads samples
- Muse sensor process → Gesture detection: detects gestures
- Muse sensor process → Core server: sends sensor events
- Core server → Menu session: dispatches input
- Core server → Sensor supervisor: supervises process
- Menu session → Phrase suggestions: requests suggestions
- Menu session → Communication menu: uses menu
- Menu session → Choice ranking: ranks choices
- Menu session → Action handlers: dispatches actions
- Menu session → Speech output: requests speech
- Phrase suggestions → Interaction history: reads summaries
- Interaction history → SQLite state: reads and writes
- Phrase suggestions → Gemini provider: requests completions
- Action handlers → Family and contacts: messages or calls
- Action handlers → Room systems: controls
- Speech output → Audio playback: provides audio
- Patient board → Web pointing: uses pointing
- Web pointing → Camera tracking: subscribes samples
- Web pointing → WebSocket client: sends tile picks
- Tablet app → Patient: provides tablet interface
- Caregiver console → Sensor supervisor: controls sensor