# lovelypoet/silentvoix architecture

> Architecture diagram of the GitHub repository lovelypoet/silentvoix, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/lovelypoet/silentvoix
- Repository: https://github.com/lovelypoet/silentvoix

## Overview

SilentVoix is a multimodal sign-language recognition platform: glove or camera input is captured, sent through API/runtime inference, and turned into a gesture result and speech. The wider product also supports dataset validation and capture, model upload/activation, fusion workflows, and monitoring. The README documents several runtime and async capabilities; source excerpts confirm selected API, model-library, database, and worker relationships, but do not expose every route or dispatch.

Main entry points: [`api/main.py`](https://github.com/lovelypoet/silentvoix/blob/main/api/main.py) (FastAPI app), [`vue-next/src/App.vue`](https://github.com/lovelypoet/silentvoix/blob/main/vue-next/src/App.vue) (Vue application), [`api/routes/gestures_predict.py`](https://github.com/lovelypoet/silentvoix/blob/main/api/routes/gestures_predict.py) (Gesture prediction), [`services/datasets/dataset_service.py`](https://github.com/lovelypoet/silentvoix/blob/main/services/datasets/dataset_service.py) (Dataset service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Input and capture"]
  node_glove["ESP32 glove<br/>[esp32_glove.cpp]"]
  node_browser["Vue application<br/>[App.vue]"]
  node_sensor_capture["Sensor capture"]
end

subgraph group_api["API and services"]
  node_api_app["FastAPI app<br/>[main.py]"]
  node_live_stream["Live WebSocket<br/>[liveWS.py]"]
  node_gesture_api["Gesture prediction"]
  node_auth["Authentication<br/>[auth_routes.py]"]
  node_core["API core"]
end

subgraph group_inference["Inference and speech"]
  node_pytorch["PyTorch runtime<br/>[app.py]"]
  node_tensorflow["TensorFlow runtime<br/>[app.py]"]
  node_tts["Text to speech<br/>[tts_service.py]"]
end

subgraph group_data["Datasets and models"]
  node_model_routes["Model library API"]
  node_model_service["Model registry"]
  node_model_records["Model records<br/>[model_service.py]"]
  node_dataset_routes["CSV library API"]
  node_dataset_service["Dataset service<br/>[dataset_service.py]"]
  node_postgres[("PostgreSQL<br/>[models.py]")]
  node_mongo[("MongoDB<br/>[database.py]")]
end

subgraph group_async["Async processing"]
  node_dataset_worker["Dataset worker<br/>[dataset_tasks.py]"]
  node_fusion_worker["Early fusion worker<br/>[app.py]"]
  node_redis["Redis broker"]
end

node_user(("User"))

node_user -->|"uses"| node_browser
node_glove -->|"streams sensor data"| node_live_stream
node_browser -->|"captures sensors"| node_sensor_capture
node_browser -->|"calls API"| node_api_app
node_api_app -->|"mounts routes"| node_live_stream
node_api_app -->|"mounts routes"| node_gesture_api
node_api_app -->|"mounts routes"| node_auth
node_live_stream -.->|"requests prediction"| node_gesture_api
node_gesture_api -.->|"dispatches inference"| node_pytorch
node_gesture_api -.->|"dispatches inference"| node_tensorflow
node_pytorch -->|"returns prediction"| node_browser
node_tensorflow -->|"returns prediction"| node_browser
node_browser -->|"speaks result"| node_tts
node_model_routes -->|"registers models"| node_model_service
node_model_service -->|"syncs registry"| node_model_records
node_model_routes -->|"stores model metadata"| node_postgres
node_dataset_routes -->|"validates CSV"| node_dataset_service
node_dataset_service -->|"records datasets"| node_postgres
node_dataset_service -->|"queues dataset job"| node_dataset_worker
node_dataset_worker -->|"uses broker"| node_redis
node_api_app -.->|"supports fusion jobs"| node_fusion_worker
node_api_app -->|"initializes connection"| node_mongo
node_api_app -->|"uses platform services"| node_core

click node_glove "https://github.com/lovelypoet/silentvoix/blob/main/firmware/esp32_glove.cpp"
click node_browser "https://github.com/lovelypoet/silentvoix/blob/main/vue-next/src/App.vue"
click node_sensor_capture "https://github.com/lovelypoet/silentvoix/blob/main/vue-next/src/composables/useSensorCaptureSession.js"
click node_api_app "https://github.com/lovelypoet/silentvoix/blob/main/api/main.py"
click node_live_stream "https://github.com/lovelypoet/silentvoix/blob/main/api/routes/liveWS.py"
click node_gesture_api "https://github.com/lovelypoet/silentvoix/blob/main/api/routes/gestures_predict.py"
click node_auth "https://github.com/lovelypoet/silentvoix/blob/main/api/routes/auth_routes.py"
click node_core "https://github.com/lovelypoet/silentvoix/tree/main/api/core"
click node_model_routes "https://github.com/lovelypoet/silentvoix/blob/main/api/routes/model_library_routes.py"
click node_model_service "https://github.com/lovelypoet/silentvoix/blob/main/services/model_library_service.py"
click node_model_records "https://github.com/lovelypoet/silentvoix/blob/main/services/models/model_service.py"
click node_dataset_routes "https://github.com/lovelypoet/silentvoix/blob/main/api/routes/admin_csv_library_routes.py"
click node_dataset_service "https://github.com/lovelypoet/silentvoix/blob/main/services/datasets/dataset_service.py"
click node_postgres "https://github.com/lovelypoet/silentvoix/blob/main/db/models.py"
click node_mongo "https://github.com/lovelypoet/silentvoix/blob/main/api/core/database.py"
click node_pytorch "https://github.com/lovelypoet/silentvoix/blob/main/ml-pytorch/app.py"
click node_tensorflow "https://github.com/lovelypoet/silentvoix/blob/main/ml-tensorflow/app.py"
click node_tts "https://github.com/lovelypoet/silentvoix/blob/main/services/tts_service.py"
click node_dataset_worker "https://github.com/lovelypoet/silentvoix/blob/main/workers/tasks/dataset_tasks.py"
click node_fusion_worker "https://github.com/lovelypoet/silentvoix/blob/main/worker-early-fusion/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_glove,node_browser,node_sensor_capture,node_user toneBlue
class node_api_app,node_live_stream,node_gesture_api,node_auth,node_core toneAmber
class node_pytorch,node_tensorflow,node_tts toneMint
class node_model_routes,node_model_service,node_model_records,node_dataset_routes,node_dataset_service,node_postgres,node_mongo toneRose
class node_dataset_worker,node_fusion_worker,node_redis toneIndigo
```

## Components

### Input and capture

- [ESP32 glove](https://github.com/lovelypoet/silentvoix/blob/main/firmware/esp32_glove.cpp): `firmware/esp32_glove.cpp`
- [Vue application](https://github.com/lovelypoet/silentvoix/blob/main/vue-next/src/App.vue): `vue-next/src/App.vue`
- [Sensor capture](https://github.com/lovelypoet/silentvoix/blob/main/vue-next/src/composables/useSensorCaptureSession.js): `vue-next/src/composables/useSensorCaptureSession.js`

### API and services

- [FastAPI app](https://github.com/lovelypoet/silentvoix/blob/main/api/main.py): `api/main.py`
- [Live WebSocket](https://github.com/lovelypoet/silentvoix/blob/main/api/routes/liveWS.py): `api/routes/liveWS.py`
- [Gesture prediction](https://github.com/lovelypoet/silentvoix/blob/main/api/routes/gestures_predict.py): `api/routes/gestures_predict.py`
- [Authentication](https://github.com/lovelypoet/silentvoix/blob/main/api/routes/auth_routes.py): `api/routes/auth_routes.py`
- [API core](https://github.com/lovelypoet/silentvoix/tree/main/api/core): `api/core`

### Inference and speech

- [PyTorch runtime](https://github.com/lovelypoet/silentvoix/blob/main/ml-pytorch/app.py): `ml-pytorch/app.py`
- [TensorFlow runtime](https://github.com/lovelypoet/silentvoix/blob/main/ml-tensorflow/app.py): `ml-tensorflow/app.py`
- [Text to speech](https://github.com/lovelypoet/silentvoix/blob/main/services/tts_service.py): `services/tts_service.py`

### Datasets and models

- [Model library API](https://github.com/lovelypoet/silentvoix/blob/main/api/routes/model_library_routes.py): `api/routes/model_library_routes.py`
- [Model registry](https://github.com/lovelypoet/silentvoix/blob/main/services/model_library_service.py): `services/model_library_service.py`
- [Model records](https://github.com/lovelypoet/silentvoix/blob/main/services/models/model_service.py): `services/models/model_service.py`
- [CSV library API](https://github.com/lovelypoet/silentvoix/blob/main/api/routes/admin_csv_library_routes.py): `api/routes/admin_csv_library_routes.py`
- [Dataset service](https://github.com/lovelypoet/silentvoix/blob/main/services/datasets/dataset_service.py): `services/datasets/dataset_service.py`
- [PostgreSQL](https://github.com/lovelypoet/silentvoix/blob/main/db/models.py): `db/models.py`
- [MongoDB](https://github.com/lovelypoet/silentvoix/blob/main/api/core/database.py): `api/core/database.py`

### Async processing

- [Dataset worker](https://github.com/lovelypoet/silentvoix/blob/main/workers/tasks/dataset_tasks.py): `workers/tasks/dataset_tasks.py`
- [Early fusion worker](https://github.com/lovelypoet/silentvoix/blob/main/worker-early-fusion/app.py): `worker-early-fusion/app.py`
- **Redis broker**

### Other components

- **User**

## Connections

- User → Vue application: uses (evidence: `README.md`)
- ESP32 glove → Live WebSocket: streams sensor data (evidence: `README.md`)
- Vue application → Sensor capture: captures sensors (evidence: `README.md`)
- Vue application → FastAPI app: calls API (evidence: `README.md`)
- FastAPI app → Live WebSocket: mounts routes (evidence: `api/main.py`)
- FastAPI app → Gesture prediction: mounts routes (evidence: `api/main.py`)
- FastAPI app → Authentication: mounts routes (evidence: `api/main.py`)
- Live WebSocket → Gesture prediction: requests prediction
- Gesture prediction → PyTorch runtime: dispatches inference
- Gesture prediction → TensorFlow runtime: dispatches inference
- PyTorch runtime → Vue application: returns prediction (evidence: `README.md`)
- TensorFlow runtime → Vue application: returns prediction (evidence: `README.md`)
- Vue application → Text to speech: speaks result (evidence: `README.md`)
- Model library API → Model registry: registers models (evidence: `api/routes/model_library_routes.py`)
- Model registry → Model records: syncs registry (evidence: `services/models/model_service.py`)
- Model library API → PostgreSQL: stores model metadata
- CSV library API → Dataset service: validates CSV (evidence: `api/routes/admin_csv_library_routes.py`)
- Dataset service → PostgreSQL: records datasets (evidence: `services/datasets/dataset_service.py`)
- Dataset service → Dataset worker: queues dataset job (evidence: `services/datasets/dataset_service.py`)
- Dataset worker → Redis broker: uses broker (evidence: `workers/tasks/dataset_tasks.py`)
- FastAPI app → Early fusion worker: supports fusion jobs
- FastAPI app → MongoDB: initializes connection (evidence: `api/main.py`)
- FastAPI app → API core: uses platform services (evidence: `api/main.py`)
