How lovelypoet/silentvoix works
SilentVoix is a multimodal sign-language recognition platform: glove or camera input is captured, sent through API/runtime inference, and turned into a gesture result and speech. The wider product also supports dataset validation and capture, model upload/activation, fusion workflows, and monitoring. The README documents several runtime and async capabilities; source excerpts confirm selected API, model-library, database, and worker relationships, but do not expose every route or dispatch.
Main entry points: main.py FastAPI app · App.vue Vue application · gestures_predict.py Gesture prediction · dataset_service.py Dataset service
Components 22
Input and capture
- ESP32 glove
firmware/esp32_glove.cpp - Vue application
vue-next/src/App.vue - Sensor capture
vue-next/src/composables/useSensorCaptureSession.js
API and services
- FastAPI app
api/main.py - Live WebSocket
api/routes/liveWS.py - Gesture prediction
api/routes/gestures_predict.py - Authentication
api/routes/auth_routes.py - API core
api/core
Inference and speech
- PyTorch runtime
ml-pytorch/app.py - TensorFlow runtime
ml-tensorflow/app.py - Text to speech
services/tts_service.py
Datasets and models
- Model library API
api/routes/model_library_routes.py - Model registry
services/model_library_service.py - Model records
services/models/model_service.py - CSV library API
api/routes/admin_csv_library_routes.py - Dataset service
services/datasets/dataset_service.py - PostgreSQL
db/models.py - MongoDB
api/core/database.py
Async processing
- Dataset worker
workers/tasks/dataset_tasks.py - Early fusion worker
worker-early-fusion/app.py - Redis broker
Other components
- User
Connections 23
- User → Vue application: uses
README.md - ESP32 glove → Live WebSocket: streams sensor data
README.md - Vue application → Sensor capture: captures sensors
README.md - Vue application → FastAPI app: calls API
README.md - FastAPI app → Live WebSocket: mounts routes
api/main.py - FastAPI app → Gesture prediction: mounts routes
api/main.py - FastAPI app → Authentication: mounts routes
api/main.py - Live WebSocket → Gesture prediction: requests prediction
- Gesture prediction → PyTorch runtime: dispatches inference
- Gesture prediction → TensorFlow runtime: dispatches inference
- PyTorch runtime → Vue application: returns prediction
README.md - TensorFlow runtime → Vue application: returns prediction
README.md - Vue application → Text to speech: speaks result
README.md - Model library API → Model registry: registers models
api/routes/model_library_routes.py - Model registry → Model records: syncs registry
services/models/model_service.py - Model library API → PostgreSQL: stores model metadata
- CSV library API → Dataset service: validates CSV
api/routes/admin_csv_library_routes.py - Dataset service → PostgreSQL: records datasets
services/datasets/dataset_service.py - Dataset service → Dataset worker: queues dataset job
services/datasets/dataset_service.py - Dataset worker → Redis broker: uses broker
workers/tasks/dataset_tasks.py - FastAPI app → Early fusion worker: supports fusion jobs
- FastAPI app → MongoDB: initializes connection
api/main.py - FastAPI app → API core: uses platform services
api/main.py