How gill003/smart-sign-language-translator-glove works
Smart Sign Language Glove translates hand motion into gesture predictions, text/speech, and a live browser visualization. The principal flow is sensor acquisition, queued motion capture, feature preparation, KNN classification, then server-sent updates to the browser; the browser animates a 3D hand and can speak predictions. The README also documents training and free-play modes and HTTP controls. Firmware behavior is concentrated in src/main.cpp; browser behavior is in data/main.js. Some server and client details are documented but their full implementation was not sampled.
Main entry points: main.js Web client · main.cpp HTTP controls · SensorData.csv Training dataset · hand.glb 3D hand model
Components 17
Wearable sensing
- Flex sensors
- MPU6050 IMU
Embedded processing
- Sensor acquisition
src/main.cpp - Motion queues
src/main.cpp - Gesture features
src/main.cpp - Training dataset
data/SensorData.csv - LittleFS storage
src/main.cpp - KNN classifier
src/main.cpp - HTTP controls
src/main.cpp - SSE event server
src/main.cpp
Browser experience
- Web client
data/main.js - 3D hand model
data/hand.glb - 3D visualization
data/main.js
Translation outputs
- Prediction display
data/main.js - Speech synthesis
data/main.js
Other components
- Glove user
- Web Speech API
Connections 22
- Glove user → Flex sensors: bends fingers
- Glove user → MPU6050 IMU: moves hand
- Flex sensors → Sensor acquisition: sensor readings
- MPU6050 IMU → Sensor acquisition: motion readings
- Sensor acquisition → Motion queues: enqueues samples
- Motion queues → Gesture features: provides window
- Gesture features → KNN classifier: classifies features
- LittleFS storage → Training dataset: reads training data
- Training dataset → KNN classifier: training examples
- HTTP controls → LittleFS storage: loads dataset
- HTTP controls → Motion queues: resets queues
- HTTP controls → KNN classifier: selects prediction mode
- KNN classifier → SSE event server: sends prediction
- Sensor acquisition → SSE event server: sends sensor events
- Web client → HTTP controls: requests controls
- Web client → SSE event server: subscribes to events
- SSE event server → Web client: streams updates
- Web client → 3D hand model: loads model
- Web client → 3D visualization: sets up scene
- 3D visualization → Prediction display: displays prediction
- 3D visualization → Speech synthesis: speaks prediction
- Speech synthesis → Web Speech API: requests speech