dhanvith-4/25btrad115_dhanvith

How dhanvith-4/25btrad115_dhanvith works

SignBridge AI translates live Indian Sign Language gestures into confirmed words and sentences. A user opens the browser interface, starts a camera, and sends hand landmarks over a WebSocket; the backend normalizes frames, buffers sequences, predicts signs, and stabilizes them into sentence state for the UI. The repository also includes gesture-data collection and model training, plus vocabulary, status, and text-to-speech endpoints. The excerpts do not establish all frontend-to-API wiring, so those links are omitted where unsupported.

Main entry points: websocket_handler.py WebSocket handler · script.js Web interface · main.py FastAPI application · dataset_collector.py Gesture data collector

Components 17

Browser experience

Web interface 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js
Camera stream
MediaPipe Hands

Translation runtime

WebSocket handler 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py
Landmark preprocessing 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/preprocessing.py
Sequence buffer 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/preprocessing.py
Sign predictor 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/predictor.py
Sentence builder 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/sentence_builder.py

Gesture modeling

Trained LSTM 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/trained_model.pt
Landmark detector 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/hand_detector.py
Gesture data collector 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/dataset_collector.py
Model training 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/train_model.py

API services

FastAPI application 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py
Vocabulary API 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py
Text-to-speech API 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py

Other components

Sign-language user
Speech synthesis

Connections 18

Sign-language user → Web interface: uses 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js
Web interface → Camera stream: requests video 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js
Web interface → MediaPipe Hands: initializes 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js
FastAPI application → Web interface: serves frontend 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py
FastAPI application → WebSocket handler: routes to 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py
Web interface → WebSocket handler: streams landmarks
WebSocket handler → Landmark preprocessing: normalizes frames 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py
WebSocket handler → Sequence buffer: appends frames 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py
WebSocket handler → Sign predictor: requests inference 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py
WebSocket handler → Sentence builder: updates sentence 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py
Sign predictor → Trained LSTM: loads weights 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/predictor.py
Gesture data collector → Landmark detector: uses detector 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/dataset_collector.py
Gesture data collector → Landmark preprocessing: uses features 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/dataset_collector.py
Model training → Gesture data collector: loads dataset 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/train_model.py
Model training → Trained LSTM: saves weights 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/train_model.py
FastAPI application → Vocabulary API: provides endpoints 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py
FastAPI application → Text-to-speech API: provides endpoint 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py
Text-to-speech API → Speech synthesis: synthesizes speech 25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py

Drawn by GitDiagram on November 14, 2026 from the repository's file tree, README and source files. Read as Markdown ·