# dhanvith-4/25btrad115_dhanvith architecture

> Architecture diagram of the GitHub repository dhanvith-4/25btrad115_dhanvith, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhanvith-4/25btrad115_dhanvith
- Repository: https://github.com/dhanvith-4/25btrad115_dhanvith

## Overview

SignBridge AI translates live Indian Sign Language gestures into confirmed words and sentences. A user opens the browser interface, starts a camera, and sends hand landmarks over a WebSocket; the backend normalizes frames, buffers sequences, predicts signs, and stabilizes them into sentence state for the UI. The repository also includes gesture-data collection and model training, plus vocabulary, status, and text-to-speech endpoints. The excerpts do not establish all frontend-to-API wiring, so those links are omitted where unsupported.

Main entry points: [`25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py`](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/websocket_handler.py) (WebSocket handler), [`25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js`](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/frontend/script.js) (Web interface), [`25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/main.py) (FastAPI application), [`25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/dataset_collector.py`](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/model/dataset_collector.py) (Gesture data collector)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_client["Browser experience"]
  node_browser["Web interface<br/>[script.js]"]
  node_camera["Camera stream"]
  node_mediapipe["MediaPipe Hands"]
end

subgraph group_runtime["Translation runtime"]
  node_ws["WebSocket handler"]
  node_preprocess["Landmark preprocessing<br/>[preprocessing.py]"]
  node_buffer["Sequence buffer<br/>[preprocessing.py]"]
  node_predictor["Sign predictor<br/>[predictor.py]"]
  node_sentence["Sentence builder"]
end

subgraph group_modeling["Gesture modeling"]
  node_model[("Trained LSTM<br/>[trained_model.pt]")]
  node_detector["Landmark detector<br/>[hand_detector.py]"]
  node_collector["Gesture data collector"]
  node_training["Model training<br/>[train_model.py]"]
end

subgraph group_services["API services"]
  node_backend["FastAPI application<br/>[main.py]"]
  node_vocabulary["Vocabulary API<br/>[main.py]"]
  node_tts["Text-to-speech API<br/>[main.py]"]
end

node_user(("Sign-language user"))
node_speech["Speech synthesis"]

node_user -->|"uses"| node_browser
node_browser -->|"requests video"| node_camera
node_browser -->|"initializes"| node_mediapipe
node_backend -->|"serves frontend"| node_browser
node_backend -->|"routes to"| node_ws
node_browser -.->|"streams landmarks"| node_ws
node_ws -->|"normalizes frames"| node_preprocess
node_ws -->|"appends frames"| node_buffer
node_ws -->|"requests inference"| node_predictor
node_ws -->|"updates sentence"| node_sentence
node_predictor -->|"loads weights"| node_model
node_collector -->|"uses detector"| node_detector
node_collector -->|"uses features"| node_preprocess
node_training -->|"loads dataset"| node_collector
node_training -->|"saves weights"| node_model
node_backend -->|"provides endpoints"| node_vocabulary
node_backend -->|"provides endpoint"| node_tts
node_tts -->|"synthesizes speech"| node_speech

click node_browser "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/frontend/script.js"
click node_backend "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/main.py"
click node_ws "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/websocket_handler.py"
click node_preprocess "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/preprocessing.py"
click node_buffer "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/preprocessing.py"
click node_predictor "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/predictor.py"
click node_sentence "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/sentence_builder.py"
click node_model "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/model/trained_model.pt"
click node_detector "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/hand_detector.py"
click node_collector "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/model/dataset_collector.py"
click node_training "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/model/train_model.py"
click node_vocabulary "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/main.py"
click node_tts "https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/main.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_browser,node_camera,node_mediapipe,node_user toneBlue
class node_ws,node_preprocess,node_buffer,node_predictor,node_sentence toneAmber
class node_model,node_detector,node_collector,node_training toneMint
class node_backend,node_vocabulary,node_tts toneRose
class node_speech toneIndigo
```

## Components

### Browser experience

- [Web interface](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/frontend/script.js): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js`
- **Camera stream**
- **MediaPipe Hands**

### Translation runtime

- [WebSocket handler](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/websocket_handler.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py`
- [Landmark preprocessing](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/preprocessing.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/preprocessing.py`
- [Sequence buffer](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/preprocessing.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/preprocessing.py`
- [Sign predictor](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/predictor.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/predictor.py`
- [Sentence builder](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/sentence_builder.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/sentence_builder.py`

### Gesture modeling

- [Trained LSTM](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/model/trained_model.pt): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/trained_model.pt`
- [Landmark detector](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/hand_detector.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/hand_detector.py`
- [Gesture data collector](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/model/dataset_collector.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/dataset_collector.py`
- [Model training](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/model/train_model.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/train_model.py`

### API services

- [FastAPI application](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/main.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`
- [Vocabulary API](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/main.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`
- [Text-to-speech API](https://github.com/dhanvith-4/25btrad115_dhanvith/blob/main/25BTRAD115_EXPERIENTIAL%20LEARNING%20ACTIVITY/backend/main.py): `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`

### Other components

- **Sign-language user**
- **Speech synthesis**

## Connections

- Sign-language user → Web interface: uses (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js`)
- Web interface → Camera stream: requests video (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js`)
- Web interface → MediaPipe Hands: initializes (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/frontend/script.js`)
- FastAPI application → Web interface: serves frontend (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`)
- FastAPI application → WebSocket handler: routes to (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`)
- Web interface → WebSocket handler: streams landmarks
- WebSocket handler → Landmark preprocessing: normalizes frames (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py`)
- WebSocket handler → Sequence buffer: appends frames (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py`)
- WebSocket handler → Sign predictor: requests inference (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py`)
- WebSocket handler → Sentence builder: updates sentence (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/websocket_handler.py`)
- Sign predictor → Trained LSTM: loads weights (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/predictor.py`)
- Gesture data collector → Landmark detector: uses detector (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/dataset_collector.py`)
- Gesture data collector → Landmark preprocessing: uses features (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/dataset_collector.py`)
- Model training → Gesture data collector: loads dataset (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/train_model.py`)
- Model training → Trained LSTM: saves weights (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/model/train_model.py`)
- FastAPI application → Vocabulary API: provides endpoints (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`)
- FastAPI application → Text-to-speech API: provides endpoint (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`)
- Text-to-speech API → Speech synthesis: synthesizes speech (evidence: `25BTRAD115_EXPERIENTIAL LEARNING ACTIVITY/backend/main.py`)