How syiyamkumar/isl-sign-language-translator works
This repository builds an ISL gesture-recognition workflow: video frames become 225-dimensional landmark sequences, which can be used to train a BiLSTM and then classify live or uploaded video. The two inference interfaces display labels and confidence; separate training and evaluation scripts support the model lifecycle. Sentence-level data preparation is present, while the documented sentence generation and multilingual translation are downstream aspirations, not implemented runtime stages. Edges reflect the shown calls and reads/writes; unsampled integrations are omitted.
Main entry points: app_streamlit.py Web Dashboard · app_webcam.py Desktop Inference · config.py Runtime Configuration · extract_features.py Feature Extraction
Components 13
Feature Preparation
- Feature Extraction
extract_features.py - Runtime Configuration
config.py
Model Lifecycle
- Dataset Loader
dataset.py - Model Training
train.py - BiLSTM Classifier
model.py - Model Evaluation
evaluate.py
Recognition Interfaces
- Desktop Inference
app_webcam.py - Web Dashboard
app_streamlit.py
Local Data
- Feature Sequences
- Model Checkpoint
Other components
- Camera
- Video Upload
- MediaPipe Holistic
Connections 26
- Camera → Desktop Inference: provides frames
app_webcam.py - Camera → Web Dashboard: provides frames
app_streamlit.py - Video Upload → Web Dashboard: provides video
app_streamlit.py - Feature Extraction → MediaPipe Holistic: processes frames
extract_features.py - Feature Extraction → Runtime Configuration: uses paths
extract_features.py - Feature Extraction → Feature Sequences: writes sequences
extract_features.py - Desktop Inference → MediaPipe Holistic: processes frames
app_webcam.py - Desktop Inference → Feature Extraction: extracts keypoints
app_webcam.py - Desktop Inference → BiLSTM Classifier: classifies sequence
app_webcam.py - Desktop Inference → Model Checkpoint: loads weights
app_webcam.py - Desktop Inference → Runtime Configuration: uses settings
app_webcam.py - Web Dashboard → MediaPipe Holistic: processes frames
app_streamlit.py - Web Dashboard → Feature Extraction: extracts keypoints
app_streamlit.py - Web Dashboard → BiLSTM Classifier: classifies sequence
app_streamlit.py - Web Dashboard → Model Checkpoint: loads weights
app_streamlit.py - Web Dashboard → Runtime Configuration: uses settings
app_streamlit.py - Dataset Loader → Feature Sequences: reads sequences
dataset.py - Dataset Loader → Runtime Configuration: uses paths
dataset.py - Model Training → Dataset Loader: loads samples
train.py - Model Training → BiLSTM Classifier: trains
train.py - Model Training → Model Checkpoint: saves weights
train.py - Model Training → Runtime Configuration: uses settings
train.py - Model Evaluation → Dataset Loader: loads samples
evaluate.py - Model Evaluation → BiLSTM Classifier: evaluates
evaluate.py - Model Evaluation → Model Checkpoint: loads weights
evaluate.py - Model Evaluation → Runtime Configuration: uses settings
evaluate.py