How redah13/real-time-asl-detection-cnn works
This repository implements a real-time ASL gesture recognizer, plus scripts to collect, check, train, and evaluate its image dataset. The main user workflow is webcam frames → hand localization and image preprocessing → CNN prediction → smoothed gesture text displayed in a window. Training and evaluation form a separate model-development workflow. Dataset and model artifacts are referenced by the scripts but are not present in the supplied tree; connections reflect only the shown code, not inspected artifacts.
Main entry points: app.py Live recognizer · train_cnn_model.py CNN training · model_evaluate.py Model evaluation · collecter_rgb.py RGB collector
Components 17
Capture and recognition
- RGB collector
collecter_rgb.py - Hand landmarks
collecter_rgb.py - Live recognizer
app.py - Hand detector
app.py - ROI preprocessing
app.py - CNN classifier
app.py - Gesture smoothing
app.py - Gesture display
app.py
Dataset and model development
- Dataset checker
verifier_rgb.py - CNN training
train_cnn_model.py - Model evaluation
model_evaluate.py
Runtime and outputs
- RGB dataset
- MediaPipe hand model
mediapipe_models/hand_landmarker.task - Trained CNN
- Evaluation metrics
metrics
Other components
- User
- Webcam
Connections 23
- User → RGB collector: starts capture
collecter_rgb.py - Webcam → RGB collector: provides frames
collecter_rgb.py - RGB collector → Hand landmarks: detects hands
collecter_rgb.py - RGB collector → RGB dataset: saves RGB crops
collecter_rgb.py - User → Dataset checker: runs check
verifier_rgb.py - Dataset checker → RGB dataset: checks images
verifier_rgb.py - User → CNN training: starts training
train_cnn_model.py - CNN training → RGB dataset: loads images
train_cnn_model.py - CNN training → Trained CNN: saves weights
train_cnn_model.py - CNN training → Evaluation metrics: saves learning curves
train_cnn_model.py - User → Model evaluation: runs evaluation
model_evaluate.py - Model evaluation → RGB dataset: loads test images
model_evaluate.py - Model evaluation → Trained CNN: loads weights
model_evaluate.py - Model evaluation → Evaluation metrics: saves confusion matrix
model_evaluate.py - User → Live recognizer: starts recognizer
app.py - Webcam → Live recognizer: provides frames
app.py - Live recognizer → Hand detector: runs detection
app.py - Hand detector → MediaPipe hand model: loads task model
app.py - Live recognizer → Trained CNN: loads weights
app.py - Live recognizer → ROI preprocessing: crops and resizes
app.py - ROI preprocessing → CNN classifier: passes normalized ROI
app.py - CNN classifier → Gesture smoothing: passes predictions
app.py - Gesture smoothing → Gesture display: updates gesture text
app.py