# redah13/real-time-asl-detection-cnn architecture

> Architecture diagram of the GitHub repository redah13/real-time-asl-detection-cnn, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/redah13/real-time-asl-detection-cnn
- Repository: https://github.com/redah13/real-time-asl-detection-cnn

## Overview

This repository implements a real-time ASL gesture recognizer, plus scripts to collect, check, train, and evaluate its image dataset. The main user workflow is webcam frames → hand localization and image preprocessing → CNN prediction → smoothed gesture text displayed in a window. Training and evaluation form a separate model-development workflow. Dataset and model artifacts are referenced by the scripts but are not present in the supplied tree; connections reflect only the shown code, not inspected artifacts.

Main entry points: [`app.py`](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py) (Live recognizer), [`train_cnn_model.py`](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/train_cnn_model.py) (CNN training), [`model_evaluate.py`](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/model_evaluate.py) (Model evaluation), [`collecter_rgb.py`](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/collecter_rgb.py) (RGB collector)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_capture["Capture and recognition"]
  node_collector["RGB collector<br/>[collecter_rgb.py]"]
  node_landmarks["Hand landmarks<br/>[collecter_rgb.py]"]
  node_live["Live recognizer<br/>[app.py]"]
  node_detector["Hand detector<br/>[app.py]"]
  node_preprocess["ROI preprocessing<br/>[app.py]"]
  node_classifier["CNN classifier<br/>[app.py]"]
  node_smoother["Gesture smoothing<br/>[app.py]"]
  node_display["Gesture display<br/>[app.py]"]
end

subgraph group_modeldev["Dataset and model development"]
  node_checker["Dataset checker<br/>[verifier_rgb.py]"]
  node_trainer["CNN training<br/>[train_cnn_model.py]"]
  node_evaluator["Model evaluation<br/>[model_evaluate.py]"]
end

subgraph group_artifacts["Runtime and outputs"]
  node_dataset[("RGB dataset")]
  node_handtask["MediaPipe hand model"]
  node_cnnmodel["Trained CNN"]
  node_metrics["Evaluation metrics"]
end

node_user(("User"))
node_camera["Webcam"]

node_user -->|"starts capture"| node_collector
node_camera -->|"provides frames"| node_collector
node_collector -->|"detects hands"| node_landmarks
node_collector -->|"saves RGB crops"| node_dataset
node_user -->|"runs check"| node_checker
node_checker -->|"checks images"| node_dataset
node_user -->|"starts training"| node_trainer
node_trainer -->|"loads images"| node_dataset
node_trainer -->|"saves weights"| node_cnnmodel
node_trainer -->|"saves learning curves"| node_metrics
node_user -->|"runs evaluation"| node_evaluator
node_evaluator -->|"loads test images"| node_dataset
node_evaluator -->|"loads weights"| node_cnnmodel
node_evaluator -->|"saves confusion matrix"| node_metrics
node_user -->|"starts recognizer"| node_live
node_camera -->|"provides frames"| node_live
node_live -->|"runs detection"| node_detector
node_detector -->|"loads task model"| node_handtask
node_live -->|"loads weights"| node_cnnmodel
node_live -->|"crops and resizes"| node_preprocess
node_preprocess -->|"passes normalized ROI"| node_classifier
node_classifier -->|"passes predictions"| node_smoother
node_smoother -->|"updates gesture text"| node_display

click node_collector "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/collecter_rgb.py"
click node_landmarks "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/collecter_rgb.py"
click node_checker "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/verifier_rgb.py"
click node_trainer "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/train_cnn_model.py"
click node_evaluator "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/model_evaluate.py"
click node_live "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py"
click node_detector "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py"
click node_handtask "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/mediapipe_models/hand_landmarker.task"
click node_preprocess "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py"
click node_classifier "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py"
click node_smoother "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py"
click node_display "https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py"
click node_metrics "https://github.com/redah13/real-time-asl-detection-cnn/tree/main/metrics"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_collector,node_landmarks,node_live,node_detector,node_preprocess,node_classifier,node_smoother,node_display,node_user toneBlue
class node_checker,node_trainer,node_evaluator toneAmber
class node_dataset,node_handtask,node_cnnmodel,node_metrics toneMint
class node_camera toneIndigo
```

## Components

### Capture and recognition

- [RGB collector](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/collecter_rgb.py): `collecter_rgb.py`
- [Hand landmarks](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/collecter_rgb.py): `collecter_rgb.py`
- [Live recognizer](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py): `app.py`
- [Hand detector](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py): `app.py`
- [ROI preprocessing](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py): `app.py`
- [CNN classifier](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py): `app.py`
- [Gesture smoothing](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py): `app.py`
- [Gesture display](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/app.py): `app.py`

### Dataset and model development

- [Dataset checker](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/verifier_rgb.py): `verifier_rgb.py`
- [CNN training](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/train_cnn_model.py): `train_cnn_model.py`
- [Model evaluation](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/model_evaluate.py): `model_evaluate.py`

### Runtime and outputs

- **RGB dataset**
- [MediaPipe hand model](https://github.com/redah13/real-time-asl-detection-cnn/blob/main/mediapipe_models/hand_landmarker.task): `mediapipe_models/hand_landmarker.task`
- **Trained CNN**
- [Evaluation metrics](https://github.com/redah13/real-time-asl-detection-cnn/tree/main/metrics): `metrics`

### Other components

- **User**
- **Webcam**

## Connections

- User → RGB collector: starts capture (evidence: `collecter_rgb.py`)
- Webcam → RGB collector: provides frames (evidence: `collecter_rgb.py`)
- RGB collector → Hand landmarks: detects hands (evidence: `collecter_rgb.py`)
- RGB collector → RGB dataset: saves RGB crops (evidence: `collecter_rgb.py`)
- User → Dataset checker: runs check (evidence: `verifier_rgb.py`)
- Dataset checker → RGB dataset: checks images (evidence: `verifier_rgb.py`)
- User → CNN training: starts training (evidence: `train_cnn_model.py`)
- CNN training → RGB dataset: loads images (evidence: `train_cnn_model.py`)
- CNN training → Trained CNN: saves weights (evidence: `train_cnn_model.py`)
- CNN training → Evaluation metrics: saves learning curves (evidence: `train_cnn_model.py`)
- User → Model evaluation: runs evaluation (evidence: `model_evaluate.py`)
- Model evaluation → RGB dataset: loads test images (evidence: `model_evaluate.py`)
- Model evaluation → Trained CNN: loads weights (evidence: `model_evaluate.py`)
- Model evaluation → Evaluation metrics: saves confusion matrix (evidence: `model_evaluate.py`)
- User → Live recognizer: starts recognizer (evidence: `app.py`)
- Webcam → Live recognizer: provides frames (evidence: `app.py`)
- Live recognizer → Hand detector: runs detection (evidence: `app.py`)
- Hand detector → MediaPipe hand model: loads task model (evidence: `app.py`)
- Live recognizer → Trained CNN: loads weights (evidence: `app.py`)
- Live recognizer → ROI preprocessing: crops and resizes (evidence: `app.py`)
- ROI preprocessing → CNN classifier: passes normalized ROI (evidence: `app.py`)
- CNN classifier → Gesture smoothing: passes predictions (evidence: `app.py`)
- Gesture smoothing → Gesture display: updates gesture text (evidence: `app.py`)
