# asharibkhannn/asl-sign-language-recognition architecture

> Architecture diagram of the GitHub repository asharibkhannn/asl-sign-language-recognition, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/asharibkhannn/asl-sign-language-recognition
- Repository: https://github.com/asharibkhannn/asl-sign-language-recognition

## Overview

##### Project Overview & Type
- This is a real-time ASL hand sign detection application using deep learning (specifically an LSTM model) and computer vision.
- It is a desktop application with a graphical interface that processes webcam input, performs image preprocessing (such as skin segmentation), and classifies ASL hand signs with a trained model.
- There’s also an integrated performance evaluation tool that computes metrics and visualizations, useful for tuning and benchmarking the system.

##### Main Components
- User/Environment:
- Webcam: The source of video frames fed into the application.
- User Interface (GUI): Provided by asl_camera_gui.py, which displays the video stream, the confidence meter, and the adjustable threshold.
- Data Preprocessing Module:
- Incorporates hand detection and skin segmentation performed on each captured frame.
- Machine Learning Component:
- ASL Classifier (using the deep learning model in asl_classifier_finetuned.h5).
- Class Mapping: The mapping file (class_mapping.npy) that translates model output indices to ASL letters.
- Launch Scripts:
- Windows: run_asl.bat (for launching the application), run_performance_evaluation.bat.
- Linux/Mac: run_asl.py and run_performance_evaluation.py.
- Performance Evaluation Module:
- Evaluation script(s) (evaluate_model_performance.py if present, or integrated code within run_performance_evaluation.py) that measures and visualizes accuracy, precision, recall, F1 score, inference times, etc.
- Outputs (e.g., reports and visualizations) that might be stored in a generated folder like performance_results.
- Dependencies/External Libraries:
- TensorFlow, OpenCV, NumPy, and Pillow, which underpin the core functionalities.

- The preprocessed image is fed into the LSTM-based classifier model (asl_classifier_finetuned.h5).
- The output is then mapped via the “Class Mapping” (class_mapping.npy) to obtain the corresponding ASL letter.
- Next, connect this result to the “User Interface” block where the recognized sign is visually displayed along with a confidence meter and an adjustable threshold slider.
- Separately, include a branch for “Performance Evaluation”:
- A separate flow when the performance evaluation scripts are run. This flow captures the same or additional data (like inference time, per-class analysis, confusion matrix) and outputs detailed reports.
- This component should connect to a “Results Visualization/Report” block, indicating that the data produced is visualized or stored.

##### Architectural Patterns and Technologies
- A modular design: front-end (GUI), processing (image preprocessing and model inference), and evaluation (performance metrics). This ensures separation of concerns and easier troubleshooting.
- The data flow is sequential (from image capture, through preprocessing, to inference, and finally display), while performance evaluation runs parallel when specifically invoked.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Launcher Scripts for Main Application
    LS1["Launch Script (Python)"]:::launcher
    LS2["Launch Script (BAT)"]:::launcher
    LS1 --> GUI
    LS2 --> GUI

    %% Webcam Input (External)
    W(("Webcam Input")):::webcam
    W --> GUI

    %% Main Application Flow
    GUI["ASL Camera GUI (Preprocessing & Display)"]:::gui
    GUI --> ML["ML Model Inference"]:::ml
    ML --> CM["Class Mapping"]:::ml
    CM --> GUI

    %% Performance Evaluation Branch
    LSE["Launch Script (Performance Eval)"]:::launcher
    LSE --> PE["Performance Evaluation Module"]:::eval
    ML --> PE
    PE --> RV["Results Visualization/Report"]:::eval

    %% Click Events
    click LS1 "https://github.com/asharibkhannn/asl-sign-language-recognition/blob/main/run_asl.py"
    click LS2 "https://github.com/asharibkhannn/asl-sign-language-recognition/blob/main/run_asl.bat"
    click GUI "https://github.com/asharibkhannn/asl-sign-language-recognition/blob/main/asl_camera_gui.py"
    click ML "https://github.com/asharibkhannn/asl-sign-language-recognition/blob/main/asl_classifier_finetuned.h5"
    click CM "https://github.com/asharibkhannn/asl-sign-language-recognition/blob/main/class_mapping.npy"

    %% Styles
    classDef webcam fill:#FFD700,stroke:#B8860B,stroke-width:2px;
    classDef gui fill:#87CEFA,stroke:#4682B4,stroke-width:2px;
    classDef ml fill:#90EE90,stroke:#228B22,stroke-width:2px;
    classDef eval fill:#FFB6C1,stroke:#FF69B4,stroke-width:2px;
    classDef launcher fill:#D3D3D3,stroke:#696969,stroke-width:2px;
```
