# arshad221b/sign-language-recognition architecture

> Architecture diagram of the GitHub repository arshad221b/sign-language-recognition, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/arshad221b/sign-language-recognition
- Repository: https://github.com/arshad221b/sign-language-recognition

## Overview

##### Project type and purpose
- This is a machine-learning based image-processing tool for sign-language recognition. It combines OpenCV for real-time ROI (region of interest) extraction with deep-learning (CNN) models (Keras/TensorFlow or PyTorch) to classify hand‐sign images.

##### File structure
- Top-level scripts & notebooks:
- ROIinOpenCV.py – real-time video capture, hand segmentation, live inference
- cnn.py, HandHisto.py, ISL.py, opencv.py – supporting scripts for model training, histogram features, image preprocessing
- ASL*.ipynb & sign_language_pytorch.ipynb – exploratory notebooks (Keras/TensorFlow vs PyTorch workflows)
- Model artifacts: CNNmodel.h5, model.h5, model_weights.h5
- Config & metadata: requirements.txt, _config.yml, LICENSE, README.md
- Paper PDF: islpaper.pdf

##### README for additional insights
- Dependencies: TensorFlow, Keras, OpenCV (and PyTorch for the alternate notebook)
- Dataset: Sign-Language-MNIST (downloaded externally)
- Primary “run” entry point: ROIinOpenCV.py

##### Main components & relationships
a. Data Ingestion
- External source: Kaggle dataset (Sign-Language-MNIST)
- Storage: local filesystem (raw .csv or images)
b. Preprocessing Module
- Scripts: HandHisto.py / opencv.py
- Tasks: ROI extraction, resizing, normalization, histogram equalization
c. Model Training
- Frameworks: Keras/TensorFlow (cnn.py) or PyTorch (sign_language_pytorch.ipynb)
- Output: serialized model files (.h5, model_weights.h5)
d. Model Artifacts
- Stored on disk: CNNmodel.h5 / model.h5 + weights
e. Inference Module
- Script: ROIinOpenCV.py (uses live webcam feed, applies preprocessing, loads model, outputs predicted sign)
f. Notebooks (ASL*.ipynb)
- Experimental workflows: prototyping architectures, hyperparameter tuning
g. External Interfaces
- Webcam or video stream (input)
- Console/UI (text overlay or print prediction)

Data flow: Dataset → Preprocessing → Training → Model Artifacts → Inference → User/UI

##### Architectural patterns & principles
- Pipeline architecture: distinct stages for data prep, training, inference
- Separation of concerns:
- Preprocessing code decoupled from model definitions
- Model training separate from inference logic
- Extensibility: ability to swap out Keras/TensorFlow vs PyTorch backends
- Modularity: standalone scripts/notebooks for each major task

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Components"
        Dataset["External Dataset\n(Sign-Language-MNIST)"]:::data
    end

    subgraph "Preprocessing Service"
        HandHisto["HandHisto.py\n(OpenCV)"]:::process
        OpenCV["opencv.py\n(OpenCV)"]:::process
    end

    subgraph "Model Training Service"
        subgraph "TensorFlow/Keras"
            CNN["cnn.py"]:::process
            ASL1["ASL.ipynb"]:::process
            ASL2["ASL2.ipynb"]:::process
            ASL3["ASLwithCNN.ipynb"]:::process
        end
        subgraph "PyTorch"
            PytorchNB["sign_language_pytorch.ipynb"]:::process
        end
    end

    subgraph "Model Repository"
        CNNmodel["CNNmodel.h5"]:::storage
        ModelH5["model.h5"]:::storage
        Weights["model_weights.h5"]:::storage
    end

    subgraph "Inference Service"
        Inference["ROIinOpenCV.py\n(OpenCV + Model)"]:::process
    end

    subgraph "External Interfaces"
        Webcam["Webcam\n(Video Stream)"]:::external
        UI["User Interface\n(Console/Overlay)"]:::external
    end

    subgraph "Documentation & Config"
        Req["requirements.txt"]:::docs
        Config["_config.yml"]:::docs
        Readme["README.md"]:::docs
        Paper["islpaper.pdf"]:::docs
        ISL["ISL.py"]:::docs
    end

    %% Data Flow
    Dataset -->|"raw data"| HandHisto
    Dataset -->|"raw data"| OpenCV
    HandHisto -->|"ROI images"| CNN
    OpenCV -->|"preprocessed images"| CNN
    CNN -->|".h5 model"| CNNmodel
    CNN -->|".h5 model"| ModelH5
    CNN -->|"weights"| Weights
    ASL1 -->|".h5 model"| CNNmodel
    ASL2 -->|".h5 model"| ModelH5
    ASL3 -->|"weights"| Weights
    PytorchNB -->|".h5 model"| CNNmodel
    CNNmodel -->|"load model"| Inference
    ModelH5 -->|"load model"| Inference
    Weights -->|"load weights"| Inference
    Webcam -->|"video stream"| Inference
    Inference -->|"predictions"| UI

    %% Click Events
    click HandHisto "https://github.com/arshad221b/sign-language-recognition/blob/master/HandHisto.py"
    click OpenCV "https://github.com/arshad221b/sign-language-recognition/blob/master/opencv.py"
    click CNN "https://github.com/arshad221b/sign-language-recognition/blob/master/cnn.py"
    click ASL1 "https://github.com/arshad221b/sign-language-recognition/blob/master/ASL.ipynb"
    click ASL2 "https://github.com/arshad221b/sign-language-recognition/blob/master/ASL2.ipynb"
    click ASL3 "https://github.com/arshad221b/sign-language-recognition/blob/master/ASLwithCNN.ipynb"
    click PytorchNB "https://github.com/arshad221b/sign-language-recognition/blob/master/sign_language_pytorch.ipynb"
    click CNNmodel "https://github.com/arshad221b/sign-language-recognition/blob/master/CNNmodel.h5"
    click ModelH5 "https://github.com/arshad221b/sign-language-recognition/blob/master/model.h5"
    click Weights "https://github.com/arshad221b/sign-language-recognition/blob/master/model_weights.h5"
    click Inference "https://github.com/arshad221b/sign-language-recognition/blob/master/ROIinOpenCV.py"
    click Req "https://github.com/arshad221b/sign-language-recognition/blob/master/requirements.txt"
    click Config "https://github.com/arshad221b/sign-language-recognition/blob/master/_config.yml"
    click Readme "https://github.com/arshad221b/sign-language-recognition/blob/master/README.md"
    click Paper "https://github.com/arshad221b/sign-language-recognition/blob/master/islpaper.pdf"
    click ISL "https://github.com/arshad221b/sign-language-recognition/blob/master/ISL.py"

    classDef data fill:#ADD8E6,stroke:#000;
    classDef process fill:#98FB98,stroke:#000;
    classDef storage fill:#FFD700,stroke:#000;
    classDef external fill:#FFA500,stroke:#000;
    classDef docs fill:#D3D3D3,stroke:#000;
```
