# emnikhil/sign-language-to-text-conversion architecture

> Architecture diagram of the GitHub repository emnikhil/sign-language-to-text-conversion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/emnikhil/sign-language-to-text-conversion
- Repository: https://github.com/emnikhil/sign-language-to-text-conversion

## Overview

##### Main Components
- Data Acquisition & Preparation:
– Folder Creation (FoldersCreation.py) and dataset organization (dataSet/trainingData and dataSet/testingData directories, each with subdirectories for “0” and A–Z).
– TrainingDataCollection.py and TestingDataCollection.py for capturing and saving images.
- Image Preprocessing Module:
– Code in the README shows image filtering, grayscale conversion, gaussian blur, thresholding, and ROI extraction.
- Model Training & Storage:
– The Models folder contains multiple model files (.h5 and .json) along with Jupyter notebooks (Model.ipynb) used for training the CNN.
– This module includes the CNN architecture, detailed in the README (Convolutional Layers, Pooling, Fully Connected Layers, and Output Layer with SoftMax).
- Inference & Classification Module:
– Application.py comprises the live application where frames from a webcam are processed, predictions are made using the trained CNN, and ambiguous cases are handled via additional classifiers (specific sets for D, R, U; T, K, D, I; and S, M, N).
- GUI Layer:
– The system includes a graphical user interface (likely created using tkinter and PIL) that displays the real-time gesture-to-text conversion, as detailed under “Creating a GUI” in the README.
- External Dependencies & Libraries:
– Listed libraries include TensorFlow, Keras, OpenCV, Pillow, among others.

##### Relationships & Data Flow
- The data starts with collection/capturing (from webcam) and is stored in the organized dataset directories.
- The image preprocessing module reads raw images, applies filters, and outputs processed images.
- These processed images are then fed into the CNN for training/updating weights (offline training phase) or used directly for inference (online, real-time recognition).
- The classification process may branch: if ambiguity is detected, additional classifier modules (for problematic symbol groups) are invoked.
- The results (predicted letters) are then aggregated to form text/sentences, which are displayed in the GUI.
- The file structure (Folders for models, datasets, images, PDFs, and presentations) supports documentation, training, testing, and presentation aspects.

##### Architectural Patterns & Design Principles
- Modular Design: Each functionality is separated into distinct modules (data collection, preprocessing, training, inference, and GUI) which helps with testing and maintenance.
- Pipeline/Data Flow Architecture: There is a clear linear flow—from image capture and processing, through classification, to final text output.
- Multi-Layered Classification: The design incorporates layered classification to improve accuracy on ambiguous classes, as noted in the README.
- Use of Pre-trained Models & Serialization: The Models directory indicates that models are likely pre-trained and stored in serialized formats (.h5 and .json), which are loaded during inference.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Acquisition & Preparation Subgraph
    subgraph "Data Acquisition & Preparation"
        FC["FoldersCreation.py"]:::data
        TDC["TrainingDataCollection.py"]:::data
        TSC["TestingDataCollection.py"]:::data
        DS["dataSet"]:::data
        AD["Acquired Data"]:::data
        FC --> AD
        TDC --> AD
        TSC --> AD
        DS --> AD
    end

    %% Main Processing Flow
    AD --> IP["Image Preprocessing & Feature Extraction<br/>(Filtering, Grayscale, Gaussian Blur, Thresholding, ROI)"]:::processing
    IP --> CNN["CNN Model Training & Storage"]:::model
    IP --> INF["Inference & Classification"]:::inference
    CNN --> INF
    INF --> GUI["GUI & Real-Time Text Display"]:::gui

    %% Ambiguity Resolution Loop within Inference
    INF --> AMB["Ambiguity Resolution Classifiers<br/>(D,R,U; T,K,D,I; S,M,N)"]:::inference
    AMB -- "resolved" --> INF

    %% External Libraries
    subgraph "External Libraries"
        OP["OpenCV"]:::external
        TF["TensorFlow/Keras"]:::external
        TK["tkinter/PIL"]:::external
    end

    %% Connections from Libraries to Modules
    OP --- IP
    TF --- CNN
    TK --- GUI

    %% Click Events for Component Mapping
    click FC "https://github.com/emnikhil/sign-language-to-text-conversion/blob/main/FoldersCreation.py"
    click TDC "https://github.com/emnikhil/sign-language-to-text-conversion/blob/main/TrainingDataCollection.py"
    click TSC "https://github.com/emnikhil/sign-language-to-text-conversion/blob/main/TestingDataCollection.py"
    click DS "https://github.com/emnikhil/sign-language-to-text-conversion/tree/main/dataSet/"
    click CNN "https://github.com/emnikhil/sign-language-to-text-conversion/tree/main/Models/"
    click IP "https://github.com/emnikhil/sign-language-to-text-conversion/blob/main/Application.py"
    click INF "https://github.com/emnikhil/sign-language-to-text-conversion/blob/main/Application.py"
    click GUI "https://github.com/emnikhil/sign-language-to-text-conversion/blob/main/Application.py"
    click AMB "https://github.com/emnikhil/sign-language-to-text-conversion/blob/main/Application.py"

    %% Styles
    classDef data fill:#b3d9ff,stroke:#0066cc,stroke-width:2px;
    classDef processing fill:#ccffcc,stroke:#33cc33,stroke-width:2px;
    classDef model fill:#e6ccff,stroke:#9900cc,stroke-width:2px;
    classDef inference fill:#ffcc99,stroke:#ff6600,stroke-width:2px;
    classDef gui fill:#ffedcc,stroke:#ff9900,stroke-width:2px;
    classDef external fill:#f9f9a9,stroke:#cccc00,stroke-width:2px;
```
