# devansh-47/sign-language-to-text-and-speech-conversion architecture

> Architecture diagram of the GitHub repository devansh-47/sign-language-to-text-and-speech-conversion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/devansh-47/sign-language-to-text-and-speech-conversion
- Repository: https://github.com/devansh-47/sign-language-to-text-and-speech-conversion

## Overview

##### Project Type & Purpose
- The project is a computer vision based full-stack application (though without a dedicated GUI, it primarily runs Python scripts) that processes real‐time images to detect American sign language hand gestures, classifies them using a convolutional neural network (CNN), and then converts the recognized gesture (or word) into both text and speech.
- Its purpose is to bridge communication for deaf and dumb people by providing a mechanism that translates sign language gestures into written text and audible speech.

##### Key Directories and Components from the File Structure
- The “AtoZ_3.1” directory contains subdirectories named A through Z. These likely represent datasets or image collections for each alphabet sign used to train or test the CNN.
- Python scripts such as “data_collection_binary.py”, “data_collection_final.py”, “final_pred.py”, and “prediction_wo_gui.py” indicate separate modules for data acquisition, preprocessing or collection, and prediction/inference logic.
- The “cnn8grps_rad1_model.h5” file is the trained Keras CNN model.

##### Main System Components and Their Relationships
a. Data Acquisition Module:
- Handles input from the webcam.
- Uses media pipe library for extracting hand landmarks.
- Prepares raw input images.
b. Pre-processing & Feature Extraction Module:
- Uses OpenCV to apply filters, convert images to grayscale, apply blurring and thresholding.
- Uses the output of mediapipe (landmark points) to generate simplified images free from background issues.
- Prepares region-of-interest (ROI) images for the CNN.
c. CNN-based Gesture Classification Module:
- The preprocessed gesture images are fed into a CNN (built with Keras/Tensorflow).
- The CNN first classifies gestures into grouped classes (8 groups) and then further refines the classification into the exact alphabet.
- Stores and loads the trained model from “cnn8grps_rad1_model.h5”.
d. Post-Processing and Conversion Module:
- Once a gesture is classified, the recognized alphabet/word is output as text.
- Uses the “pyttsx3” library (or a similar text-to-speech package) to convert text into speech.
e. User Interaction / Interface:
- Even though there might not be a complete GUI, the system interacts with the user via the command line or minimal user display where prediction outputs are shown.
f. Data Storage/Organization:
- The “AtoZ_3.1” folder serves as storage for training/testing images (organized by sign letter), while Python scripts manage the flow from data capture to prediction.

##### Architectural Patterns and Design Principles
- A modular pipeline architecture where each module is responsible for a specific task (data capture, preprocessing, classification, and post-processing).
- The use of directional data flow is important – data moves sequentially from acquisition to final speech output.
- Separation of concerns: The system’s image preprocessing, CNN model inference, and text-to-speech conversion are clearly isolated.
- High reusability of helper modules such as the mediapipe-based hand detection and OpenCV pre-processing functions.

##### Adding Specific Notations
– The lower layers include physical devices and raw datasets.
– The mid layers include image processing and machine learning.
– The top layers handle user interaction and output (text and speech).

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["Webcam Input + Mediapipe Extraction"]:::start
    B["Data Collection (Binary)"]:::acquisition
    C["Data Collection & Pre-Processing (Final)"]:::preprocessing
    subgraph "CNN-based Gesture Classification"
        D["Load CNN Model"]:::cnn
        E["Perform Inference"]:::cnn
    end
    F["Post-Processing & Conversion (Text & Speech)"]:::postprocessing
    G["Data Storage (AtoZ Dataset)"]:::storage

    A --> B
    B --> C
    C --> D
    D --> E
    E --> F
    G -->|"ProvidesTrainingData"| C

    click B "https://github.com/devansh-47/sign-language-to-text-and-speech-conversion/blob/master/data_collection_binary.py"
    click C "https://github.com/devansh-47/sign-language-to-text-and-speech-conversion/blob/master/data_collection_final.py"
    click D "https://github.com/devansh-47/sign-language-to-text-and-speech-conversion/blob/master/cnn8grps_rad1_model.h5"
    click E "https://github.com/devansh-47/sign-language-to-text-and-speech-conversion/blob/master/final_pred.py"
    click F "https://github.com/devansh-47/sign-language-to-text-and-speech-conversion/blob/master/prediction_wo_gui.py"
    click G "https://github.com/devansh-47/sign-language-to-text-and-speech-conversion/blob/master/AtoZ_3.1"

    classDef start fill:#e0e0e0,stroke:#333,stroke-width:2px;
    classDef acquisition fill:#a6cee3,stroke:#1f78b4,stroke-width:2px;
    classDef preprocessing fill:#b2df8a,stroke:#33a02c,stroke-width:2px;
    classDef cnn fill:#fb9a99,stroke:#e31a1c,stroke-width:2px;
    classDef postprocessing fill:#fdbf6f,stroke:#ff7f00,stroke-width:2px;
    classDef storage fill:#cab2d6,stroke:#6a3d9a,stroke-width:2px;
```
