# shag527/indian-sign-language-recognition architecture

> Architecture diagram of the GitHub repository shag527/indian-sign-language-recognition, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shag527/indian-sign-language-recognition
- Repository: https://github.com/shag527/indian-sign-language-recognition

## Overview

##### Project Type and Purpose
- This project is a full-stack recognition system that processes Indian Sign Language gestures. It involves collecting hand gesture images, preprocessing them using both background subtraction and skin segmentation, extracting SURF features using a Bag-of-Words approach, classifying the gestures with several machine learning models (KNN, SVM, logistic regression, Naive Bayes, CNN), and providing outputs as both text and speech.
- Dual modes are supported: one for regular sign-based input and one for reverse sign recognition (processing spoken words and displaying corresponding sign images).

##### File Structure
- The top-level “Code” folder is divided into several functional modules:
- Bag of Features – handles histogram generation from SURF descriptors.
- Classification – contains various classifier implementations (CNN, KNN, SVM, logistic regression, naive bayes) along with their dataset CSV files.
- Predict signs – integrates the prediction functionality, reverse recognition logic, dataset creation, and a main tkinter UI.
- Preprocessing – contains notebooks that perform dataset division and image preprocessing.
- Additional folders such as “Images” (for sample datasets and output visuals) and “Scripts” (environment activation scripts) support deployment and running the project.

##### Extracting Details from the README
- The README outlines the data workflow:
- Getting started instructions including dependency management.
- Preprocessing using background subtraction and skin segmentation.
- Feature extraction using SURF and clustering via MiniKbatchMeans to create a bag of visual words histogram.
- Employing multiple classifiers followed by accuracy analysis.
- A dual-mode approach to support both text-to-speech conversion and reverse sign recognition using speech-to-text via Google Speech API.
- It also mentions the user interaction via a tkinter GUI, where users log in, choose services, or create their own sign dataset.

##### Core Components
a. Modules/Components:
- Data Input & Dataset: Representing the manually created dataset (images) accessible via the “Images” folder.
- Preprocessing Module: Split into subcomponents running in Jupyter notebooks (background subtraction and skin segmentation). This module processes raw images to filter out background/noise.
- Feature Extraction Module (Bag of Features): Uses SURF to extract and cluster features and then generate histograms as CSV files.
- Classification Module: Contains various ML classifiers (CNN, KNN, SVM, Logistic Regression, Naive Bayes) with CSV data input and output predictions.
- Prediction & Reverse Recognition Module: Incorporates the prediction logic, reverse recognition using speech input (Google Speech API) and displaying results through a tkinter-based UI.
- Output & Communication Module: Handles converting the predictions to text and speech output (via Pyttsx3) and updating the GUI.

b. Relationships & Interactions:
- A clear linear data flow exists starting from raw image acquisition → Preprocessing → Feature Extraction → Classification, with the outputs funneled into the UI for both sign prediction and reverse sign recognition.
- There is also a feedback loop for creating/updating the dataset and retraining the classifiers.
- The UI component (main.py in “Predict signs”) interacts with the classification modules and then triggers appropriate output functionality.

c. Architectural Patterns & Design Principles:
- Modular design: Each functional aspect (preprocessing, feature extraction, classification) is divided into separate directories to ensure loose coupling.
- Layered Architecture: Separation of preprocessing/feature extraction from classification and UI.
- Integration of machine learning pipelines with user interface components.

d. Key Technologies & Frameworks:
- Programming Language: Python (3.7.1)
- Libraries: OpenCV (and opencv-contrib for SURF), scikit-learn (for classifiers like KNN, SVM, etc.), TensorFlow/Keras or similar for CNN, Pyttsx3 for text to speech, and tkinter for GUI.
- Tools for environment/ dependency management: pip, requirements.txt, and conda.

##### Specific Elements
- Data flow from the raw input to preprocessing → feature extraction → classification → UI output.
- Interactions between the GUI and backend processing.
- For data processing components (use rectangles),
- Incorporate annotations where necessary, such as indicating that the “Bag of Features” module outputs CSV files used by classifier scripts.

## Diagram (Mermaid)

```mermaid
flowchart TD
    DataInput["Data Input & Dataset"]:::data
    Preprocessing["Preprocessing<br/>(Background Subtraction & Skin Segmentation)"]:::preprocess
    FeatureExtraction["Feature Extraction<br/>(SURF + BOVW)"]:::feature
    Classification["Classifier Ensemble<br/>(CNN, KNN, SVM, Logistic Regression, Naive Bayes)"]:::classification
    Prediction["Prediction & Reverse Recognition<br/>(TTS & UI)"]:::prediction

    DataInput -->|"rawImages"| Preprocessing
    Preprocessing -->|"processedImages"| FeatureExtraction
    FeatureExtraction -->|"featureHistograms"| Classification
    Classification -->|"predictedSigns"| Prediction
    Prediction -->|"datasetUpdate"| DataInput

    click DataInput "https://github.com/shag527/indian-sign-language-recognition/tree/master/Images"
    click Preprocessing "https://github.com/shag527/indian-sign-language-recognition/tree/master/Code/Preprocessing"
    click FeatureExtraction "https://github.com/shag527/indian-sign-language-recognition/tree/master/Code/Bag of Features"
    click Classification "https://github.com/shag527/indian-sign-language-recognition/tree/master/Code/Classification"
    click Prediction "https://github.com/shag527/indian-sign-language-recognition/tree/master/Code/Predict signs"

    classDef data fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef preprocess fill:#d4edda,stroke:#333,stroke-width:2px;
    classDef feature fill:#fff3cd,stroke:#333,stroke-width:2px;
    classDef classification fill:#f8d7da,stroke:#333,stroke-width:2px;
    classDef prediction fill:#d1ecf1,stroke:#333,stroke-width:2px;
```
