# paresh-0007/signlanguagedetectionusinglstm architecture

> Architecture diagram of the GitHub repository paresh-0007/signlanguagedetectionusinglstm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/paresh-0007/signlanguagedetectionusinglstm
- Repository: https://github.com/paresh-0007/signlanguagedetectionusinglstm

## Overview

##### Project Type and Purpose
- This is a Python‐based machine-learning application for real-time American Sign Language (ASL) recognition.
- It consists of two main pipelines:
- Training pipeline – collects hand-gesture data, preprocesses, trains an LSTM model, generates artifacts (model.json, model.h5) and logging data.
- Inference (serving) pipeline – captures live video, detects hand landmarks using MediaPipe, feeds preprocessed feature sequences into the trained LSTM, and displays recognized letters.

##### File Structure Analysis
- Top-level: LICENSE, README.md, requirement.txt, plus the code folder SignLanguageDetectionUsingLstm/.
- SignLanguageDetectionUsingLstm/:
- Data collection & organization: makefolders.py, collectdata.py
- Model training: trainmodel.py, model.json (architecture), model.h5 (weights)
- Inference/app: app.py
- Experiment artifacts: Logs/ (TensorBoard events), Confusion.ipynb, PPT_…pptx, Report_….pdf

##### Key Architecture Insights from README
- Real-time webcam capture (OpenCV) + Google MediaPipe hand-landmark extraction.
- Sequence-based classification using a TensorFlow-Keras LSTM.
- Output mapping from predicted label to alphabet letter.
- Logging via TensorBoard for monitoring training.

a. Main Components:
- User Webcam (video feed input)
- MediaPipe Hand Detector (landmark extractor)
- LSTM Inference Engine (loads model.json + model.h5)
- Visualization/UI (OpenCV window with predicted text overlay)
- File System (dataset folders, model artifacts, TensorBoard logs)
- Training Pipeline (collectdata.py → makefolders.py → trainmodel.py → produce artifacts & Logs)

b. Relationships & Data Flow:
– Training flow: Raw webcam/images → makefolders.py → collectdata.py → dataset folders → preprocessing → trainmodel.py → model.json + model.h5 + Logs → stored in file system.
– Inference flow: Webcam → MediaPipe → preprocessing → LSTM model → prediction → UI.

c. Architectural Patterns & Principles:
– Pipeline pattern: separation of concerns between data collection, preprocessing, model training, and inference.
– Clear modularization of utility functions vs. orchestration scripts.
– Use of external library services (MediaPipe, TensorFlow).

d. Technologies & Frameworks:
– Python 3.9, OpenCV for capture & display
– MediaPipe for hand-landmark detection
– TensorFlow/Keras LSTM for sequence classification
– TensorBoard for training logs & visualization

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Input
    WebcamLive["User Webcam<br/>(OpenCV)"]:::input
    WebcamLive -->|"raw video frames"| MakeFolders

    subgraph "Training Pipeline"
        direction TB
        MakeFolders["makefolders.py"]:::processing
        CollectData["collectdata.py"]:::processing
        Dataset["Dataset Folders"]:::storage
        PreprocessorTrain["Preprocessing Module"]:::processing
        TrainModel["trainmodel.py"]:::processing
        Artifacts["model.json<br/>model.h5"]:::storage
        Logs["TensorBoard Logs"]:::storage
        Experiment["Experiment Analysis<br/>(Notebook/PPT/PDF)"]:::output

        MakeFolders -->|"organize folders"| CollectData
        CollectData -->|"save images"| Dataset
        Dataset -->|"load data"| PreprocessorTrain
        PreprocessorTrain -->|"training sequences"| TrainModel
        TrainModel -->|"save model"| Artifacts
        TrainModel -->|"write logs"| Logs
        Logs -->|"analyze"| Experiment
    end

    subgraph "Inference Pipeline"
        direction TB
        WebcamLive2["User Webcam<br/>(OpenCV)"]:::input
        MediaPipe["MediaPipe Detector"]:::external
        PreprocessorInfer["Preprocessing Module"]:::processing
        LSTMModel["LSTM Inference Engine"]:::processing
        UI["UI / Overlay"]:::output

        WebcamLive2 -->|"video frame"| MediaPipe
        MediaPipe -->|"landmark coords"| PreprocessorInfer
        PreprocessorInfer -->|"feature sequence"| LSTMModel
        LSTMModel -->|"predicted label"| UI
    end

    %% External Dependencies
    subgraph "External Libraries"
        TensorFlow["TensorFlow / Keras"]:::external
        MediaPipeExt["MediaPipe"]:::external
        OpenCVExt["OpenCV"]:::external
        TensorBoardExt["TensorBoard"]:::external
    end
    TrainModel --> TensorFlowExt
    LSTMModel --> TensorFlowExt
    MediaPipe --> MediaPipeExt
    WebcamLive --> OpenCVExt
    WebcamLive2 --> OpenCVExt
    Logs --> TensorBoardExt

    %% Project Configuration
    subgraph "Project Files"
        direction TB
        Req["requirement.txt"]:::storage
        Readme["README.md"]:::storage
        License["LICENSE"]:::storage
    end

    %% Click events
    click MakeFolders "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/makefolders.py"
    click CollectData "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/collectdata.py"
    click PreprocessorTrain "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/shape.py"
    click PreprocessorTrain "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/data.py"
    click PreprocessorTrain "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/function.py"
    click TrainModel "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/trainmodel.py"
    click Artifacts "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/model.json"
    click Artifacts "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/model.h5"
    click Logs "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/Logs/train/events.out.tfevents.*"
    click Experiment "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/Confusion.ipynb"
    click Experiment "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/PPT_SE_A_G1_SignLanguageRecognition.pptx"
    click Experiment "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/Report_SE_A_G1_SignLanguageRecognition.pdf"
    click UI "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/SignLanguageDetectionUsingLstm/app.py"
    click Req "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/requirement.txt"
    click Readme "https://github.com/paresh-0007/signlanguagedetectionusinglstm/blob/main/README.md"
    click License "https://github.com/paresh-0007/signlanguagedetectionusinglstm/tree/main/LICENSE"

    %% Styles
    classDef input fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef processing fill:#FFA500,stroke:#333,stroke-width:1px
    classDef storage fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef output fill:#90EE90,stroke:#333,stroke-width:1px
    classDef external fill:#CCCCFF,stroke:#333,stroke-width:1px
```
