# dgovor/sign-language-translator architecture

> Architecture diagram of the GitHub repository dgovor/sign-language-translator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dgovor/sign-language-translator
- Repository: https://github.com/dgovor/sign-language-translator

## Overview

- Data Collection Component – represented by data_collection.py and its integration with the MediaPipe Holistic pipeline for capturing hand landmark data.
- Model Training Component – driven by model.py where the neural network (built using a Sequential model with LSTM and Dense layers) is defined, trained, and validated using the collected datasets.
- Real Time Prediction Component – handled by main.py which streams video, processes frames using MediaPipe Holistic, feeds sequences of landmarks to the trained model for sign predictions, and then passes the predicted sentence to the grammar correction module using language_tool_python.
- Utility Functions – represented by my_functions.py that may contain support routines used across the data collection, training, or real time prediction processes.
- Data/Assets – include directories (img) and potentially other assets that support user interfaces for instructions or result displays.

##### Relationships and interactions
- That data from the Data Collection module is used to build training datasets, which then feed into the Model Training module.
- From Model Training, once the network is trained, its output (the trained model) is used by the Real Time Prediction module to translate input frames into text.
- Integration with MediaPipe Holistic in both Data Collection and Real Time Predictions—these external services are vital to the system’s operation.

##### Architectural details
- That the project uses a neural network architecture with specific layers: multiple LSTM layers, Dense layers, ReLU activation in hidden layers, and Softmax at the output.
- That data is segmented into sequences (e.g., series of 10 frames) before being sent to the model – this is essential for understanding the temporal dynamics processed by LSTM.
- Mention the use of the Adam optimizer used in model training, and how training-validation splitting (e.g., 90/10 split) is done.
- Design principle of modularity by indicating that the system’s functionalities (collection, training, prediction) are encapsulated within their own scripts.

##### Relevant technologies and external services
- Python 3.6+
- Java (as required by LanguageTool)
- MediaPipe Holistic for hand tracking
- language_tool_python for grammar correction
- Data visualization components (e.g., the img directory with example gifs that illustrate process steps)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Acquisition Layer
    subgraph "Data Acquisition Layer"
        DC["Data Collection\n(data_collection.py)"]:::data
        MP["MediaPipe Holistic"]:::external
        DC -->|"captures landmark-sequence"| MP
    end

    %% Model Training Layer
    subgraph "Model Training Layer"
        MT["Model Training\n(model.py)"]:::training
        NN["Neural Network\n(LSTM/Dense, Adam, 90/10 split)"]:::training
        MT -->|"builds/trains"| NN
    end

    %% Real Time Prediction Layer
    subgraph "Real Time Prediction Layer"
        RTP["Real Time Prediction\n(main.py)"]:::prediction
        RT_P["Prediction Engine\n(Frame Capture, Sign Prediction)"]:::prediction
        GC["Grammar Correction\n(language_tool_python)"]:::external
        RTP -->|"streams video"| RT_P
        RT_P -->|"outputs predicted-text"| GC
        RTP --- MP
    end

    %% Utility and Assets
    UT["Utility Functions\n(my_functions.py)"]:::utility
    DA["Data/Assets\n(img)"]:::assets

    %% Inter-layer connections
    DC -->|"collected data"| MT
    NN -->|"trained model"| RTP

    %% External Dependencies connections
    DC --- MP
    RTP --- GC

    %% Click events
    click DC "https://github.com/dgovor/sign-language-translator/blob/main/data_collection.py"
    click MT "https://github.com/dgovor/sign-language-translator/blob/main/model.py"
    click RTP "https://github.com/dgovor/sign-language-translator/blob/main/main.py"
    click UT "https://github.com/dgovor/sign-language-translator/blob/main/my_functions.py"
    click DA "https://github.com/dgovor/sign-language-translator/tree/main/img"

    %% Styles
    classDef data fill:#a2d9ce,stroke:#117864,stroke-width:2px;
    classDef training fill:#fad7a0,stroke:#d35400,stroke-width:2px;
    classDef prediction fill:#aed6f1,stroke:#2471a3,stroke-width:2px;
    classDef utility fill:#d7bde2,stroke:#7d3c98,stroke-width:2px;
    classDef external fill:#f9e79f,stroke:#b7950b,stroke-width:2px;
```
