# pranav6266/realtime-handsign-to-audio-converter architecture

> Architecture diagram of the GitHub repository pranav6266/realtime-handsign-to-audio-converter, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pranav6266/realtime-handsign-to-audio-converter
- Repository: https://github.com/pranav6266/realtime-handsign-to-audio-converter

## Overview

##### Project type and purpose
- This is a desktop Python application for real-time American Sign Language (ASL) recognition and audio output. It includes two logical subsystems:
a) Offline training pipeline (data collection → dataset creation → model training)
b) Real-time inference pipeline (camera input → hand-landmark detection → classification → audio output)

##### File structure
Top-level files and folders:
- collect_images.py, create_dataset.py, train_classifer.py → training pipeline scripts
- hand-sign-recognizer_copy.py → main inference application
- labels_alphabet.txt, labels_words.txt → class label definitions
- model_alphabet.p, model_words.p → serialized classification models
- Words/ (presumably contains sample word images)
- Requirements: OpenCV, MediaPipe, NumPy, Tkinter, pyttsx3, pickle

##### README for additional insights
- The README confirms two models (alphabet & words), real-time webcam capture, 50%+ accuracy threshold over 10 frames, and audio output via pyttsx3.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Offline Training Pipeline"
        direction TB
        DataCollect["Data Collection\ncollect_images.py"]:::training
        DatasetCreate["Dataset Creation\ncreate_dataset.py"]:::training
        ModelTrain["Model Training\ntrain_classifer.py"]:::training
        DataCollect --> DatasetCreate
        DatasetCreate --> ModelTrain
        ModelTrain --> ModelFileAlphabet
        ModelTrain --> ModelFileWords
    end

    subgraph "Real-Time Inference Pipeline"
        direction TB
        Camera["Webcam (OpenCV)"]:::source
        Preprocessor["MediaPipe Hand-Landmark Detector"]:::processing
        Camera --> Preprocessor

        Preprocessor --> AlphabetClassifier
        Preprocessor --> WordsClassifier

        AlphabetClassifier["Alphabet Classifier\n(model_alphabet.p)"]:::processing
        WordsClassifier["Words Classifier\n(model_words.p)"]:::processing

        AlphabetClassifier --> Aggregator
        WordsClassifier --> Aggregator

        Aggregator["Prediction Aggregator\n(10 frames @>50% accuracy)"]:::processing
        Aggregator --> LabelLookup

        subgraph "Storage"
            direction TB
            LabelAlphabet["Alphabet Labels\n(labels_alphabet.txt)"]:::storage
            LabelWords["Word Labels\n(labels_words.txt)"]:::storage
            SampleWords["Sample Word Images\n(Words/)"]:::storage
        end

        LabelLookup["Label Lookup"]:::processing
        Aggregator --> LabelLookup
        LabelLookup --> TextToSpeech
        LabelLookup --> UI

        TextToSpeech["pyttsx3 Audio Engine"]:::source
        UI["Tkinter UI"]:::source
    end

    ModelFileAlphabet["model_alphabet.p"]:::storage
    ModelFileWords["model_words.p"]:::storage
    ModelFileAlphabet --> AlphabetClassifier
    ModelFileWords --> WordsClassifier

    LabelAlphabet --> LabelLookup
    LabelWords --> LabelLookup

    %% Click Events
    click DataCollect "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/collect_images.py"
    click DatasetCreate "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/create_dataset.py"
    click ModelTrain "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/train_classifer.py"
    click AlphabetClassifier "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/hand-sign-recognizer_copy.py"
    click WordsClassifier "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/hand-sign-recognizer_copy.py"
    click ModelFileAlphabet "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/model_alphabet.p"
    click ModelFileWords "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/model_words.p"
    click LabelAlphabet "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/labels_alphabet.txt"
    click LabelWords "https://github.com/pranav6266/realtime-handsign-to-audio-converter/blob/main/labels_words.txt"
    click SampleWords "https://github.com/pranav6266/realtime-handsign-to-audio-converter/tree/main/Words/"

    %% Styles
    classDef source fill:#DDEEFF,stroke:#333,stroke-width:1px
    classDef processing fill:#DDFFCC,stroke:#333,stroke-width:1px
    classDef storage fill:#CCE5FF,stroke:#333,stroke-width:1px
    classDef training fill:#FFE5CC,stroke:#333,stroke-width:1px
```
