# amulyakallakuri/real-time-alphabet-recognition-asl architecture

> Architecture diagram of the GitHub repository amulyakallakuri/real-time-alphabet-recognition-asl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amulyakallakuri/real-time-alphabet-recognition-asl
- Repository: https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl

## Overview

##### Project type and purpose
- This is a real-time computer-vision full-stack application for recognizing ASL alphabet letters from live video, evaluating multiple CNN backbones, and optionally autocorrecting and synthesizing speech output.
- It includes both a training pipeline (data preparation, model comparison, checkpointing) and an inference GUI pipeline (hand detection, classification, autocorrect, display/speech).

##### File structure
Top-level:
- data/ – raw and augmented image datasets for training and testing
- proposal.tex – project proposal document
- src/ – Python modules implementing data utilities, model definitions, training, inference, GUI, and spell-checker
- README.md – high-level project description

src/ modules of note:
- data_utils.py, split_train_val.py – data loading, augmentation, train/validation split
- cnn.py, ResNet50.py, VGG16.py, EfficientNetV2L.py – model definitions/wrappers
- trainer.py – orchestrates training loop, metrics logging, checkpoint saving
- test_inference.py – command-line inference on sample images
- handDetector.py – locates and crops hand region from camera frames
- GUI.py – real-time application entry point, connects video stream to inference and display
- Auto_Correct_SpellChecker.py, word_corrector.py – post-processing of recognized letters into words and autocorrect

##### Key insights from README
- Core task: static ASL alphabet recognition (29 classes + “non-ASL” class)
- Evaluation criteria: Top-1/Top-5 accuracy + inference FPS
- Multiple CNN backbones investigated; final model balances speed and accuracy
- Real-time feedback and optional text-to-speech output

##### Diagramming recommendations
- Interactions (e.g., “trainer.py writes models to /checkpoints,” “GUI.py calls handDetector.py”)
- Performance metrics near components (e.g., “Inference: 30 FPS on 3080Ti”)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Training Pipeline
    subgraph "Development / Training"
        direction TB
        Data["Raw & Augmented Data"]:::data
        DataUtils["data_utils.py"]:::processing
        Split["split_train_val.py"]:::processing
        Trainer["trainer.py"]:::processing
        CNN["cnn.py"]:::model
        ResNet["ResNet50.py"]:::model
        VGG["VGG16.py"]:::model
        EffNet["EfficientNetV2L.py"]:::model
        Checkpoints["Model Checkpoints"]:::repo
        README["README.md"]:::doc
        Proposal["proposal.tex"]:::doc

        Data --> DataUtils --> Split --> Trainer --> Checkpoints
        Trainer -->|"defines interface"| CNN
        CNN --> ResNet
        CNN --> VGG
        CNN --> EffNet
        DataUtils -->|"documented in"| README
        Proposal
    end

    %% Inference Application
    subgraph "Deployment / Inference"
        direction TB
        Camera["Live Camera Frames"]:::external
        HandDet["handDetector.py"]:::processing
        CLIInf["test_inference.py"]:::interface
        GUI["GUI.py"]:::interface
        WordCorr["word_corrector.py"]:::processing
        AutoCorr["Auto_Correct_SpellChecker.py"]:::processing
        TTS["Text-to-Speech Service"]:::external

        Camera --> HandDet
        HandDet --> GUI
        HandDet --> CLIInf --> WordCorr --> GUI
        GUI --> AutoCorr --> GUI
        GUI -->|"sends text"| TTS
    end

    %% Click Events
    click Data "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/tree/main/data/"
    click DataUtils "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/data_utils.py"
    click Split "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/split_train_val.py"
    click CNN "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/cnn.py"
    click ResNet "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/ResNet50.py"
    click VGG "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/VGG16.py"
    click EffNet "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/EfficientNetV2L.py"
    click Trainer "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/trainer.py"
    click HandDet "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/handDetector.py"
    click CLIInf "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/test_inference.py"
    click GUI "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/GUI.py"
    click WordCorr "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/word_corrector.py"
    click AutoCorr "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/src/Auto_Correct_SpellChecker.py"
    click Proposal "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/proposal.tex"
    click README "https://github.com/amulyakallakuri/real-time-alphabet-recognition-asl/blob/main/README.md"

    %% Styles
    classDef data fill:#e3f2fd,stroke:#90caf9,color:#0d47a1
    classDef processing fill:#ffecb3,stroke:#ffb300,color:#e65100
    classDef model fill:#c8e6c9,stroke:#66bb6a,color:#1b5e20
    classDef repo fill:#d1c4e9,stroke:#ab47bc,color:#4a148c
    classDef interface fill:#ffe0b2,stroke:#ff8f00,color:#bf360c
    classDef external fill:#ffcdd2,stroke:#e57373,color:#b71c1c
    classDef doc fill:#f0f4c3,stroke:#aed581,color:#33691e
```
