# bacemkarray/asl-to-english-translator architecture

> Architecture diagram of the GitHub repository bacemkarray/asl-to-english-translator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bacemkarray/asl-to-english-translator
- Repository: https://github.com/bacemkarray/asl-to-english-translator

## Overview

##### Project type and purpose
- This is a full-stack AI application for real-time ASL-to-English translation.
- The “frontend/acernity_ui” folder houses a Next.js-based UI; “modelTrainingAndPrediction” contains Python scripts and TensorFlow/Keras models for both training and live prediction; “opencvdownloads” is an experimental sandbox for OpenCV-based frame capture and processing.

##### File structure
- Top-level:
- frontend/acernity_ui: Next.js app (React, Tailwind CSS)
- modelTrainingAndPrediction: Python ML pipeline (frame capture, predictionHandler, .h5 classifiers, Jupyter notebooks)
- opencvdownloads: proof-of-concept scripts for raw OpenCV frame handling
- Configuration files (.eslintrc, tsconfig, postcss, tailwind) indicate a standard React/Next.js build and styling pipeline.

##### README for additional insights
- Describes real-time ASL gesture capture → English text conversion
- Lists Python, OpenCV, TensorFlow/Keras, Streamlit (note: the file tree doesn’t contain a Streamlit UI—likely an earlier iteration)
- Workflow: video frames in → ML model predicts sign → text displayed in UI

a. Main components:
1) UI Layer (Next.js “acernity_ui”)
2) Prediction Service (Python microservice hosting frameAndPredictionHandling.py and predictionHandler.py, exposing a REST/WebSocket API)
3) ML Model Store (TensorFlow/Keras .h5 files)
4) Offline Training Pipeline (Jupyter notebooks & helper scripts produce new .h5 models)
5) Experimental Sandbox (OpenCV scripts under opencvdownloads) – optional, separate from production
6) External Inputs/Outputs (User’s Webcam, Browser, Optional Database or Logging service)
b. Relationships & interactions:
- The Next.js UI captures webcam frames via getUserMedia → batches frames → sends HTTP/WebSocket requests to the Prediction Service.
- The Prediction Service loads the .h5 model from ML Model Store → runs inference → returns JSON { sign: “A”, confidence: 0.98 }.
- UI renders translated text in real time.
- Offline Training Pipeline reads labeled datasets → trains new models → writes updated .h5 files back to ML Model Store.
- (Optional) Experimental Sandbox can feed new data or test alternate pipelines but is isolated from production inference.
c. Architectural patterns & principles:
- Microservice separation: UI vs. ML service
- Model versioning: store multiple .h5 files, swap by naming convention or endpoint parameter
- Stateless prediction API for scalability
- Offline/online split: heavy training jobs do not interfere with low-latency inference
d. Key technologies & frameworks:
- Frontend: Next.js, React, Tailwind CSS, TypeScript
- Backend: Python (FastAPI/Flask), OpenCV, TensorFlow/Keras
- Data exchange: REST or WebSocket, JSON payloads
- Packaging/build: npm, Python venv/requirements.txt

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Inputs
    Webcam((Webcam)):::external
    Browser((Browser)):::external

    %% Frontend/UI Layer
    subgraph "Frontend/UI" 
        UI["Next.js UI (acernity_ui)"]:::frontend
        Page["app/page.tsx"]:::frontend
        Layout["app/layout.tsx"]:::frontend
        UICom["app/uicom.tsx"]:::frontend
        Components["components/"]:::frontend
    end

    %% Prediction Service
    subgraph "Prediction Service (FastAPI/Flask)" 
        FrameHandler["frameAndPredictionHandling.py"]:::service
        PredictHandler["predictionHandler.py"]:::service
        TestPort["testPort.py"]:::service
    end

    %% ML Model Store
    subgraph "ML Model Store" 
        LineModel["lineASLClassifier.h5"]:::ml
        NewModel["newASLClassifier.h5"]:::ml
    end

    %% Offline Training Pipeline
    subgraph "Offline Training Pipeline" 
        Notebook1["signLanguageClassification.ipynb"]:::ml
        Notebook2["newASLClassifier.ipynb"]:::ml
        TrainScript["frameAndPredictionHandling.py"]:::ml
    end

    %% Experimental Sandbox
    subgraph "Experimental Sandbox" 
        SandboxDir["opencvdownloads/"]:::sandbox
    end

    %% Data Flows
    Webcam -->|"video frames"| Browser
    Browser -->|"capture via getUserMedia"| UI
    UI -->|"POST /predict {frames} JSON"| PredictHandler
    PredictHandler -->|"loads .h5 model"| LineModel
    PredictHandler -->|"returns {sign,confidence} JSON"| UI
    UI --> Browser
    Notebook1 -->|"trains new model"| NewModel
    Notebook2 -->|"trains new model"| NewModel
    TrainScript --> NewModel
    SandboxDir -.-> PredictHandler

    %% Click Events
    click UI "https://github.com/bacemkarray/asl-to-english-translator/tree/main/frontend/acernity_ui"
    click Page "https://github.com/bacemkarray/asl-to-english-translator/blob/main/frontend/acernity_ui/app/page.tsx"
    click Layout "https://github.com/bacemkarray/asl-to-english-translator/blob/main/frontend/acernity_ui/app/layout.tsx"
    click UICom "https://github.com/bacemkarray/asl-to-english-translator/blob/main/frontend/acernity_ui/app/uicom.tsx"
    click Components "https://github.com/bacemkarray/asl-to-english-translator/tree/main/frontend/acernity_ui/components"
    click FrameHandler "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/frameAndPredictionHandling.py"
    click PredictHandler "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/predictionHandler.py"
    click TestPort "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/testPort.py"
    click LineModel "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/lineASLClassifier.h5"
    click NewModel "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/newASLClassifier.h5"
    click Notebook1 "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/signLanguageClassification.ipynb"
    click Notebook2 "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/newASLClassifier.ipynb"
    click TrainScript "https://github.com/bacemkarray/asl-to-english-translator/blob/main/modelTrainingAndPrediction/frameAndPredictionHandling.py"
    click SandboxDir "https://github.com/bacemkarray/asl-to-english-translator/tree/main/opencvdownloads"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#0366d6,color:#0366d6;
    classDef service fill:#DFFFE0,stroke:#228B22,color:#228B22;
    classDef ml fill:#FFE5CC,stroke:#FF8C00,color:#FF8C00;
    classDef sandbox fill:#E0E0E0,stroke:#808080,color:#808080;
    classDef external fill:#F0F0F0,stroke:#999999,color:#999999;
```
