# muhammadmahi585/sign_language_glove_application architecture

> Architecture diagram of the GitHub repository muhammadmahi585/sign_language_glove_application, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muhammadmahi585/sign_language_glove_application
- Repository: https://github.com/muhammadmahi585/sign_language_glove_application

## Overview

##### Project type and purpose
- This is a full-stack IoT/mobile application that translates sign-language gestures captured by a smart glove into text via a REST API and displays results on a cross-platform Flutter app.
- Key flows:
– Hardware (Arduino firmware) reads sensors → Bluetooth →
– Flutter mobile app (Android/iOS) receives raw sensor data → HTTP →
– FastAPI backend loads a pre-trained RF model → Inference → HTTP response →
– Mobile app UI displays translated word/alphabet.

##### File structure
- Top-level directories:
– Sign_Glove_ArduinoCode: Arduino Nano firmware (.ino)
– model: Python training scripts, FastAPI “app.py” with rf_model.pkl
– android & ios: platform-specific scaffolding generated by Flutter
– lib: Flutter source (main.dart + screens/)
– assets/images: UI assets for alphabets & numbers
– test: Flutter widget_test.dart
- Config/build files: pubspec.yaml (Flutter), gradle scripts, .metadata, .gitignore, analysis_options.yaml
- This separation indicates a three-tier architecture (hardware, client, server) plus an ML training pipeline.

##### README for additional insights
- Confirms use of Arduino Nano + flex sensors + gyroscope + Bluetooth
- Backend: FastAPI (Python), served via uvicorn, optionally exposed via ngrok
- Mobile: Flutter (Dart) with screens for Bluetooth pairing, live communication, and home/education modules

##### Main components
a. Hardware Layer (Green)
- Flex sensors + gyroscope on glove → Arduino Nano → JDY-SP-311 Bluetooth module
- Arduino firmware (Sign_Glove_ArduinoCode.ino)
b. Mobile Layer (Blue)
- Flutter Frontend (lib/main.dart + screens/)
- Bluetooth Manager (connect/pair)
- HTTP Client (REST calls to FastAPI)
- UI Screens: HomeScreen, BluetoothScreen, CommunicationScreen
c. Backend Layer (Orange)
- FastAPI server (model/app.py)
- RF model loader (rf_model.pkl) and inference logic (data.py)
- Uvicorn ASGI server
- Optional ngrok tunnel for external access
d. ML Training Pipeline (Gray, separate)
- training.py → trains Random Forest on CSV sensor data under model/Words & model/alphabets
- Outputs rf_model.pkl

##### Architectural patterns and design principles
- Client-Server (Mobile = thin client, Backend = monolithic inference service)
- Separation of concerns: firmware vs. mobile UI vs. server logic vs. training code
- Pluggable ML model: rf_model.pkl can be swapped with improved models
- Cross-platform UI using Flutter for code reuse

##### Technologies and integration points
- Hardware: Arduino IDE, C/C++, Bluetooth
- Mobile: Flutter (Dart), Android & iOS manifests, Gradle, Xcode project
- Backend: Python 3, FastAPI, uvicorn, scikit-learn (or joblib)
- DevOps/Exposure: ngrok (tunnel), pip virtualenv (.venv)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Hardware Layer
    subgraph "Hardware Layer"
        direction TB
        ArduinoFirmware["Arduino Firmware (.ino)"]:::hardware
        ArduinoNano["Arduino Nano + Flex Sensors + Gyroscope"]:::hardware
        BluetoothModule["JDY-SP-311 Bluetooth Module"]:::hardware
    end

    %% Mobile Layer
    subgraph "Mobile Layer"
        direction TB
        MainDart["lib/main.dart"]:::mobile
        HomeScreen["HomeScreen"]:::mobile
        BluetoothScreen["BluetoothScreen"]:::mobile
        CommunicationScreen["CommunicationScreen"]:::mobile
        AndroidHost["MainActivity.kt"]:::mobile
        iOSHost["AppDelegate.swift"]:::mobile
    end

    %% Backend Layer
    subgraph "Backend Layer"
        direction TB
        Uvicorn["model/main.py (Uvicorn Launcher)"]:::backend
        FastAPIApp["model/app.py (FastAPI /predict)"]:::backend
        DataLoader["model/data.py (Inference & Model Loader)"]:::backend
        ModelArtifact["rf_model.pkl"]:::artifact
        ngrok["ngrok Tunnel"]:::external
    end

    %% ML Training Pipeline
    subgraph "ML Training Pipeline"
        direction TB
        DataWords["Words CSV Dataset"]:::training
        DataAlpha["Alphabets CSV Dataset"]:::training
        TrainingScript["model/training.py"]:::training
    end

    %% Connections Hardware->Mobile
    ArduinoFirmware --> ArduinoNano
    ArduinoNano --> BluetoothModule
    BluetoothModule -->|"Bluetooth RAW Data"| BluetoothScreen

    %% Mobile interactions
    BluetoothScreen -->|"Parsed Sensor Frames"| CommunicationScreen
    CommunicationScreen -->|"HTTP POST JSON /predict"| FastAPIApp
    FastAPIApp -->|"Prediction JSON"| CommunicationScreen
    CommunicationScreen --> HomeScreen
    MainDart --> HomeScreen
    MainDart --> BluetoothScreen
    MainDart --> CommunicationScreen
    AndroidHost --> MainDart
    iOSHost --> MainDart

    %% Backend internal
    Uvicorn --> FastAPIApp
    FastAPIApp --> DataLoader
    DataLoader --> ModelArtifact
    ngrok --> Uvicorn
    CommunicationScreen -.-> ngrok

    %% Training pipeline
    DataWords --> TrainingScript
    DataAlpha --> TrainingScript
    TrainingScript --> ModelArtifact

    %% Click events
    click ArduinoFirmware "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/Sign_Glove_ArduinoCode/Sign_Glove_ArduinoCode.ino"
    click MainDart "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/lib/main.dart"
    click HomeScreen "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/lib/screens/HomeScreen.dart"
    click BluetoothScreen "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/lib/screens/BluetoothScreen.dart"
    click CommunicationScreen "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/lib/screens/CommunicationScreen.dart"
    click AndroidHost "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/android/app/src/main/kotlin/com/example/sign_glove_application/MainActivity.kt"
    click iOSHost "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/ios/Runner/AppDelegate.swift"
    click FastAPIApp "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/model/app.py"
    click Uvicorn "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/model/main.py"
    click DataLoader "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/model/data.py"
    click ModelArtifact "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/model/rf_model.pkl"
    click TrainingScript "https://github.com/muhammadmahi585/sign_language_glove_application/blob/master/model/training.py"
    click DataWords "https://github.com/muhammadmahi585/sign_language_glove_application/tree/master/model/Words"
    click DataAlpha "https://github.com/muhammadmahi585/sign_language_glove_application/tree/master/model/alphabets"

    %% Styles
    classDef hardware fill:#e0ffe0,stroke:#2d662d,color:#2d662d
    classDef mobile fill:#e0f0ff,stroke:#1a4d8f,color:#1a4d8f
    classDef backend fill:#fff0e0,stroke:#b35c00,color:#b35c00
    classDef artifact fill:#fff5cc,stroke:#bf9b30,color:#bf9b30
    classDef training fill:#f0f0f0,stroke:#666666,color:#333333
    classDef external fill:#fdf0ff,stroke:#a347a3,color:#a347a3
```
