How muhammadmahi585/sign_language_glove_application works
Project type and purpose
• This is a full-stack IoT/mobile application that translates sign-language gestures captured by a smart glove into text via a REST API and displays results on a cross-platform Flutter app.
• Key flows:
– Hardware (Arduino firmware) reads sensors → Bluetooth →
– Flutter mobile app (Android/iOS) receives raw sensor data → HTTP →
– FastAPI backend loads a pre-trained RF model → Inference → HTTP response →
– Mobile app UI displays translated word/alphabet.
File structure
• Top-level directories:
– Sign_Glove_ArduinoCode: Arduino Nano firmware (.ino)
– model: Python training scripts, FastAPI “app.py” with rf_model.pkl
– android & ios: platform-specific scaffolding generated by Flutter
– lib: Flutter source (main.dart + screens/)
– assets/images: UI assets for alphabets & numbers
– test: Flutter widget_test.dart
• Config/build files: pubspec.yaml (Flutter), gradle scripts, .metadata, .gitignore, analysis_options.yaml
• This separation indicates a three-tier architecture (hardware, client, server) plus an ML training pipeline.
README for additional insights
• Confirms use of Arduino Nano + flex sensors + gyroscope + Bluetooth
• Backend: FastAPI (Python), served via uvicorn, optionally exposed via ngrok
• Mobile: Flutter (Dart) with screens for Bluetooth pairing, live communication, and home/education modules
Main components
a. Hardware Layer (Green)
• Flex sensors + gyroscope on glove → Arduino Nano → JDY-SP-311 Bluetooth module
• Arduino firmware (Sign_Glove_ArduinoCode.ino)
b. Mobile Layer (Blue)
• Flutter Frontend (lib/main.dart + screens/)
• Bluetooth Manager (connect/pair)
• HTTP Client (REST calls to FastAPI)
• UI Screens: HomeScreen, BluetoothScreen, CommunicationScreen
c. Backend Layer (Orange)
• FastAPI server (model/app.py)
• RF model loader (rf_model.pkl) and inference logic (data.py)
• Uvicorn ASGI server
• Optional ngrok tunnel for external access
d. ML Training Pipeline (Gray, separate)
• training.py → trains Random Forest on CSV sensor data under model/Words & model/alphabets
• Outputs rf_model.pkl
Architectural patterns and design principles
• Client-Server (Mobile = thin client, Backend = monolithic inference service)
• Separation of concerns: firmware vs. mobile UI vs. server logic vs. training code
• Pluggable ML model: rf_model.pkl can be swapped with improved models
• Cross-platform UI using Flutter for code reuse
Technologies and integration points
• Hardware: Arduino IDE, C/C++, Bluetooth
• Mobile: Flutter (Dart), Android & iOS manifests, Gradle, Xcode project
• Backend: Python 3, FastAPI, uvicorn, scikit-learn (or joblib)
• DevOps/Exposure: ngrok (tunnel), pip virtualenv (.venv)