How milansinghal2004/gesture-bridge works
Project Type and Purpose
• This is a Python‐based machine learning application for real-time sign-language recognition and “air drawing.”
• It consists of two main subsystems:
a. A data-preparation and model-training pipeline (HaGrid)
b. A real-time gesture-detection and drawing UI (AirCanvas)
• The root directory also holds shared scripts (e.g., launcher.py, evaluate_yolo.py) and the trained model (best.pt).
File Structure
• Top-level folders:
• AirCanvas/: real-time inference and UI
• HaGrid/: notebooks and scripts for data prep, training, evaluation
• Root scripts:
• launcher.py – entry point to choose tasks (train vs. infer)
• evaluate_yolo.py – evaluation harness for the YOLO model
• button.py, try.py – auxiliary/testing scripts
• best.pt – final trained YOLO weights
• requirements.txt – Python dependencies
• Key patterns:
• ML training pipeline separated from inference/UI
• Modular Python packages under src/ for clean separation (e.g., hand_tracker, gesture, drawing)
README
• Confirms the project name: “Gesture-Bridge—Sign-Language-Recognition”
Main Components & Interactions
A. Data-Preparation & Model-Training (HaGrid)
1. Raw data ingestion (images/videos)
2. Data_Preparation.ipynb & Implementation scripts
3. Label encoding (label_encoder.pkl)
4. Model training (Implementation_using_Mediapipepy.py)
5. Output: best_model.pkl / best.pt
6. Evaluation notebooks / eval.py
B. Inference & UI (AirCanvas)
1. Input: live camera feed (OpenCV)
2. hand_tracker module: detect & track hands (Mediapipe/YOLO)
3. gesture module: classify gestures via loaded model (best.pt)
4. drawing module: convert gesture classes into strokes on the canvas
5. ui module: display the drawing window and controls
C. Orchestration & Shared Components
• launcher.py: CLI or GUI menu to kick off either training (HaGrid) or inference (AirCanvas).
• evaluate_yolo.py: standalone evaluation tool using trained weights.
• requirements.txt: list of external libraries (OpenCV, PyTorch, Mediapipe, etc.).
Architectural Patterns and Design Principles
• Layered Separation:
• Data Layer: raw images, label encoders, trained weights
• ML Layer: training scripts & evaluation, inference code
• Application Layer: gesture→drawing logic
• Presentation Layer: UI for real-time display
• Modularity: each package under src/ is a self-contained component.
• Single Responsibility: e.g., hand_tracker only tracks landmarks; drawing only renders strokes.