# kinivi/hand-gesture-recognition-mediapipe architecture

> Architecture diagram of the GitHub repository kinivi/hand-gesture-recognition-mediapipe, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kinivi/hand-gesture-recognition-mediapipe
- Repository: https://github.com/kinivi/hand-gesture-recognition-mediapipe

## Overview

##### Main Components
- Application Entry Point (app.py): This is the demo/inference script that captures webcam video, applies MediaPipe to extract hand keypoints, and then routes the data to the ML inference modules.
- MediaPipe & Video Capture: Represents the frontend process where data is captured from the webcam and processed in real time using MediaPipe for hand landmark detection.
- Keypoint Classification Module:
- Inference Module: model/keypoint_classifier/keypoint_classifier.py
- Trained Model Files: keypoint_classifier.tflite, keypoint_classifier.hdf5
- Training Data and Labels: keypoint.csv and keypoint_classifier_label.csv (used in keypoint_classification.ipynb)
- Point History Classification Module:
- Inference Module: model/point_history_classifier/point_history_classifier.py
- Trained Model Files: point_history_classifier.tflite, point_history_classifier.hdf5
- Training Data and Labels: point_history.csv and point_history_classifier_label.csv (used in point_history_classification.ipynb)
- Utility Module:
- FPS Measurement: utils/cvfpscalc.py

##### Important Architectural Patterns & Design Principles
- Modularity: The separation of inference code, training notebooks, and utility functions demonstrates a modular architecture.
- Extensibility: The use of CSV files and separate directories for training data implies that updating or extending learning classes is straightforward.
- Inference vs. Training Separation: Clearly distinguish between the training pipeline (handled via Jupyter notebooks) and the inference pipeline (run via app.py).

##### Technologies & Frameworks
- MediaPipe for hand keypoint detection.
- OpenCV for accessing webcam and handling image operations.
- Tensorflow/TFLite for running the machine learning models.
- Python as the main programming language.
- Jupyter Notebooks used for model training and experimentation.

## Diagram (Mermaid)

```mermaid
graph TD
  %% Real-Time Inference Subgraph
  subgraph "Real-Time Inference"
    A["Application Entry Point"]:::realtime
    M["MediaPipe Hand Landmark Extraction"]:::realtime
    K["Keypoint Classification Inference"]:::realtime
    P["Point History Classification Inference"]:::realtime
    F["FPS Utility"]:::utility
  end

  %% Model Training Subgraph
  subgraph "Model Training"
    %% Keypoint Classification Training Subgraph
    subgraph "Keypoint Classification Training"
      KTN["Keypoint Training Notebook"]:::training
      KTN2["Keypoint Training Notebook (Alternate)"]:::training
      KTData["Keypoint Model Files & Data"]:::training
      %% Keypoint Model Files
      KTF["Keypoint TFLite Model"]:::training
      KHF["Keypoint HDF5 Model"]:::training
      KCSV["Keypoint CSV Data"]:::training
      KLabel["Keypoint Label CSV"]:::training
    end

    %% Point History Classification Training Subgraph
    subgraph "Point History Classification Training"
      Ptn["Point History Training Notebook"]:::training
      PTData["Point History Model Files & Data"]:::training
      %% Point History Model Files
      PTF["Point History TFLite Model"]:::training
      PHF["Point History HDF5 Model"]:::training
      PCSV["Point History CSV Data"]:::training
      PLabel["Point History Label CSV"]:::training
    end
  end

  %% Data Flow in Real-Time Inference
  A -->|"captures"| M
  M -->|"extracts"| K
  M -->|"extracts"| P
  M -->|"monitors"| F

  %% Connections from Inference to Training (dotted lines)
  K -.->|"uses"| KTData
  K -.->|"supportedBy"| KTN
  K -.->|"supportedBy"| KTN2
  P -.->|"uses"| PTData
  P -.->|"supportedBy"| Ptn

  %% Model Files Associations within Training Subgraphs
  KTData -->|"contains"| KTF
  KTData -->|"contains"| KHF
  KTData -->|"contains"| KCSV
  KTData -->|"contains"| KLabel

  PTData -->|"contains"| PTF
  PTData -->|"contains"| PHF
  PTData -->|"contains"| PCSV
  PTData -->|"contains"| PLabel

  %% Styles
  classDef realtime fill:#d3f8d3,stroke:#027502,stroke-width:2px;
  classDef training fill:#e1e8f8,stroke:#164e63,stroke-width:2px;
  classDef utility fill:#f8e1d3,stroke:#a56319,stroke-width:2px;

  %% Click Events
  click A "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/app.py"
  click K "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/keypoint_classifier/keypoint_classifier.py"
  click P "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/point_history_classifier/point_history_classifier.py"
  click F "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/utils/cvfpscalc.py"
  click KTN "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/keypoint_classification.ipynb"
  click KTN2 "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/keypoint_classification_EN.ipynb"
  click KTF "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/keypoint_classifier/keypoint_classifier.tflite"
  click KHF "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/keypoint_classifier/keypoint_classifier.hdf5"
  click KCSV "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/keypoint_classifier/keypoint.csv"
  click KLabel "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/keypoint_classifier/keypoint_classifier_label.csv"
  click Ptn "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/point_history_classification.ipynb"
  click PTF "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/point_history_classifier/point_history_classifier.tflite"
  click PHF "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/point_history_classifier/point_history_classifier.hdf5"
  click PCSV "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/point_history_classifier/point_history.csv"
  click PLabel "https://github.com/kinivi/hand-gesture-recognition-mediapipe/blob/main/model/point_history_classifier/point_history_classifier_label.csv"
```
