How kinivi/hand-gesture-recognition-mediapipe works
Main Components
• Application Entry Point (app.py): This is the demo/inference script that captures webcam video, applies MediaPipe to extract hand keypoints, and then routes the data to the ML inference modules.
• MediaPipe & Video Capture: Represents the frontend process where data is captured from the webcam and processed in real time using MediaPipe for hand landmark detection.
• Keypoint Classification Module:
• Inference Module: model/keypoint_classifier/keypoint_classifier.py
• Trained Model Files: keypoint_classifier.tflite, keypoint_classifier.hdf5
• Training Data and Labels: keypoint.csv and keypoint_classifier_label.csv (used in keypoint_classification.ipynb)
• Point History Classification Module:
• Inference Module: model/point_history_classifier/point_history_classifier.py
• Trained Model Files: point_history_classifier.tflite, point_history_classifier.hdf5
• Training Data and Labels: point_history.csv and point_history_classifier_label.csv (used in point_history_classification.ipynb)
• Utility Module:
• FPS Measurement: utils/cvfpscalc.py
Important Architectural Patterns & Design Principles
• Modularity: The separation of inference code, training notebooks, and utility functions demonstrates a modular architecture.
• Extensibility: The use of CSV files and separate directories for training data implies that updating or extending learning classes is straightforward.
• Inference vs. Training Separation: Clearly distinguish between the training pipeline (handled via Jupyter notebooks) and the inference pipeline (run via app.py).
Technologies & Frameworks
• MediaPipe for hand keypoint detection.
• OpenCV for accessing webcam and handling image operations.
• Tensorflow/TFLite for running the machine learning models.
• Python as the main programming language.
• Jupyter Notebooks used for model training and experimentation.