How rfypych/hand-tracker-mouse-controller works
Main Components
• Webcam & Video Feed – The source of input using OpenCV to capture frames.
• Hand Tracker Module – HandTracker (in hand_tracker.py) that uses Mediapipe to detect hand landmarks in each frame.
• Gesture Detector Module – GestureDetector (in gesture_detector.py) that interprets the hand landmarks to determine specific gestures such as click, drag, scroll, etc.
• Mouse Controller Module – MouseController (in mouse_controller.py) employing PyAutoGUI to actually control the mouse movements and actions.
• Calibration GUI Module – Calibration GUI (in calibration_gui.py) which provides a Tkinter interface for real-time adjustments (like smoothing and threshold settings).
• Sound Feedback Module – SoundFeedback (in sound_feedback.py) that uses Pygame to provide audio cues for certain actions.
• Assets – External resources (sound files in the assets directory) that are used by the SoundFeedback module.
Relationships & Interactions
• The entry point (main.py) orchestrates the overall flow: it initializes the video capture, processes each frame through the Hand Tracker, passes landmark data to the Gesture Detector, and then feeds resulting commands to the Mouse Controller.
• The Calibration GUI runs in parallel, allowing live adjustments to sensitivity and gesture thresholds that are fed back into the gesture detection logic.
• Depending on the recognized gesture, the system may trigger corresponding sound feedback using the Sound Feedback module.
• The Gesture Detector and Mouse Controller modules interact directly: when a gesture is detected (e.g., a click or scroll), the Mouse Controller receives these commands and translates them into computer input using PyAutoGUI.
• The assets (sound files) are referenced by the SoundFeedback module.
Architectural Patterns and Design Principles
• Modular Design: Each functionality (tracking, detection, control, calibration, sound) is separated into its own module, which promotes clear responsibilities and easier maintenance.
• Real-time Processing: Continuous video processing and event handling emphasizes the need for efficient module interactions and quick data flow from webcam to user response.
• Loose Coupling: The orchestration via main.py encourages loosely coupled modules where the Calibration GUI and core computation modules (hand tracking/gesture detection) can operate independently, yet influence each other through shared configurable parameters.
Relevant Technologies and Frameworks
• OpenCV for accessing and processing video feed.
• Mediapipe for robust hand landmark detection.
• PyAutoGUI for interfacing with system-level mouse controls.
• Tkinter for building a lightweight GUI for calibration.
• Pygame for handling audio feedback.
• NumPy for necessary numerical calculations.
Final Touches
• Labels are clear, e.g., "Hand Tracker (Mediapipe)", "Gesture Detector (Custom Logic)", "Mouse Controller (PyAutoGUI)", etc.