How prithanjan/virtual_mouse works
1. Repository purpose and runtime shape
- This is a single-machine, webcam-driven desktop application in Python.
- It combines two sequential computer-vision workflows: face-recognition attendance first, then a gesture-controlled virtual mouse.
2. Top-level orchestration
- main.py is the entry point and the coordinator for the whole user flow.
- It runs the attendance phase first, then switches into the gesture mouse phase after the user presses q.
- The application appears to be stateful within one process/session, not a set of independent services.
3. Phase 1: Face recognition attendance subsystem
- face_recog.py handles webcam-based face detection/recognition.
- It compares live camera frames against images in known_faces/.
- When a match is found, it records the person once per session and writes an attendance entry with name, date, and time.
4. Face data boundary
- known_faces/ is the input dataset for the attendance subsystem.
- The naming convention of the files is part of the architecture: filenames act as the identity source for recognition.
- This folder is effectively the system’s static enrollment store.
5. Attendance output boundary
- Attendance is persisted to attendance.csv.
- That file is an external artifact of the system, separate from the live runtime.
- The readme indicates the file is auto-created and append-only in practice for daily logs.
6. Phase switch / workflow control
- The attendance window acts as a gate before gesture control begins.
- q is the explicit user-controlled transition signal from attendance to mouse control.
- This makes the application a two-stage interactive pipeline rather than a continuously running background controller.
7. Phase 2: Gesture virtual mouse subsystem
- The gesture mouse logic is likely split between main.py and volumenbrightness.py, with the latter handling the audio side.
- Hand tracking is done with MediaPipe using the bundled hand_landmarker.task model file.
- The left hand is used for pointer movement and clicking; the right hand is used for volume control.
8. Input-to-action mapping
- Live webcam frames are analyzed for hand landmarks.
- Landmark positions are translated into desktop actions:
- index finger movement → cursor movement
- index + middle finger proximity → click
- thumb + index pinch → volume adjustment
- This is the key interaction boundary between vision processing and OS control.
9. OS integration boundary
- Mouse control is performed through PyAutoGUI.
- Volume control uses pycaw/comtypes, which ties the app to Windows audio APIs.
- This means the gesture layer is not just visual feedback; it directly manipulates host OS state.
10. Shared computer-vision infrastructure
- Both phases use OpenCV for webcam capture and display windows.
- Both are interactive and frame-driven, but they differ in what they do with the frames:
- phase 1: recognition and logging
- phase 2: gesture interpretation and system control
11. Model and dependency constraints
- face_recognition implies dlib-based face embeddings and requires Python 3.11 according to the readme.
- MediaPipe provides the hand-tracking model/runtime for the gesture phase.
- The architecture is therefore sensitive to Python/runtime version and Windows-specific dependencies.
12. Overall architecture diagram view
- The clearest diagram is: Webcam input feeding into two sequential processing pipelines.
- Pipeline A: camera → face recognition (face_recog.py) → attendance log (attendance.csv)
- Pipeline B: camera → hand landmark model (hand_landmarker.task) → gesture interpretation → OS mouse/volume control
- main.py sits above both as the control plane, while known_faces/ and attendance.csv are the persistent data boundaries.
Main entry points: main.py Gesture core · face_recog.py Face recog · volumenbrightness.py Volume module · known_faces Known faces
Components 14
Control Plane
Top-level orchestration and session flow
- main.py · Entry point that runs attendance first, then hands off to gesture control
main.py - Q switch · User keypress that transitions from attendance mode to gesture mouse mode
- Webcam · Shared live camera feed consumed by both phases
- OpenCV · Frame capture, display, and camera/window handling used across both workflows
Attendance Pipeline
Webcam face recognition and logging
- Face recog · Captures webcam frames, detects faces, matches identities, and records attendance
face_recog.py - Face engine · Embedding-based face comparison used to identify enrolled people
Gesture Pipeline
Hand tracking and desktop control
- Gesture core · Interprets hand landmarks into mouse and volume actions
main.py - Volume module · Audio-side gesture handling backed by system volume APIs
volumenbrightness.py - Hand tracking · Extracts hand landmarks and finger positions from webcam frames
Data & Model
Persistent inputs and bundled model assets
- Known faces · Static face image folder whose filenames define enrolled identities
known_faces - Attendance log · Append-only attendance artifact written with name, date, and time
- Hand model · Bundled hand landmark task file used for live hand tracking
hand_landmarker.task
Host OS
Desktop and audio actions triggered by gestures
- Mouse control · Pointer movement and click actions executed on the host desktop
- Volume control · System volume changes driven by right-hand gestures
Connections 16
- main.py → Face recog: start attendance · Main begins the face-recognition session first
- Face recog → Webcam: capture frames · Attendance reads live camera frames
- Webcam → OpenCV: stream · Camera feed enters the shared vision runtime
- OpenCV → Face engine: detect/match · Frames are processed for face detection and identity matching
- Face engine → Known faces: compare against · Enrollment images define recognized identities
- Face engine → Attendance log: write record · A matched person is logged once per session
- Face recog → Q switch: wait for q · Attendance phase blocks until the user advances
- Q switch → Gesture core: switch phase · The keypress transitions into gesture control
- main.py → Gesture core: run mouse mode · Main hands control to the gesture subsystem after attendance
- Gesture core → Webcam: capture frames · Gesture mode consumes the same live camera feed
- Webcam → Hand tracking: track hands · Frames are analyzed for hand landmarks
- Hand tracking → Hand model: use model · Landmark extraction depends on the bundled MediaPipe task
- Gesture core → Mouse control: move/click · Left-hand gestures drive cursor movement and clicks
- Gesture core → Volume module: route audio gestures · Right-hand gestures are sent to the audio control module
- Volume module → Volume control: adjust volume · Volume gestures manipulate host audio state
- Gesture core → OpenCV: display feedback · Interactive windows and frame overlays are managed through OpenCV