# prithanjan/virtual_mouse architecture

> Architecture diagram of the GitHub repository prithanjan/virtual_mouse, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/prithanjan/virtual_mouse
- Repository: https://github.com/prithanjan/virtual_mouse

## Overview

1. **Repository purpose and runtime shape**
   - This is a **single-machine, webcam-driven desktop application** in Python.
   - It combines two sequential computer-vision workflows: **face-recognition attendance** first, then a **gesture-controlled virtual mouse**.

2. **Top-level orchestration**
   - `main.py` is the entry point and the coordinator for the whole user flow.
   - It runs the attendance phase first, then switches into the gesture mouse phase after the user presses `q`.
   - The application appears to be **stateful within one process/session**, not a set of independent services.

3. **Phase 1: Face recognition attendance subsystem**
   - `face_recog.py` handles webcam-based face detection/recognition.
   - It compares live camera frames against images in `known_faces/`.
   - When a match is found, it records the person once per session and writes an attendance entry with **name, date, and time**.

4. **Face data boundary**
   - `known_faces/` is the input dataset for the attendance subsystem.
   - The naming convention of the files is part of the architecture: filenames act as the identity source for recognition.
   - This folder is effectively the system’s **static enrollment store**.

5. **Attendance output boundary**
   - Attendance is persisted to `attendance.csv`.
   - That file is an external artifact of the system, separate from the live runtime.
   - The readme indicates the file is auto-created and append-only in practice for daily logs.

6. **Phase switch / workflow control**
   - The attendance window acts as a gate before gesture control begins.
   - `q` is the explicit user-controlled transition signal from attendance to mouse control.
   - This makes the application a **two-stage interactive pipeline** rather than a continuously running background controller.

7. **Phase 2: Gesture virtual mouse subsystem**
   - The gesture mouse logic is likely split between `main.py` and `volumenbrightness.py`, with the latter handling the audio side.
   - Hand tracking is done with MediaPipe using the bundled `hand_landmarker.task` model file.
   - The left hand is used for pointer movement and clicking; the right hand is used for volume control.

8. **Input-to-action mapping**
   - Live webcam frames are analyzed for hand landmarks.
   - Landmark positions are translated into desktop actions:
     - index finger movement → cursor movement
     - index + middle finger proximity → click
     - thumb + index pinch → volume adjustment
   - This is the key interaction boundary between vision processing and OS control.

9. **OS integration boundary**
   - Mouse control is performed through **PyAutoGUI**.
   - Volume control uses **pycaw/comtypes**, which ties the app to Windows audio APIs.
   - This means the gesture layer is not just visual feedback; it directly manipulates host OS state.

10. **Shared computer-vision infrastructure**
   - Both phases use **OpenCV** for webcam capture and display windows.
   - Both are interactive and frame-driven, but they differ in what they do with the frames:
     - phase 1: recognition and logging
     - phase 2: gesture interpretation and system control

11. **Model and dependency constraints**
   - `face_recognition` implies `dlib`-based face embeddings and requires **Python 3.11** according to the readme.
   - MediaPipe provides the hand-tracking model/runtime for the gesture phase.
   - The architecture is therefore sensitive to Python/runtime version and Windows-specific dependencies.

12. **Overall architecture diagram view**
   - The clearest diagram is: **Webcam input** feeding into two sequential processing pipelines.
   - Pipeline A: camera → face recognition (`face_recog.py`) → attendance log (`attendance.csv`)
   - Pipeline B: camera → hand landmark model (`hand_landmarker.task`) → gesture interpretation → OS mouse/volume control
   - `main.py` sits above both as the control plane, while `known_faces/` and `attendance.csv` are the persistent data boundaries.

Main entry points: [`main.py`](https://github.com/prithanjan/virtual_mouse/blob/master/main.py) (Gesture core), [`face_recog.py`](https://github.com/prithanjan/virtual_mouse/blob/master/face_recog.py) (Face recog), [`volumenbrightness.py`](https://github.com/prithanjan/virtual_mouse/blob/master/volumenbrightness.py) (Volume module), [`known_faces`](https://github.com/prithanjan/virtual_mouse/tree/master/known_faces) (Known faces)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_grp_control["Control Plane"]
  node_main["main.py<br/>orchestrator<br/>[main.py]"]
  node_gate_q(("Q switch<br/>phase gate"))
  node_webcam(("Webcam<br/>video input"))
  node_cv_runtime["OpenCV<br/>vision runtime"]
end

subgraph group_grp_attendance["Attendance Pipeline"]
  node_face_recog["Face recog<br/>cv attendance<br/>[face_recog.py]"]
  node_face_engine{{"Face engine<br/>face recognition"}}
end

subgraph group_grp_gesture["Gesture Pipeline"]
  node_gesture_core["Gesture core<br/>gesture control<br/>[main.py]"]
  node_volumenbrightness["Volume module<br/>audio control"]
  node_hand_tracking{{"Hand tracking<br/>landmark extraction"}}
end

subgraph group_grp_data["Data & Model"]
  node_known_faces[("Known faces<br/>enrollment set")]
  node_attendance_log["Attendance log<br/>csv output"]
  node_hand_model["Hand model<br/>mediapipe model"]
end

subgraph group_grp_os["Host OS"]
  node_mouse_control["Mouse control<br/>desktop input"]
  node_volume_control["Volume control<br/>audio output"]
end

node_main -->|"start attendance"| node_face_recog
node_face_recog -->|"capture frames"| node_webcam
node_webcam -->|"stream"| node_cv_runtime
node_cv_runtime -->|"detect/match"| node_face_engine
node_face_engine -->|"compare against"| node_known_faces
node_face_engine -->|"write record"| node_attendance_log
node_face_recog -->|"wait for q"| node_gate_q
node_gate_q -->|"switch phase"| node_gesture_core
node_main -->|"run mouse mode"| node_gesture_core
node_gesture_core -->|"capture frames"| node_webcam
node_webcam -->|"track hands"| node_hand_tracking
node_hand_tracking -->|"use model"| node_hand_model
node_gesture_core -->|"move/click"| node_mouse_control
node_gesture_core -->|"route audio gestures"| node_volumenbrightness
node_volumenbrightness -->|"adjust volume"| node_volume_control
node_gesture_core -->|"display feedback"| node_cv_runtime

click node_main "https://github.com/prithanjan/virtual_mouse/blob/master/main.py"
click node_face_recog "https://github.com/prithanjan/virtual_mouse/blob/master/face_recog.py"
click node_known_faces "https://github.com/prithanjan/virtual_mouse/tree/master/known_faces"
click node_gesture_core "https://github.com/prithanjan/virtual_mouse/blob/master/main.py"
click node_hand_model "https://github.com/prithanjan/virtual_mouse/blob/master/hand_landmarker.task"
click node_volumenbrightness "https://github.com/prithanjan/virtual_mouse/blob/master/volumenbrightness.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_gate_q,node_webcam,node_cv_runtime toneBlue
class node_face_recog,node_face_engine toneAmber
class node_gesture_core,node_volumenbrightness,node_hand_tracking toneMint
class node_known_faces,node_attendance_log,node_hand_model toneRose
class node_mouse_control,node_volume_control toneIndigo
```

## Components

### Control Plane

Top-level orchestration and session flow

- [main.py](https://github.com/prithanjan/virtual_mouse/blob/master/main.py): orchestrator · Entry point that runs attendance first, then hands off to gesture control · `main.py`
- **Q switch**: phase gate · User keypress that transitions from attendance mode to gesture mouse mode
- **Webcam**: video input · Shared live camera feed consumed by both phases
- **OpenCV**: vision runtime · Frame capture, display, and camera/window handling used across both workflows

### Attendance Pipeline

Webcam face recognition and logging

- [Face recog](https://github.com/prithanjan/virtual_mouse/blob/master/face_recog.py): cv attendance · Captures webcam frames, detects faces, matches identities, and records attendance · `face_recog.py`
- **Face engine**: face recognition · Embedding-based face comparison used to identify enrolled people

### Gesture Pipeline

Hand tracking and desktop control

- [Gesture core](https://github.com/prithanjan/virtual_mouse/blob/master/main.py): gesture control · Interprets hand landmarks into mouse and volume actions · `main.py`
- [Volume module](https://github.com/prithanjan/virtual_mouse/blob/master/volumenbrightness.py): audio control · Audio-side gesture handling backed by system volume APIs · `volumenbrightness.py`
- **Hand tracking**: landmark extraction · Extracts hand landmarks and finger positions from webcam frames

### Data & Model

Persistent inputs and bundled model assets

- [Known faces](https://github.com/prithanjan/virtual_mouse/tree/master/known_faces): enrollment set · Static face image folder whose filenames define enrolled identities · `known_faces`
- **Attendance log**: csv output · Append-only attendance artifact written with name, date, and time
- [Hand model](https://github.com/prithanjan/virtual_mouse/blob/master/hand_landmarker.task): mediapipe model · Bundled hand landmark task file used for live hand tracking · `hand_landmarker.task`

### Host OS

Desktop and audio actions triggered by gestures

- **Mouse control**: desktop input · Pointer movement and click actions executed on the host desktop
- **Volume control**: audio output · System volume changes driven by right-hand gestures

## Connections

- main.py → Face recog: start attendance · Main begins the face-recognition session first
- Face recog → Webcam: capture frames · Attendance reads live camera frames
- Webcam → OpenCV: stream · Camera feed enters the shared vision runtime
- OpenCV → Face engine: detect/match · Frames are processed for face detection and identity matching
- Face engine → Known faces: compare against · Enrollment images define recognized identities
- Face engine → Attendance log: write record · A matched person is logged once per session
- Face recog → Q switch: wait for q · Attendance phase blocks until the user advances
- Q switch → Gesture core: switch phase · The keypress transitions into gesture control
- main.py → Gesture core: run mouse mode · Main hands control to the gesture subsystem after attendance
- Gesture core → Webcam: capture frames · Gesture mode consumes the same live camera feed
- Webcam → Hand tracking: track hands · Frames are analyzed for hand landmarks
- Hand tracking → Hand model: use model · Landmark extraction depends on the bundled MediaPipe task
- Gesture core → Mouse control: move/click · Left-hand gestures drive cursor movement and clicks
- Gesture core → Volume module: route audio gestures · Right-hand gestures are sent to the audio control module
- Volume module → Volume control: adjust volume · Volume gestures manipulate host audio state
- Gesture core → OpenCV: display feedback · Interactive windows and frame overlays are managed through OpenCV
