How foxprint666/cnn-based-drowsiness-detection-with-advanced-preprocessing-and-speed-optimisation works
1. What this repo is
- This is a local, real-time driver drowsiness detection system, not a web or service app.
- The runtime path is a computer-vision pipeline that ingests camera frames, extracts face/eye/mouth features, runs a PyTorch classifier, and triggers alerts.
2. Primary execution boundary
- main.py is the top-level orchestrator for the live system.
- config.py centralizes camera settings and detection thresholds, so the runtime behavior is mostly controlled by configuration rather than hardcoded values.
3. Core runtime pipeline
- The live system is split into four main modules under pipeline/:
- capture.py for frame acquisition
- preprocessing.py for face/landmark processing and feature extraction
- inference.py for eye-state model execution
- alerting.py for state evaluation and user notification
- The architecture is staged, so each module owns one phase of the data flow.
4. Frame acquisition subsystem
- pipeline/capture.py is the camera boundary.
- It reads frames from a local device using OpenCV, likely driven by CAMERA_ID, CAMERA_WIDTH, and CAMERA_HEIGHT.
- This is the only subsystem that touches the live video source.
5. Preprocessing and feature extraction
- pipeline/preprocessing.py handles face detection and landmark-based measurements.
- The repo uses MediaPipe Face Landmarker (models/face_landmarker.task) to get dense facial landmarks.
- From those landmarks it derives the key safety signals described in the README: eye-region crops / eye aspect ratio, mouth aspect ratio for yawning, and head pose estimates using cv2.solvePnP.
6. Model inference subsystem
- pipeline/inference.py loads and runs the PyTorch eye-state classifier.
- The model artifact is models/mobilenet_v2_eye_state.pth, which implies a MobileNetV2-based binary or probabilistic eye-state model.
- The README also indicates optional GPU acceleration via PyTorch AMP/autocast, so inference can run on CPU or CUDA depending on configuration.
7. Alerting and decision logic
- pipeline/alerting.py is the policy boundary that turns raw signals into user-facing outcomes.
- It combines the CNN eye-state probability with head-pose and yawning cues to decide between normal, distracted/yawning warning, and critical drowsiness alarm.
- Audio alerting is part of this layer, using a lightweight local playback dependency rather than any external notification service.
8. Data flow between modules
- The system is built as a multi-threaded async pipeline with queue handoffs between stages.
- Frames move forward from capture → preprocessing → inference → alerting, while intermediate features can be forwarded separately to the alerting logic.
- The important architectural property is that feature extraction and classification are decoupled, which helps real-time throughput and avoids blocking the camera thread.
9. Model and artifact boundary
- The models/ directory is the deployment bundle for runtime assets, not source code.
- It contains both the MediaPipe task file and the trained PyTorch weights, so the repo packages the full runtime dependency set needed for inference.
- This means the project has a clear split between offline-trained artifacts and online detection code.
10. Training and evaluation subsystem
- The train/ directory is a separate offline ML workflow:
- train/dataset.py for dataset loading/preparation
- train/train.py for training the eye-state model
- train/evaluate.py for validation/testing
- This is a distinct boundary from the live pipeline: training produces the .pth model consumed by pipeline/inference.py.
11. Technology profile that matters to architecture
- The repo depends on OpenCV for camera/image handling, MediaPipe for facial landmarks, PyTorch for the classifier, NumPy for numeric processing, and playsound for alerts.
- The system is therefore a desktop CV + ML application with local inference and no external backend or cloud dependency.
12. How to draw it
- A correct architecture diagram should show:
- a local camera feeding a capture stage,
- preprocessing/feature extraction as the central transform,
- a PyTorch inference stage for eye-state estimation,
- a rule-based alerting stage combining multiple signals,
- and a separate offline training path that produces the model artifact used at runtime.
Main entry points: preprocessing.py Preprocess · inference.py Eye inference · train.py Training · main.py Main loop
Components 12
Live detection
Real-time camera-to-alert pipeline
- Main loop · Top-level runtime entrypoint coordinating the live pipeline.
main.py - Config · Centralized camera and detection thresholds.
config.py - Frame capture · Reads frames from the local camera device.
pipeline/capture.py - Preprocess · Runs face/landmark processing and derives eye, mouth, and pose signals.
pipeline/preprocessing.py - Eye inference · Loads and executes the eye-state classifier.
pipeline/inference.py - Alert policy · Combines model output and behavioral cues to trigger warnings or alarms.
pipeline/alerting.py - Camera device · Live video source consumed by the capture stage.
Model assets
Packaged runtime artifacts used by the detector
- Face landmarker · Dense facial landmark model used for geometric measurements.
models/face_landmarker.task - Eye model · PyTorch MobileNetV2 eye-state classifier artifact.
models/mobilenet_v2_eye_state.pth
Offline training
Separate ML workflow that produces the runtime model
- Dataset prep · Loads and prepares training data for the eye-state model.
train/dataset.py - Training · Trains the classifier that becomes the runtime model artifact.
train/train.py - Evaluation · Validates and tests the trained model.
train/evaluate.py
Connections 13
- Main loop → Config: reads settings · Runtime behavior is driven by centralized configuration.
- Main loop → Frame capture: starts pipeline · Main loop launches the live capture stage.
- Camera device → Frame capture: video frames · Capture reads frames from the local device.
- Frame capture → Preprocess: frames · Raw frames flow into feature extraction.
- Preprocess → Face landmarker: landmarks · Preprocessing relies on packaged face-landmarker assets.
- Preprocess → Eye inference: eye crops · Derived eye-region features are passed to the classifier.
- Preprocess → Alert policy: pose/yawn cues · Behavioral signals are forwarded for decision making.
- Eye inference → Alert policy: eye state · Classifier output contributes to alert decisions.
- Alert policy → Main loop: status · Alerting returns the current driver state to the orchestrator.
- Eye inference → Eye model: loads weights · Inference consumes the trained PyTorch artifact.
- Training → Eye model: produces · Training emits the deployed model weights.
- Dataset prep → Training: training data · Prepared datasets feed the training stage.
- Training → Evaluation: validated by · Trained models are assessed by evaluation.