How ubicomplab/rppg-toolbox works
Project Type and Purpose
• This is a Python-based research toolbox (rPPG-Toolbox) for camera-based physiological sensing. It serves as both a benchmarking platform (unified CLI) and a development framework (plugin-style loaders, neural methods, and unsupervised methods).
• Key indicators: a single entrypoint (main.py), dozens of YAML config files under configs/, modular directories for dataset, neural_methods, unsupervised_methods, and an evaluation/ folder.
High-Level File/Directory Analysis
• main.py: CLI orchestrator that reads a YAML config, sets up data loaders, chooses supervised or unsupervised pipelines, runs training/testing/inference, and triggers evaluation.
• config.py + configs/: global definitions of all parameters; user-facing experiment definitions in YAML.
• dataset/: raw data ingestion, preprocessing, face detection (Haar/YOLO5Face), chunking, caching.
• neural_methods/: supervised neural modules, each with a model/ submodule (PyTorch architectures) and a trainer/ submodule (training, validation, testing routines).
• unsupervised_methods/: traditional rPPG algorithms abstracted under a common predictor interface.
• evaluation/: metric computation (MAE, MAPE, Bland-Altman, etc.), post-processing.
• tools/: auxiliary scripts and notebooks (data viz, motion augmentation via Mamba C++ library).
• final_model_release/ & model_outputs/: pre-trained weights and serialized results.
Core Components and Their Interactions
A. CLI & Config Manager
• main.py parses command-line args → loads YAML into a Config object.
B. Data Pipeline
2. DataLoader Factory (reads Config.DATASET → selects loader in dataset/data_loader/)
3. Preprocessor & Face Detector (Haar cascade or YOLO5Face module)
4. Chunker / Cropper / Augmenter (optional motion augmentation)
5. Cache Manager (writes preprocessed .npy to CACHED_PATH)
6. DataFileList Generator (train/val/test CSV lists)
C. Algorithm Pipelines (branching)
• Model Registry (imports models in neural_methods/model/)
• Trainer Factory (instantiates correct Trainer in neural_methods/trainer/)
• GPU Compute (PyTorch loop)
• Checkpoint Manager (saves best .pth models under LOG.PATH)
• Predictor (implements methods in unsupervised_methods/methods/)
D. Evaluation Engine
• Takes predictions (signal time series) + ground truth → metrics (in evaluation/)
• Generates plots (loss curves, Bland-Altman, scatter) → writes to LOG.PATH
E. Visualization Tools (notebook layer)
• tools/preprocessing_viz and tools/output_signal_viz for offline analysis
F. External Libraries/Services
• PyTorch, OpenCV, CUDA, YOLO5Face model weights, Mamba C++ via Python bindings
Architectural Patterns & Design Principles
• Plugin/Factory Pattern: data loaders, neural trainers, unsupervised methods are discovered and instantiated at runtime via naming conventions.
• Separation of Concerns: configuration, data ingestion, training/inference, evaluation, and visualization are isolated.
• Caching Layer: expensive preprocessing steps (face detection, cropping) are cached to disk.
• Extensibility Points: add new dataset loader under dataset/data_loader, new model/trainer in neural_methods/, or new unsupervised algorithm under unsupervised_methods/methods.
Essential Elements
• Plugin directories (dataset/data_loader, neural_methods/model, unsupervised_methods/methods) as extensibility zones.