How synbol/parameter-efficient-transfer-learning-benchmark works
Project Type and Purpose
• This repo is a modular CLI-based benchmarking toolkit for visual parameter-efficient transfer learning (PETL). It is not a web app but a multi-task ML pipeline supporting Image Classification, Video Action Recognition, and Semantic Segmentation.
• It provides:
• A set of YAML configs for experiments (ImageClassification/configs, VideoActionRecognition, SemanticSegmentation).
• Data-loading & preprocessing modules.
• A library of PETL-augmented Vision Transformer backbones.
• Training & evaluation engines with orchestration scripts.
High-Level Architecture Components
• Config Manager (orange rectangle)
• Reads YAML files under ImageClassification/configs (and analogously for other tasks).
• Supplies experiment parameters (dataset, model type, hyperparameters).
• Local filesystem or remote (HuggingFace links) holding FGVC, VTAB, Kinetics-400, SSv2, COCO, ADE20K, VOC.
• Data Loader & Transform Pipeline (teal rectangle)
• Modules under */dataloader: implements dataset classes, CSV readers, PyTorch transforms.
• Model Factory (blue rectangle)
• Uses timm.create_model() plus custom wrappers in ImageClassification/models or VideoActionRecognition/models.
• Implements various PETL methods (Adapter, LoRA, BitFit, Hydra, etc.).
• Training Engine (green rectangle)
• Core loop in train/train.py (and engine_finetune.py for video).
• Manages optimizer, scheduler, forward/backward pass, checkpoint saving, and testing.
• Orchestration Scripts (gray parallelogram)
• Shell scripts under ImageClassification/scripts and VideoActionRecognition/train/*.sh.
• Invoke Python training scripts over all datasets.
• Utilities (purple rectangle)
• Logging, seeding, metric calculations in utils.py and */utils/*.
• External Services (brown cloud)
• Pretrained model storage (Google Cloud, GitHub releases), HuggingFace datasets.
Relationships & Data Flow
1. User runs a shell script (run_vit_lora.sh) →
2. Script parses args & calls Python training entrypoint (train_model_lora.py) →
3. Training script calls Config Manager (get_config()) →
4. Config Manager returns a dict of hyperparameters →
5. Training Engine requests Data Loader (vtab.get_data()) →
6. Data Loader reads Data Sources, applies Transforms, returns PyTorch DataLoaders →
7. Training Engine calls Model Factory to build a PETL-augmented model →
8. Training Engine enters train loop: forward, loss, backward, optimizer.step(), scheduler.step(), logging via Utilities →
9. At evaluation checkpoints, Training Engine calls test() on test DataLoader →
10. Metrics and checkpoints are written to disk.
Architectural Patterns & Design Principles
• Modular plugin design: new PETL methods are added as new modules in models/vision_transformer_<method>.py and corresponding train_model_<method>.py.
• Factory pattern: model instantiation via a unified interface (timm.create_model + custom tuning modes).
• Separation of concerns: configs ↔ data loaders ↔ models ↔ training engine ↔ scripts ↔ utils.
• YAML-driven experiments decouple code logic from hyperparameters.