# synbol/parameter-efficient-transfer-learning-benchmark architecture

> Architecture diagram of the GitHub repository synbol/parameter-efficient-transfer-learning-benchmark, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/synbol/parameter-efficient-transfer-learning-benchmark
- Repository: https://github.com/synbol/parameter-efficient-transfer-learning-benchmark

## Overview

##### Project Type and Purpose
- This repo is a modular CLI-based benchmarking toolkit for visual parameter-efficient transfer learning (PETL). It is not a web app but a multi-task ML pipeline supporting Image Classification, Video Action Recognition, and Semantic Segmentation.
- It provides:
- A set of YAML configs for experiments (ImageClassification/configs, VideoActionRecognition, SemanticSegmentation).
- Data-loading & preprocessing modules.
- A library of PETL-augmented Vision Transformer backbones.
- Training & evaluation engines with orchestration scripts.

##### High-Level Architecture Components
- Config Manager (orange rectangle)
- Reads YAML files under `ImageClassification/configs` (and analogously for other tasks).
- Supplies experiment parameters (dataset, model type, hyperparameters).
- Local filesystem or remote (HuggingFace links) holding FGVC, VTAB, Kinetics-400, SSv2, COCO, ADE20K, VOC.
- Data Loader & Transform Pipeline (teal rectangle)
- Modules under `*/dataloader`: implements dataset classes, CSV readers, PyTorch transforms.
- Model Factory (blue rectangle)
- Uses `timm.create_model()` plus custom wrappers in `ImageClassification/models` or `VideoActionRecognition/models`.
- Implements various PETL methods (Adapter, LoRA, BitFit, Hydra, etc.).
- Training Engine (green rectangle)
- Core loop in `train/train.py` (and `engine_finetune.py` for video).
- Manages optimizer, scheduler, forward/backward pass, checkpoint saving, and testing.
- Orchestration Scripts (gray parallelogram)
- Shell scripts under `ImageClassification/scripts` and `VideoActionRecognition/train/*.sh`.
- Invoke Python training scripts over all datasets.
- Utilities (purple rectangle)
- Logging, seeding, metric calculations in `utils.py` and `*/utils/*`.
- External Services (brown cloud)
- Pretrained model storage (Google Cloud, GitHub releases), HuggingFace datasets.

##### Relationships & Data Flow
1. User runs a shell script (`run_vit_lora.sh`) →
2. Script parses args & calls Python training entrypoint (`train_model_lora.py`) →
3. Training script calls Config Manager (`get_config()`) →
4. Config Manager returns a dict of hyperparameters →
5. Training Engine requests Data Loader (`vtab.get_data()`) →
6. Data Loader reads Data Sources, applies Transforms, returns PyTorch DataLoaders →
7. Training Engine calls Model Factory to build a PETL-augmented model →
8. Training Engine enters train loop: forward, loss, backward, optimizer.step(), scheduler.step(), logging via Utilities →
9. At evaluation checkpoints, Training Engine calls test() on test DataLoader →
10. Metrics and checkpoints are written to disk.

##### Architectural Patterns & Design Principles
- Modular plugin design: new PETL methods are added as new modules in `models/vision_transformer_<method>.py` and corresponding `train_model_<method>.py`.
- Factory pattern: model instantiation via a unified interface (`timm.create_model` + custom tuning modes).
- Separation of concerns: configs ↔ data loaders ↔ models ↔ training engine ↔ scripts ↔ utils.
- YAML-driven experiments decouple code logic from hyperparameters.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Image Classification Pipeline
    subgraph "Image Classification Pipeline"
        direction TB
        IC_Script["Orchestration Scripts"]:::scripts
        IC_Config["Config Manager"]:::config
        IC_Data["Data Loader & Transform Pipeline"]:::data
        IC_Model["Model Factory"]:::model
        IC_Entrypoints["Training Entrypoints"]:::training
        IC_Engine["Training Engine"]:::training
    end

    %% Video Action Recognition Pipeline
    subgraph "Video Action Recognition Pipeline"
        direction TB
        Video_Script["Orchestration Scripts"]:::scripts
        Video_Config["Config Manager"]:::config
        Video_Data["Data Loader & Transform Pipeline"]:::data
        Video_Model["Model Factory"]:::model
        Video_Entrypoints["Training Entrypoints"]:::training
        Video_Engine["Training Engine"]:::training
    end

    %% Semantic Segmentation Pipeline
    subgraph "Semantic Segmentation Pipeline"
        direction TB
        SS_Config["Config Manager"]:::config
        SS_Data["Data Loader & Transform Pipeline"]:::data
        SS_Model["Model Factory"]:::model
        SS_Entrypoints["Training Entrypoints"]:::training
        SS_Engine["Training Engine"]:::training
    end

    %% Common Utilities and External Services
    Utils["Utilities"]:::util
    Utils_Video["VideoActionRecognition Utilities"]:::util
    IO_Utils["Dataset I/O Utilities"]:::util
    ExtServices["External Services"]:::external

    %% Connections Image Classification
    IC_Script -->|"run_script()"| IC_Config
    IC_Config -->|"load_config()"| IC_Data
    IC_Data -->|"get_data()"| IC_Model
    IC_Model -->|"create_model()"| IC_Entrypoints
    IC_Entrypoints -->|"initialize()"| IC_Engine
    IC_Engine -->|"train_epoch()"| Utils
    IC_Engine -->|"log()"| IO_Utils
    IC_Engine -->|"save_checkpoint()"| ExtServices

    %% Connections Video Action Recognition
    Video_Script -->|"run_script()"| Video_Config
    Video_Config -->|"load_config()"| Video_Data
    Video_Data -->|"get_data()"| Video_Model
    Video_Model -->|"create_model()"| Video_Entrypoints
    Video_Entrypoints -->|"initialize()"| Video_Engine
    Video_Engine -->|"train_epoch()"| Utils_Video
    Video_Engine -->|"log()"| IO_Utils
    Video_Engine -->|"save_checkpoint()"| ExtServices

    %% Connections Semantic Segmentation
    SS_Config -->|"load_config()"| SS_Data
    SS_Data -->|"get_data()"| SS_Model
    SS_Model -->|"create_model()"| SS_Entrypoints
    SS_Entrypoints -->|"initialize()"| SS_Engine
    SS_Engine -->|"train_epoch()"| Utils
    SS_Engine -->|"log()"| IO_Utils
    SS_Engine -->|"save_checkpoint()"| ExtServices

    %% Legend
    subgraph Legend
        direction TB
        L1["Config Manager"]:::config
        L2["Data Pipeline"]:::data
        L3["Model Components"]:::model
        L4["Training Engine"]:::training
        L5["Orchestration Scripts"]:::scripts
        L6["Utilities"]:::util
        L7["External Services"]:::external
    end

    %% Click Events
    click IC_Config "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/tree/main/ImageClassification/configs"
    click IC_Data "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/tree/main/ImageClassification/dataloader"
    click Video_Data "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/tree/main/VideoActionRecognition/dataloader"
    click IC_Model "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/tree/main/ImageClassification/models"
    click Video_Model "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/tree/main/VideoActionRecognition/models"
    click IC_Engine "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/blob/main/ImageClassification/train/train.py"
    click IC_Entrypoints "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/blob/main/ImageClassification/train/train_model_*.py"
    click Video_Engine "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/blob/main/VideoActionRecognition/engine_finetune.py"
    click Video_Entrypoints "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/blob/main/VideoActionRecognition/train/train_model_*.py"
    click IC_Script "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/blob/main/ImageClassification/scripts/run_vit_*.sh"
    click Utils "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/blob/main/utils.py"
    click Utils_Video "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/tree/main/VideoActionRecognition/utils"
    click IO_Utils "https://github.com/synbol/parameter-efficient-transfer-learning-benchmark/blob/main/ImageClassification/dataloader/io_utils.py"

    %% Styles
    classDef config fill:#FFA500,stroke:#333,stroke-width:1px
    classDef data fill:#20B2AA,stroke:#333,stroke-width:1px
    classDef model fill:#1E90FF,stroke:#333,stroke-width:1px
    classDef training fill:#32CD32,stroke:#333,stroke-width:1px
    classDef scripts fill:#D3D3D3,stroke:#333,stroke-width:1px
    classDef util fill:#9370DB,stroke:#333,stroke-width:1px
    classDef external fill:#A52A2A,stroke:#333,stroke-width:1px
```
