# rezaakb/peft-vit architecture

> Architecture diagram of the GitHub repository rezaakb/peft-vit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rezaakb/peft-vit
- Repository: https://github.com/rezaakb/peft-vit

## Overview

##### Project Type and Purpose
- This is a research-oriented machine-learning pipeline for parameter-efficient fine-tuning of self-supervised Vision Transformers (ViTs) on image classification tasks.
- It uses PyTorch (and PyTorch Lightning via `main.py`), configurable via YAML files under `configs/`, to run training (“fit”), evaluation (“test”), linear probing, LoRA, Block Expansion, or full fine-tuning on standard image datasets.

##### File-Structure Analysis
- Top level
- `main.py` – entry point with CLI (Lightning Trainer + Hydra-style config)
- `configs/` – organized config groups (`full`, `linear`, `lora`, `block`, `add_three`) each with dataset- and hyperparameter-specific YAMLs
- `src/` – core modules:
– `data.py` (data loaders, transforms)
– `model.py` (ViT instantiation, strategy wrappers)
– `loss.py`, `mixup.py`, `eval_knn.py`, `utils.py`
- `requirements.txt`, `README.md`
- This separation suggests a modular CLI-driven pipeline: configuration → data pipeline → model instantiation (with adaptation strategy) → training/evaluation → checkpoints/logs.

##### README Insights
- Describes four fine-tuning modes: full, linear probing, LoRA, Block Expansion, plus an extra “add_three” strategy.
- Demonstrates usage patterns (`python main.py fit --config configs/...`) and evaluation commands.
- Shows that models are pretrained ViTs (e.g., `vit-b16-224-in21k`) anddatasets include CIFAR-10/100, Pets37, Flowers102, Food101, DTD, or custom.

a. Main Components
- CLI & Config Loader (Hydra-like)
- Data Pipeline (datasets, augmentations, DataLoader)
- Model Component (ViT backbone + adaptation strategy plugin)
- Trainer Engine (PyTorch Lightning)
- Loss & Metric Modules (CrossEntropy, mixup, k-NN eval)
- Checkpointing & Logging (TensorBoard or Weights & Biases)
- Storage/Output (config snapshots, model checkpoints, logs)
- External Resources: GPU/CPU, File System, Remote Dataset URLs
b. Relationships & Interactions
1. User invokes `main.py` CLI → config loader parses selected YAML.
2. Config → instantiates DataModule (`src/data.py`) → loads dataset, applies transforms.
3. Config + model strategy choice → builds ViT model wrapper (`src/model.py`) with chosen parameter-efficient method.
4. Model + DataModule → handed to Trainer engine → runs training loops invoking loss (`src/loss.py`) and optional mixup (`src/mixup.py`).
5. After each epoch/step → checkpoints & logs written to disk (`output/...`).
6. Evaluation mode (`test`) → loads checkpoint + config → runs evaluation via `eval_knn.py` or Lightning’s test loop.
c. Architectural Patterns
- Modular CLI with Hydra-style config groups
- Data-Module pattern (separation of data loading from model)
- Strategy Pattern in Model (pluggable adapters for full, linear, LoRA, block)
- PyTorch Lightning orchestration (trainer abstraction)
d. Key Technologies
- PyTorch & PyTorch Lightning
- Vision Transformer (Timm or native ViT implementation)
- YAML-based configurations
- Optional external loggers (TensorBoard, WandB)

## Diagram (Mermaid)

```mermaid
graph TD
    subgraph "Configuration Layer"
        CLI["CLI/Config Loader"]:::orchestration
        Configs["Configuration Files (Hydra)"]:::orchestration
    end
    click CLI "https://github.com/rezaakb/peft-vit/blob/main/main.py"
    click Configs "https://github.com/rezaakb/peft-vit/tree/main/configs/"

    subgraph "Data Pipeline"
        DataModule["DataModule"]:::data
        ExternalDS["External Dataset URLs"]:::external
    end
    click DataModule "https://github.com/rezaakb/peft-vit/blob/main/src/data.py"

    subgraph "Model Layer"
        Model["ViT Model + Strategy"]:::model
    end
    click Model "https://github.com/rezaakb/peft-vit/blob/main/src/model.py"

    subgraph "Orchestration"
        Trainer["Trainer Engine"]:::orchestration
    end
    click Trainer "https://github.com/rezaakb/peft-vit/blob/main/main.py"

    subgraph "Loss & Metrics"
        Loss["Loss Modules"]:::data
        Mixup["Mixup Module"]:::data
        KNN["k-NN Evaluation"]:::data
    end
    click Loss "https://github.com/rezaakb/peft-vit/blob/main/src/loss.py"
    click Mixup "https://github.com/rezaakb/peft-vit/blob/main/src/mixup.py"
    click KNN "https://github.com/rezaakb/peft-vit/blob/main/src/eval_knn.py"

    Checkpoint["Checkpoint & Logging"]:::external
    click Checkpoint "https://github.com/rezaakb/peft-vit/blob/main/src/utils.py"

    FS["File System"]:::external
    GPU["GPU/CPU"]:::external
    Assets["Static Diagrams & Assets"]:::external
    Req["Package Dependencies"]:::external
    click Assets "https://github.com/rezaakb/peft-vit/tree/main/assets/"
    click Req "https://github.com/rezaakb/peft-vit/blob/main/requirements.txt"

    %% Connections
    CLI --> Configs
    Configs --> DataModule
    Configs --> Model
    DataModule --> Trainer
    Model --> Trainer
    Trainer --> Loss
    Trainer --> Mixup
    Trainer --> KNN
    Trainer --> Checkpoint
    DataModule --> ExternalDS
    Trainer --> FS
    Trainer --> GPU

    classDef data fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef model fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef orchestration fill:#FFA500,stroke:#000,stroke-width:1px;
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px;
```
