# doem97/deblora architecture

> Architecture diagram of the GitHub repository doem97/deblora, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/doem97/deblora
- Repository: https://github.com/doem97/deblora

## Overview

#### Purpose and execution surface
This repository implements experiments for “De-Biased Representation Learning for Long-tailed PEFT in Remote Sensing.” Its workflow compares zero-shot CLIP, full fine-tuning, LoRA, and calibrated/pLoRA-style representations, then evaluates them through feature extraction, clustering/calibration, and linear probing. Primary Python entry points are `main.py` and `trainer.py`; reproducible experiment sequences are exposed as shell launchers under `exp/`, such as `exp/extract_lora_feat.sh` and `exp/lora_linprob.sh`.

#### Configuration-driven experiment selection
Experiments are composed from dataset, backbone, and tuner YAMLs rather than hardcoded per-dataset programs. Dataset definitions cover long-tailed natural-image and remote-sensing datasets; model definitions select CLIP, ImageNet-21K ViT, or SatMAE variants; tuner definitions select Flora/LoRA parameter-efficient adaptation variants.
- `configs/data/dota.yaml`
- `configs/model/clip_vit_b16.yaml`
- `configs/tuner/flora.yaml`

#### Dataset ingestion and long-tail protocol
Dataset adapters provide class labels and train/validation/test splits for CIFAR-100, DOTA, FUSRS, ImageNet-LT, iNaturalist 2018, and Places-LT. Shared long-tailed data handling is centralized separately from individual dataset loaders. Dataset metadata and split files are repository assets, while image/data roots are expected under `data/` or linked there.
- `datasets/lt_data.py`
- `datasets/dota.py`
- `datasets/imagenet_lt.py`

#### Backbone and adaptation layer
The model layer wraps visual encoders and classifiers, including CLIP visual/text processing, SatMAE ViTs, and PEFT modules. The vendored `clip/` package supplies CLIP model construction and tokenization; project wrappers in `models/` integrate these encoders into training/probing workflows. A second `src/models/` branch supports DIFT/diffusion-derived feature backbones and alternate CLIP/DINO implementations.
- `models/models.py`
- `models/peft_modules.py`
- `clip/model.py`

#### Feature production and persistence boundary
Feature extraction is a first-class intermediate stage: zero-shot, fine-tuned, and LoRA representations are extracted and written for later reuse. Precomputed feature artifacts can be downloaded through Hugging Face-oriented scripts, decoupling expensive encoder inference from downstream clustering and probes. Diffusion/intermediate feature extraction scripts form a parallel path for remote-sensing experiments.
- `exp/extract_0shot_feat.sh`
- `output/features/hf_features.sh`
- `scripts/extract_feat_mid+lora.py`

#### Calibration, clustering, and evaluation flow
The pLoRA workflow consumes LoRA features, clusters feature space, calibrates representations, and then runs linear probes. Clustering supports CPU/GPU K-means and GPU HDBSCAN, indicating optional RAPIDS/CUDA infrastructure; `rapids.hdbscan.doc` documents that dependency boundary. Classifier/probe implementations are separated from model adaptation code.
- `scripts/feat_cluster_kmeans_gpu_fp32_v0.3.py`
- `scripts/feat_cluster_hdbscan_gpu_fp32_v0.4.py`
- `models/classifiers.py`

#### Runtime and external dependencies
The project is Python 3.8-oriented, uses PyTorch-style model training/inference implied by the CLIP and ViT implementations, and installs dependencies from `requirements.txt`. External services/assets include Hugging Face-hosted datasets/features and locally provisioned image datasets. GPU execution is material for LoRA training, feature extraction, and GPU clustering; CPU clustering alternatives are also present.
- `requirements.txt`
- `data/hf_dataset.sh`
- `scripts/feat_cluster_cpu_kmeans_v0.2.py`

Main entry points: [`main.py`](https://github.com/doem97/deblora/blob/main/main.py) (Main entry point), [`trainer.py`](https://github.com/doem97/deblora/blob/main/trainer.py) (Training orchestration), [`output/features/hf_features.sh`](https://github.com/doem97/deblora/blob/main/output/features/hf_features.sh) (Downloaded features), [`models/models.py`](https://github.com/doem97/deblora/blob/main/models/models.py) (Encoder wrappers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_config["Experiment Configuration"]
  node_dataset_config["Dataset config<br/>YAML<br/>[dota.yaml]"]
  node_model_config["Backbone config<br/>YAML<br/>[clip_vit_b16.yaml]"]
  node_tuner_config["Tuner config<br/>YAML<br/>[flora.yaml]"]
end

subgraph group_data["Data and Assets"]
  node_dataset_adapters["Dataset adapters<br/>dataset loaders<br/>[dota.py]"]
  node_longtail_data["Long-tail protocol<br/>data handling<br/>[lt_data.py]"]
  node_local_datasets[("Local image datasets<br/>dataset assets<br/>[hf_dataset.sh]")]
  node_huggingface_features[("Downloaded features<br/>feature assets<br/>[hf_features.sh]")]
end

subgraph group_models["Models and Adaptation"]
  node_model_wrappers["Encoder wrappers<br/>model integration<br/>[models.py]"]
  node_peft["LoRA / Flora modules<br/>PEFT modules<br/>[peft_modules.py]"]
  node_clip["Vendored CLIP<br/>CLIP encoder<br/>[model.py]"]
  node_classifiers["Probe classifiers<br/>classifier implementations<br/>[classifiers.py]"]
end

subgraph group_pipeline["Experiment Pipeline"]
  node_launchers["Experiment launchers<br/>shell workflows"]
  node_main["Main entry point<br/>Python entry point<br/>[main.py]"]
  node_trainer["Training orchestration<br/>trainer<br/>[trainer.py]"]
  node_feature_extraction["Feature extraction<br/>representation export"]
  node_diffusion_features["Diffusion feature path<br/>alternate extraction"]
  node_gpu_clustering["GPU clustering<br/>K-means / HDBSCAN"]
  node_cpu_clustering["CPU clustering<br/>K-means"]
  node_calibrated_features[("Calibrated representations<br/>feature artifacts")]
  node_linear_probe["Linear probing<br/>evaluation<br/>[lora_linprob.sh]"]
end

node_dependencies["Python dependencies<br/>runtime requirements<br/>[requirements.txt]"]

node_launchers -->|"starts"| node_main
node_main -->|"orchestrates"| node_trainer
node_dataset_config -->|"selects data"| node_main
node_model_config -->|"selects backbone"| node_main
node_tuner_config -->|"selects adaptation"| node_main
node_trainer -->|"uses"| node_longtail_data
node_longtail_data -->|"coordinates splits"| node_dataset_adapters
node_dataset_adapters -->|"loads"| node_local_datasets
node_trainer -->|"trains"| node_model_wrappers
node_model_wrappers -->|"wraps"| node_clip
node_model_wrappers -->|"applies"| node_peft
node_trainer -->|"produces models for"| node_feature_extraction
node_feature_extraction -->|"writes or reuses"| node_huggingface_features
node_diffusion_features -.->|"adds alternate features"| node_huggingface_features
node_huggingface_features -->|"clusters"| node_gpu_clustering
node_huggingface_features -.->|"clusters"| node_cpu_clustering
node_gpu_clustering -->|"calibrates"| node_calibrated_features
node_cpu_clustering -.->|"calibrates"| node_calibrated_features
node_calibrated_features -->|"evaluates"| node_linear_probe
node_linear_probe -->|"uses"| node_classifiers
node_dependencies -.->|"runtime"| node_main

click node_launchers "https://github.com/doem97/deblora/blob/main/exp/extract_lora_feat.sh"
click node_main "https://github.com/doem97/deblora/blob/main/main.py"
click node_trainer "https://github.com/doem97/deblora/blob/main/trainer.py"
click node_dataset_config "https://github.com/doem97/deblora/blob/main/configs/data/dota.yaml"
click node_model_config "https://github.com/doem97/deblora/blob/main/configs/model/clip_vit_b16.yaml"
click node_tuner_config "https://github.com/doem97/deblora/blob/main/configs/tuner/flora.yaml"
click node_dataset_adapters "https://github.com/doem97/deblora/blob/main/datasets/dota.py"
click node_longtail_data "https://github.com/doem97/deblora/blob/main/datasets/lt_data.py"
click node_local_datasets "https://github.com/doem97/deblora/blob/main/data/hf_dataset.sh"
click node_huggingface_features "https://github.com/doem97/deblora/blob/main/output/features/hf_features.sh"
click node_model_wrappers "https://github.com/doem97/deblora/blob/main/models/models.py"
click node_peft "https://github.com/doem97/deblora/blob/main/models/peft_modules.py"
click node_clip "https://github.com/doem97/deblora/blob/main/clip/model.py"
click node_feature_extraction "https://github.com/doem97/deblora/blob/main/exp/extract_0shot_feat.sh"
click node_diffusion_features "https://github.com/doem97/deblora/blob/main/scripts/extract_feat_mid%2Blora.py"
click node_gpu_clustering "https://github.com/doem97/deblora/blob/main/scripts/feat_cluster_kmeans_gpu_fp32_v0.3.py"
click node_cpu_clustering "https://github.com/doem97/deblora/blob/main/scripts/feat_cluster_cpu_kmeans_v0.2.py"
click node_linear_probe "https://github.com/doem97/deblora/blob/main/exp/lora_linprob.sh"
click node_classifiers "https://github.com/doem97/deblora/blob/main/models/classifiers.py"
click node_dependencies "https://github.com/doem97/deblora/blob/main/requirements.txt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset_config,node_model_config,node_tuner_config toneBlue
class node_dataset_adapters,node_longtail_data,node_local_datasets,node_huggingface_features toneAmber
class node_model_wrappers,node_peft,node_clip,node_classifiers toneMint
class node_launchers,node_main,node_trainer,node_feature_extraction,node_diffusion_features,node_gpu_clustering,node_cpu_clustering,node_calibrated_features,node_linear_probe toneRose
class node_dependencies toneNeutral
```

## Components

### Experiment Configuration

Composable dataset, backbone, and tuner selection.

- [Dataset config](https://github.com/doem97/deblora/blob/main/configs/data/dota.yaml): YAML · `configs/data/dota.yaml`
- [Backbone config](https://github.com/doem97/deblora/blob/main/configs/model/clip_vit_b16.yaml): YAML · `configs/model/clip_vit_b16.yaml`
- [Tuner config](https://github.com/doem97/deblora/blob/main/configs/tuner/flora.yaml): YAML · `configs/tuner/flora.yaml`

### Data and Assets

Long-tailed datasets and reusable feature artifacts.

- [Dataset adapters](https://github.com/doem97/deblora/blob/main/datasets/dota.py): dataset loaders · Provide labels and train, validation, and test splits. · `datasets/dota.py`
- [Long-tail protocol](https://github.com/doem97/deblora/blob/main/datasets/lt_data.py): data handling · Shared long-tailed sampling and split handling. · `datasets/lt_data.py`
- [Local image datasets](https://github.com/doem97/deblora/blob/main/data/hf_dataset.sh): dataset assets · Image roots are provisioned or linked locally. · `data/hf_dataset.sh`
- [Downloaded features](https://github.com/doem97/deblora/blob/main/output/features/hf_features.sh): feature assets · Precomputed artifacts decouple inference from downstream analysis. · `output/features/hf_features.sh`

### Models and Adaptation

Encoders, CLIP, and PEFT adaptation.

- [Encoder wrappers](https://github.com/doem97/deblora/blob/main/models/models.py): model integration · Integrates visual encoders into training and probing. · `models/models.py`
- [LoRA / Flora modules](https://github.com/doem97/deblora/blob/main/models/peft_modules.py): PEFT modules · `models/peft_modules.py`
- [Vendored CLIP](https://github.com/doem97/deblora/blob/main/clip/model.py): CLIP encoder · Constructs CLIP models and tokenization. · `clip/model.py`
- [Probe classifiers](https://github.com/doem97/deblora/blob/main/models/classifiers.py): classifier implementations · `models/classifiers.py`

### Experiment Pipeline

Training, extraction, calibration, and probing.

- [Experiment launchers](https://github.com/doem97/deblora/blob/main/exp/extract_lora_feat.sh): shell workflows · Reproducible sequences for extraction and linear probing. · `exp/extract_lora_feat.sh`
- [Main entry point](https://github.com/doem97/deblora/blob/main/main.py): Python entry point · `main.py`
- [Training orchestration](https://github.com/doem97/deblora/blob/main/trainer.py): trainer · Runs full fine-tuning and PEFT experiments. · `trainer.py`
- [Feature extraction](https://github.com/doem97/deblora/blob/main/exp/extract_0shot_feat.sh): representation export · Produces zero-shot, fine-tuned, and LoRA representations. · `exp/extract_0shot_feat.sh`
- [Diffusion feature path](https://github.com/doem97/deblora/blob/main/scripts/extract_feat_mid%2Blora.py): alternate extraction · Extracts intermediate diffusion-derived remote-sensing features. · `scripts/extract_feat_mid+lora.py`
- [GPU clustering](https://github.com/doem97/deblora/blob/main/scripts/feat_cluster_kmeans_gpu_fp32_v0.3.py): K-means / HDBSCAN · Optional CUDA/RAPIDS clustering for calibration. · `scripts/feat_cluster_kmeans_gpu_fp32_v0.3.py`
- [CPU clustering](https://github.com/doem97/deblora/blob/main/scripts/feat_cluster_cpu_kmeans_v0.2.py): K-means · CPU alternative to GPU clustering. · `scripts/feat_cluster_cpu_kmeans_v0.2.py`
- **Calibrated representations**: feature artifacts · pLoRA-style representations produced from clustered LoRA features.
- [Linear probing](https://github.com/doem97/deblora/blob/main/exp/lora_linprob.sh): evaluation · Evaluates extracted and calibrated representations. · `exp/lora_linprob.sh`

### Other components

- [Python dependencies](https://github.com/doem97/deblora/blob/main/requirements.txt): runtime requirements · Python 3.8 and PyTorch-oriented runtime dependencies. · `requirements.txt`

## Connections

- Experiment launchers → Main entry point: starts
- Main entry point → Training orchestration: orchestrates
- Dataset config → Main entry point: selects data
- Backbone config → Main entry point: selects backbone
- Tuner config → Main entry point: selects adaptation
- Training orchestration → Long-tail protocol: uses
- Long-tail protocol → Dataset adapters: coordinates splits
- Dataset adapters → Local image datasets: loads
- Training orchestration → Encoder wrappers: trains
- Encoder wrappers → Vendored CLIP: wraps
- Encoder wrappers → LoRA / Flora modules: applies
- Training orchestration → Feature extraction: produces models for
- Feature extraction → Downloaded features: writes or reuses
- Diffusion feature path → Downloaded features: adds alternate features
- Downloaded features → GPU clustering: clusters
- Downloaded features → CPU clustering: clusters
- GPU clustering → Calibrated representations: calibrates
- CPU clustering → Calibrated representations: calibrates
- Calibrated representations → Linear probing: evaluates
- Linear probing → Probe classifiers: uses
- Python dependencies → Main entry point: runtime