# hcmusmaster/medvit architecture

> Architecture diagram of the GitHub repository hcmusmaster/medvit, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hcmusmaster/medvit
- Repository: https://github.com/hcmusmaster/medvit

## Overview

1. **Repository purpose**
   - This is a Python research repo for reproducing the **MedViT** medical image classifier and running experiments on three dataset families: **MedMNIST**, **PCAM**, and **LC25000**.
   - The architecture is centered on a pretrained vision backbone plus dataset-specific training/evaluation entrypoints.

2. **Core model layer**
   - The main model implementation lives in **`MedViT.py`** at the repo root.
   - There is also **`CustomDataset/MedViT.py`**, suggesting the model is reused inside the package-specific training pipeline for custom datasets.
   - In an architecture diagram, this is the central compute block: it receives image tensors and outputs classification logits.

3. **Data pipeline and dataset abstractions**
   - **`CustomDataset/datasets.py`** likely contains dataset wrappers and preprocessing logic.
   - **`CustomDataset/samplers.py`** indicates custom batching or class-balancing behavior, which is important for medical image class imbalance.
   - **`CustomDataset/download_pcam_parallel.sh`** and **`CustomDataset/download_checkpoint.sh`** show the repo includes dataset/checkpoint acquisition as part of the workflow.

4. **Training orchestration**
   - **`CustomDataset/engine.py`**, **`CustomDataset/losses.py`**, and **`CustomDataset/main.py`** form the training core for the package-based workflow.
   - These modules likely connect: dataset loader → sampler → model forward pass → loss computation → optimizer/update loop → checkpointing/metrics.
   - This is the main boundary where data preparation ends and model optimization begins.

5. **Experiment entrypoints**
   - Dataset-specific training scripts live under **`train/`**:
     - **`train/train_all_medmnist.py`**
     - **`train/train_pcam.py`**
     - **`train/train_lc25000.py`**
   - The README also mentions **`CustomDataset/train.sh`**, which suggests a shell-based launch path for the package workflow.
   - These scripts are the top-level orchestration layer an engineer would use to start experiments.

6. **Evaluation path**
   - Evaluation is separated into standalone scripts under **`test/`**:
     - **`test/evaluate_mnist_ckpt.py`**
     - **`test/evaluate_pcam_ckpt.py`**
     - **`test/evaluate_lc25000_ckpt.py`**
   - Each script loads a trained checkpoint, rebuilds the model and dataset pipeline, and computes metrics for the chosen dataset split.
   - This separation makes training and inference/evaluation distinct architectural stages.

7. **Checkpoint and transfer-learning flow**
   - The README references **ImageNet-1K pretrained MedViT checkpoints** for small/base/large variants.
   - The repo supports fine-tuning via flags such as **`--ckpt`** and **`--freeze-backbone`**, so the architecture includes an optional pretrained-weight boundary before task-specific heads are trained.
   - This is important because much of the repo’s value comes from transfer learning rather than training from scratch.

8. **Package and runtime structure**
   - The project is organized as an installable Python package via **`pyproject.toml`**, **`requirements.txt`**, and **`uv.lock`**.
   - **`.python-version`** suggests a pinned interpreter version.
   - This is a typical PyTorch-style research runtime: scripts are executed directly, but the codebase is packaged for local editable installs with **`pip install -e .`**.

9. **Shared utilities and supporting code**
   - There are utility modules at both the root (**`utils.py`**) and package level (**`CustomDataset/utils.py`**), likely handling logging, metrics, seed control, checkpoints, or transforms.
   - **`CustomDataset/__init__.py`** exposes the package namespace so training scripts can import the dataset stack cleanly.

10. **Documentation and reproducibility assets**
   - **`README.md`** describes the primary usage flow and pretrained resources.
   - **`CustomDataset.md`** and the notebooks (**`Colab_MedViT.ipynb`**, **`Instructions.ipynb`**) indicate interactive / instructional workflows alongside script-based runs.
   - **`images/structure.png`** is the model diagram referenced in the README; it documents the high-level MedViT hierarchy.

11. **External dependencies and data sources**
   - The repo explicitly depends on external medical image datasets: **PCAM** and **LC25000**, plus the MedMNIST suite.
   - Training and evaluation scripts expect a **`data_root`** argument, so data is external to the repo and mounted/pointed to at runtime.
   - The model also depends on external pretrained checkpoints, which are fetched separately rather than stored in source control.

12. **How the system fits together**
   - A clean mental model for the architecture is:  
     **dataset download / local data root → dataset wrapper & sampler → MedViT backbone → training engine & loss → checkpoint artifact → evaluation script**.
   - The repo’s main boundaries are therefore: **model definition**, **data loading/preprocessing**, **training orchestration**, and **evaluation/inference**.

Main entry points: [`CustomDataset/datasets.py`](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/datasets.py) (Datasets), [`MedViT.py`](https://github.com/hcmusmaster/medvit/blob/main/MedViT.py) (MedViT), [`CustomDataset/main.py`](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/main.py) (Main entry), [`README.md`](https://github.com/hcmusmaster/medvit/blob/main/README.md) (README)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Core model"]
  node_medvit_root["MedViT<br/>model<br/>[MedViT.py]"]
  node_medvit_pkg["Pkg MedViT<br/>model reuse<br/>[MedViT.py]"]
end

subgraph group_data["Data layer"]
  node_custom_dataset["Datasets<br/>dataset wrappers<br/>[datasets.py]"]
  node_samplers["Samplers<br/>batch policy<br/>[samplers.py]"]
  node_download_pcam["PCAM download<br/>data fetch"]
  node_download_ckpt["Checkpoint download<br/>artifact fetch"]
end

subgraph group_train["Training"]
  node_custom_main["Main entry<br/>training entrypoint<br/>[main.py]"]
  node_engine["Engine<br/>train loop<br/>[engine.py]"]
  node_losses["Losses<br/>objectives<br/>[losses.py]"]
  node_custom_utils["Pkg utils<br/>training utilities<br/>[utils.py]"]
  node_train_medmnist["Train MedMNIST<br/>experiment script"]
  node_train_pcam["Train PCAM<br/>experiment script<br/>[train_pcam.py]"]
  node_train_lc25000["Train LC25000<br/>experiment script<br/>[train_lc25000.py]"]
end

subgraph group_eval["Evaluation"]
  node_eval_mnist["Eval MedMNIST<br/>evaluation script"]
  node_eval_pcam["Eval PCAM<br/>evaluation script"]
  node_eval_lc25000["Eval LC25000<br/>evaluation script"]
end

subgraph group_docs["Docs & assets"]
  node_readme["README<br/>documentation<br/>[README.md]"]
  node_customdoc["CustomDataset doc<br/>documentation<br/>[CustomDataset.md]"]
end

subgraph group_runtime["Runtime"]
  node_pyproject["Python runtime<br/>packaging<br/>[pyproject.toml]"]
  node_requirements["Requirements<br/>dependencies<br/>[requirements.txt]"]
  node_uvlock["UV lock<br/>lockfile<br/>[uv.lock]"]
end

node_custom_main -->|"loads"| node_custom_dataset
node_custom_main -->|"uses"| node_samplers
node_custom_main -->|"trains"| node_medvit_pkg
node_custom_main -->|"runs"| node_engine
node_engine -->|"computes"| node_losses
node_engine -->|"relies on"| node_custom_utils
node_engine -->|"optimizes"| node_medvit_pkg
node_train_medmnist -->|"trains"| node_medvit_root
node_train_medmnist -->|"loads"| node_custom_dataset
node_train_pcam -->|"trains"| node_medvit_root
node_train_pcam -->|"loads"| node_custom_dataset
node_train_lc25000 -->|"trains"| node_medvit_root
node_train_lc25000 -->|"loads"| node_custom_dataset
node_eval_mnist -->|"loads"| node_medvit_root
node_eval_mnist -->|"evaluates"| node_custom_dataset
node_eval_pcam -->|"loads"| node_medvit_root
node_eval_pcam -->|"evaluates"| node_custom_dataset
node_eval_lc25000 -->|"loads"| node_medvit_root
node_eval_lc25000 -->|"evaluates"| node_custom_dataset
node_download_pcam -.->|"fetches data"| node_custom_dataset
node_download_ckpt -.->|"fetches weights"| node_medvit_root
node_readme -.->|"documents"| node_train_medmnist
node_readme -.->|"documents"| node_train_pcam
node_readme -.->|"documents"| node_train_lc25000
node_readme -.->|"documents"| node_eval_mnist
node_customdoc -.->|"documents"| node_custom_main
node_pyproject -.->|"packages"| node_custom_main
node_requirements -.->|"pins"| node_pyproject
node_uvlock -.->|"locks"| node_requirements

click node_medvit_root "https://github.com/hcmusmaster/medvit/blob/main/MedViT.py"
click node_medvit_pkg "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/MedViT.py"
click node_custom_dataset "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/datasets.py"
click node_samplers "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/samplers.py"
click node_download_pcam "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/download_pcam_parallel.sh"
click node_download_ckpt "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/download_checkpoint.sh"
click node_custom_main "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/main.py"
click node_engine "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/engine.py"
click node_losses "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/losses.py"
click node_custom_utils "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/utils.py"
click node_train_medmnist "https://github.com/hcmusmaster/medvit/blob/main/train/train_all_medmnist.py"
click node_train_pcam "https://github.com/hcmusmaster/medvit/blob/main/train/train_pcam.py"
click node_train_lc25000 "https://github.com/hcmusmaster/medvit/blob/main/train/train_lc25000.py"
click node_eval_mnist "https://github.com/hcmusmaster/medvit/blob/main/test/evaluate_mnist_ckpt.py"
click node_eval_pcam "https://github.com/hcmusmaster/medvit/blob/main/test/evaluate_pcam_ckpt.py"
click node_eval_lc25000 "https://github.com/hcmusmaster/medvit/blob/main/test/evaluate_lc25000_ckpt.py"
click node_readme "https://github.com/hcmusmaster/medvit/blob/main/README.md"
click node_customdoc "https://github.com/hcmusmaster/medvit/blob/main/CustomDataset.md"
click node_pyproject "https://github.com/hcmusmaster/medvit/blob/main/pyproject.toml"
click node_requirements "https://github.com/hcmusmaster/medvit/blob/main/requirements.txt"
click node_uvlock "https://github.com/hcmusmaster/medvit/blob/main/uv.lock"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_medvit_root,node_medvit_pkg toneBlue
class node_custom_dataset,node_samplers,node_download_pcam,node_download_ckpt toneAmber
class node_custom_main,node_engine,node_losses,node_custom_utils,node_train_medmnist,node_train_pcam,node_train_lc25000 toneMint
class node_eval_mnist,node_eval_pcam,node_eval_lc25000 toneRose
class node_readme,node_customdoc toneIndigo
class node_pyproject,node_requirements,node_uvlock toneTeal
```

## Components

### Core model

MedViT backbone and its package-level reuse.

- [MedViT](https://github.com/hcmusmaster/medvit/blob/main/MedViT.py): model · Primary MedViT implementation at the repository root. · `MedViT.py`
- [Pkg MedViT](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/MedViT.py): model reuse · Package-scoped MedViT variant used by the custom dataset pipeline. · `CustomDataset/MedViT.py`

### Data layer

Dataset wrappers, sampling, and acquisition helpers.

- [Datasets](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/datasets.py): dataset wrappers · Medical dataset loading, preprocessing, and split handling. · `CustomDataset/datasets.py`
- [Samplers](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/samplers.py): batch policy · Sampling and class-balancing logic for training. · `CustomDataset/samplers.py`
- [PCAM download](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/download_pcam_parallel.sh): data fetch · Parallel dataset download helper for PCAM. · `CustomDataset/download_pcam_parallel.sh`
- [Checkpoint download](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/download_checkpoint.sh): artifact fetch · Helper for fetching pretrained checkpoints. · `CustomDataset/download_checkpoint.sh`

### Training

Training orchestration, objectives, and launch scripts.

- [Main entry](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/main.py): training entrypoint · Package-based training launcher and CLI wiring. · `CustomDataset/main.py`
- [Engine](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/engine.py): train loop · Optimization loop, evaluation hooks, and checkpoint flow. · `CustomDataset/engine.py`
- [Losses](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/losses.py): objectives · Training losses used by the custom dataset pipeline. · `CustomDataset/losses.py`
- [Pkg utils](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset/utils.py): training utilities · Shared helpers for the package-level workflow. · `CustomDataset/utils.py`
- [Train MedMNIST](https://github.com/hcmusmaster/medvit/blob/main/train/train_all_medmnist.py): experiment script · Top-level training script for MedMNIST experiments. · `train/train_all_medmnist.py`
- [Train PCAM](https://github.com/hcmusmaster/medvit/blob/main/train/train_pcam.py): experiment script · Top-level training script for PCAM experiments. · `train/train_pcam.py`
- [Train LC25000](https://github.com/hcmusmaster/medvit/blob/main/train/train_lc25000.py): experiment script · Top-level training script for LC25000 experiments. · `train/train_lc25000.py`

### Evaluation

Checkpoint-based evaluation entrypoints.

- [Eval MedMNIST](https://github.com/hcmusmaster/medvit/blob/main/test/evaluate_mnist_ckpt.py): evaluation script · Checkpoint evaluation for MedMNIST runs. · `test/evaluate_mnist_ckpt.py`
- [Eval PCAM](https://github.com/hcmusmaster/medvit/blob/main/test/evaluate_pcam_ckpt.py): evaluation script · Checkpoint evaluation for PCAM runs. · `test/evaluate_pcam_ckpt.py`
- [Eval LC25000](https://github.com/hcmusmaster/medvit/blob/main/test/evaluate_lc25000_ckpt.py): evaluation script · Checkpoint evaluation for LC25000 runs. · `test/evaluate_lc25000_ckpt.py`

### Docs & assets

Usage guidance and supporting materials.

- [README](https://github.com/hcmusmaster/medvit/blob/main/README.md): documentation · Primary usage and workflow documentation. · `README.md`
- [CustomDataset doc](https://github.com/hcmusmaster/medvit/blob/main/CustomDataset.md): documentation · Supplementary package-specific documentation. · `CustomDataset.md`

### Runtime

Python packaging and dependency management.

- [Python runtime](https://github.com/hcmusmaster/medvit/blob/main/pyproject.toml): packaging · Project packaging and dependency definition. · `pyproject.toml`
- [Requirements](https://github.com/hcmusmaster/medvit/blob/main/requirements.txt): dependencies · Pinned Python dependencies for the repo. · `requirements.txt`
- [UV lock](https://github.com/hcmusmaster/medvit/blob/main/uv.lock): lockfile · Locked dependency resolution for the Python environment. · `uv.lock`

## Connections

- Main entry → Datasets: loads · Builds dataset pipelines for training.
- Main entry → Samplers: uses · Selects sampling strategy for batches.
- Main entry → Pkg MedViT: trains · Instantiates the package-level MedViT model.
- Main entry → Engine: runs · Hands off control to the training loop.
- Engine → Losses: computes · Evaluates the optimization objective.
- Engine → Pkg utils: relies on · Uses shared training helpers and metrics.
- Engine → Pkg MedViT: optimizes · Backpropagates through the MedViT model.
- Train MedMNIST → MedViT: trains · Top-level MedMNIST experiments use the core model.
- Train MedMNIST → Datasets: loads · Uses dataset wrappers for MedMNIST data.
- Train PCAM → MedViT: trains · PCAM experiments instantiate the core model.
- Train PCAM → Datasets: loads · Uses dataset wrappers for PCAM data.
- Train LC25000 → MedViT: trains · LC25000 experiments instantiate the core model.
- Train LC25000 → Datasets: loads · Uses dataset wrappers for LC25000 data.
- Eval MedMNIST → MedViT: loads · Rebuilds the model from a checkpoint.
- Eval MedMNIST → Datasets: evaluates · Reuses dataset pipeline for metrics.
- Eval PCAM → MedViT: loads · Rebuilds the model from a checkpoint.
- Eval PCAM → Datasets: evaluates · Reuses dataset pipeline for metrics.
- Eval LC25000 → MedViT: loads · Rebuilds the model from a checkpoint.
- Eval LC25000 → Datasets: evaluates · Reuses dataset pipeline for metrics.
- PCAM download → Datasets: fetches data · Prepares external PCAM data for training.
- Checkpoint download → MedViT: fetches weights · Retrieves pretrained MedViT checkpoints.
- README → Train MedMNIST: documents · Describes the MedMNIST training workflow.
- README → Train PCAM: documents · Describes the PCAM training workflow.
- README → Train LC25000: documents · Describes the LC25000 training workflow.
- README → Eval MedMNIST: documents · Describes checkpoint evaluation usage.
- CustomDataset doc → Main entry: documents · Explains package-specific execution flow.
- Python runtime → Main entry: packages · Defines editable package/runtime configuration.
- Requirements → Python runtime: pins · Lists runtime dependencies.
- UV lock → Requirements: locks · Locks the resolved dependency set.
