How hcmusmaster/medvit works
1. Repository purpose
- This is a Python research repo for reproducing the MedViT medical image classifier and running experiments on three dataset families: MedMNIST, PCAM, and LC25000.
- The architecture is centered on a pretrained vision backbone plus dataset-specific training/evaluation entrypoints.
2. Core model layer
- The main model implementation lives in MedViT.py at the repo root.
- There is also CustomDataset/MedViT.py, suggesting the model is reused inside the package-specific training pipeline for custom datasets.
- In an architecture diagram, this is the central compute block: it receives image tensors and outputs classification logits.
3. Data pipeline and dataset abstractions
- CustomDataset/datasets.py likely contains dataset wrappers and preprocessing logic.
- CustomDataset/samplers.py indicates custom batching or class-balancing behavior, which is important for medical image class imbalance.
- CustomDataset/download_pcam_parallel.sh and CustomDataset/download_checkpoint.sh show the repo includes dataset/checkpoint acquisition as part of the workflow.
4. Training orchestration
- CustomDataset/engine.py, CustomDataset/losses.py, and CustomDataset/main.py form the training core for the package-based workflow.
- These modules likely connect: dataset loader → sampler → model forward pass → loss computation → optimizer/update loop → checkpointing/metrics.
- This is the main boundary where data preparation ends and model optimization begins.
5. Experiment entrypoints
- Dataset-specific training scripts live under train/:
- train/train_all_medmnist.py
- train/train_pcam.py
- train/train_lc25000.py
- The README also mentions CustomDataset/train.sh, which suggests a shell-based launch path for the package workflow.
- These scripts are the top-level orchestration layer an engineer would use to start experiments.
6. Evaluation path
- Evaluation is separated into standalone scripts under test/:
- test/evaluate_mnist_ckpt.py
- test/evaluate_pcam_ckpt.py
- test/evaluate_lc25000_ckpt.py
- Each script loads a trained checkpoint, rebuilds the model and dataset pipeline, and computes metrics for the chosen dataset split.
- This separation makes training and inference/evaluation distinct architectural stages.
7. Checkpoint and transfer-learning flow
- The README references ImageNet-1K pretrained MedViT checkpoints for small/base/large variants.
- The repo supports fine-tuning via flags such as --ckpt and --freeze-backbone, so the architecture includes an optional pretrained-weight boundary before task-specific heads are trained.
- This is important because much of the repo’s value comes from transfer learning rather than training from scratch.
8. Package and runtime structure
- The project is organized as an installable Python package via pyproject.toml, requirements.txt, and uv.lock.
- .python-version suggests a pinned interpreter version.
- This is a typical PyTorch-style research runtime: scripts are executed directly, but the codebase is packaged for local editable installs with pip install -e ..
9. Shared utilities and supporting code
- There are utility modules at both the root (utils.py) and package level (CustomDataset/utils.py), likely handling logging, metrics, seed control, checkpoints, or transforms.
- CustomDataset/__init__.py exposes the package namespace so training scripts can import the dataset stack cleanly.
10. Documentation and reproducibility assets
- README.md describes the primary usage flow and pretrained resources.
- CustomDataset.md and the notebooks (Colab_MedViT.ipynb, Instructions.ipynb) indicate interactive / instructional workflows alongside script-based runs.
- images/structure.png is the model diagram referenced in the README; it documents the high-level MedViT hierarchy.
11. External dependencies and data sources
- The repo explicitly depends on external medical image datasets: PCAM and LC25000, plus the MedMNIST suite.
- Training and evaluation scripts expect a data_root argument, so data is external to the repo and mounted/pointed to at runtime.
- The model also depends on external pretrained checkpoints, which are fetched separately rather than stored in source control.
12. How the system fits together
- A clean mental model for the architecture is:
dataset download / local data root → dataset wrapper & sampler → MedViT backbone → training engine & loss → checkpoint artifact → evaluation script.
- The repo’s main boundaries are therefore: model definition, data loading/preprocessing, training orchestration, and evaluation/inference.
Main entry points: datasets.py Datasets · MedViT.py MedViT · main.py Main entry · README.md README
Components 21
Core model
MedViT backbone and its package-level reuse.
- MedViT · Primary MedViT implementation at the repository root.
MedViT.py - Pkg MedViT · Package-scoped MedViT variant used by the custom dataset pipeline.
CustomDataset/MedViT.py
Data layer
Dataset wrappers, sampling, and acquisition helpers.
- Datasets · Medical dataset loading, preprocessing, and split handling.
CustomDataset/datasets.py - Samplers · Sampling and class-balancing logic for training.
CustomDataset/samplers.py - PCAM download · Parallel dataset download helper for PCAM.
CustomDataset/download_pcam_parallel.sh - Checkpoint download · Helper for fetching pretrained checkpoints.
CustomDataset/download_checkpoint.sh
Training
Training orchestration, objectives, and launch scripts.
- Main entry · Package-based training launcher and CLI wiring.
CustomDataset/main.py - Engine · Optimization loop, evaluation hooks, and checkpoint flow.
CustomDataset/engine.py - Losses · Training losses used by the custom dataset pipeline.
CustomDataset/losses.py - Pkg utils · Shared helpers for the package-level workflow.
CustomDataset/utils.py - Train MedMNIST · Top-level training script for MedMNIST experiments.
train/train_all_medmnist.py - Train PCAM · Top-level training script for PCAM experiments.
train/train_pcam.py - Train LC25000 · Top-level training script for LC25000 experiments.
train/train_lc25000.py
Evaluation
Checkpoint-based evaluation entrypoints.
- Eval MedMNIST · Checkpoint evaluation for MedMNIST runs.
test/evaluate_mnist_ckpt.py - Eval PCAM · Checkpoint evaluation for PCAM runs.
test/evaluate_pcam_ckpt.py - Eval LC25000 · Checkpoint evaluation for LC25000 runs.
test/evaluate_lc25000_ckpt.py
Docs & assets
Usage guidance and supporting materials.
- README · Primary usage and workflow documentation.
README.md - CustomDataset doc · Supplementary package-specific documentation.
CustomDataset.md
Runtime
Python packaging and dependency management.
- Python runtime · Project packaging and dependency definition.
pyproject.toml - Requirements · Pinned Python dependencies for the repo.
requirements.txt - UV lock · Locked dependency resolution for the Python environment.
uv.lock
Connections 29
- Main entry → Datasets: loads · Builds dataset pipelines for training.
- Main entry → Samplers: uses · Selects sampling strategy for batches.
- Main entry → Pkg MedViT: trains · Instantiates the package-level MedViT model.
- Main entry → Engine: runs · Hands off control to the training loop.
- Engine → Losses: computes · Evaluates the optimization objective.
- Engine → Pkg utils: relies on · Uses shared training helpers and metrics.
- Engine → Pkg MedViT: optimizes · Backpropagates through the MedViT model.
- Train MedMNIST → MedViT: trains · Top-level MedMNIST experiments use the core model.
- Train MedMNIST → Datasets: loads · Uses dataset wrappers for MedMNIST data.
- Train PCAM → MedViT: trains · PCAM experiments instantiate the core model.
- Train PCAM → Datasets: loads · Uses dataset wrappers for PCAM data.
- Train LC25000 → MedViT: trains · LC25000 experiments instantiate the core model.
- Train LC25000 → Datasets: loads · Uses dataset wrappers for LC25000 data.
- Eval MedMNIST → MedViT: loads · Rebuilds the model from a checkpoint.
- Eval MedMNIST → Datasets: evaluates · Reuses dataset pipeline for metrics.
- Eval PCAM → MedViT: loads · Rebuilds the model from a checkpoint.
- Eval PCAM → Datasets: evaluates · Reuses dataset pipeline for metrics.
- Eval LC25000 → MedViT: loads · Rebuilds the model from a checkpoint.
- Eval LC25000 → Datasets: evaluates · Reuses dataset pipeline for metrics.
- PCAM download → Datasets: fetches data · Prepares external PCAM data for training.
- Checkpoint download → MedViT: fetches weights · Retrieves pretrained MedViT checkpoints.
- README → Train MedMNIST: documents · Describes the MedMNIST training workflow.
- README → Train PCAM: documents · Describes the PCAM training workflow.
- README → Train LC25000: documents · Describes the LC25000 training workflow.
- README → Eval MedMNIST: documents · Describes checkpoint evaluation usage.
- CustomDataset doc → Main entry: documents · Explains package-specific execution flow.
- Python runtime → Main entry: packages · Defines editable package/runtime configuration.
- Requirements → Python runtime: pins · Lists runtime dependencies.
- UV lock → Requirements: locks · Locks the resolved dependency set.