# microsoft/swin-transformer architecture

> Architecture diagram of the GitHub repository microsoft/swin-transformer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/microsoft/swin-transformer
- Repository: https://github.com/microsoft/swin-transformer

## Overview

#### Purpose and execution surface
This is a PyTorch training/evaluation repository for ImageNet image classification with Swin-family hierarchical vision backbones. It supports supervised Swin/SwinV2/Swin-MLP training, SimMIM masked-image pretraining and fine-tuning, and distributed Swin-MoE runs. Primary entry points are `main.py`, `main_moe.py`, `main_simmim_pt.py`, and `main_simmim_ft.py`.

#### Configuration-driven experiment selection
Runtime behavior is selected from YAML architecture/task presets and merged configuration logic. Configs encode model scale, resolution, dataset/class count, optimization, and training schedule; command-line options can override them.
- `config.py`
- `configs/swin/swin_tiny_patch4_window7_224.yaml`
- `configs/simmim/simmim_pretrain__swinv2_base__img192_window12__800ep.yaml`

#### Standard supervised training pipeline
`main.py` orchestrates distributed initialization, config loading, model/dataloader/optimizer/scheduler construction, checkpoint resume or pretrained-weight loading, epoch training, validation, and artifact logging. The main control flow is configuration → dataset loaders → model forward/loss/backpropagation → optimizer/scheduler update → checkpoint and metrics.
- `main.py`
- `optimizer.py`
- `lr_scheduler.py`

#### Data ingestion boundary
The data layer creates PyTorch datasets and loaders for ImageNet-style folders, including ImageNet-22K label handling and optional zip/cached-image access. Distributed-aware samplers partition training work across processes. This isolates filesystem/archive layout and label mapping from model execution.
- `data/build.py`
- `data/cached_image_folder.py`
- `data/imagenet22k_dataset.py`

#### Model factory and backbone variants
A model builder dispatches configured model types to Swin V1, Swin V2, or shifted-window MLP implementations. Swin backbones implement hierarchical patch embedding, alternating regular/shifted local-window stages, patch merging, normalization, and classifier head. Model-specific code is separated from training orchestration.
- `models/build.py`
- `models/swin_transformer.py`
- `models/swin_transformer_v2.py`
- `models/swin_mlp.py`

#### SimMIM self-supervised branch
SimMIM has dedicated launchers and dataset paths for masking-based pretraining and downstream fine-tuning. The SimMIM wrapper combines a selected Swin/SwinV2 encoder with masked-image reconstruction behavior; pretrained representations then flow into the fine-tuning runner.
- `main_simmim_pt.py`
- `main_simmim_ft.py`
- `models/simmim.py`
- `data/data_simmim_pt.py`

#### Mixture-of-Experts branch
Swin-MoE uses a separate runner and model implementation because expert routing and distributed execution differ materially from dense models. It relies on the external Tutel MoE system and supplies MoE-specific runtime utilities and configurations targeting multi-GPU expert layouts.
- `main_moe.py`
- `models/swin_transformer_moe.py`
- `utils_moe.py`
- `configs/swinmoe/swin_moe_small_patch4_window12_192_64expert_64gpu_22k.yaml`

#### Runtime, acceleration, and persistence
The material runtime is Python/PyTorch on CUDA-capable GPUs, with distributed multi-process training and optional mixed precision/Apex noted by the project. Standard utilities centralize distributed setup, checkpointing, metric reduction, and logging. An optional compiled CUDA/C++ window-processing extension accelerates window partition/reversal operations at the Python boundary.
- `utils.py`
- `logger.py`
- `kernels/window_process/window_process.py`
- `kernels/window_process/swin_window_process_kernel.cu`

Main entry points: [`main.py`](https://github.com/microsoft/swin-transformer/blob/main/main.py) (Supervised runner), [`models/build.py`](https://github.com/microsoft/swin-transformer/blob/main/models/build.py) (Model factory), [`configs/swin/swin_tiny_patch4_window7_224.yaml`](https://github.com/microsoft/swin-transformer/blob/main/configs/swin/swin_tiny_patch4_window7_224.yaml) (YAML presets), [`data/build.py`](https://github.com/microsoft/swin-transformer/blob/main/data/build.py) (Dataset and loaders)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experiment["Experiment control"]
  node_configs["YAML presets<br/>experiment configs"]
  node_config_merge["Config merge<br/>configuration<br/>[config.py]"]
  node_supervised_runner["Supervised runner<br/>training entry point<br/>[main.py]"]
  node_optimizer["Optimizer builder<br/>optimization<br/>[optimizer.py]"]
  node_scheduler["LR scheduler<br/>optimization<br/>[lr_scheduler.py]"]
  node_simmim_pretrain["SimMIM pretraining<br/>training entry point<br/>[main_simmim_pt.py]"]
  node_simmim_finetune["SimMIM fine-tuning<br/>training entry point<br/>[main_simmim_ft.py]"]
  node_moe_runner["Swin-MoE runner<br/>distributed entry point<br/>[main_moe.py]"]
end

subgraph group_data["Data ingestion"]
  node_data_build["Dataset and loaders<br/>data builder<br/>[build.py]"]
  node_cached_images["Cached/zip images<br/>dataset backend"]
  node_imagenet22k["ImageNet-22K labels<br/>dataset backend"]
  node_simmim_data["SimMIM data path<br/>pretraining data<br/>[data_simmim_pt.py]"]
end

subgraph group_models["Model variants"]
  node_model_factory["Model factory<br/>model builder<br/>[build.py]"]
  node_swin_backbones["Swin dense backbones<br/>vision backbone"]
  node_swinv2["Swin V2 backbone<br/>vision backbone"]
  node_swin_mlp["Swin-MLP backbone<br/>vision backbone<br/>[swin_mlp.py]"]
  node_simmim_model["SimMIM wrapper<br/>masked reconstruction model<br/>[simmim.py]"]
  node_moe_model["Swin-MoE model<br/>expert-routed backbone"]
end

subgraph group_runtime["Distributed runtime"]
  node_moe_runtime["MoE runtime utilities<br/>distributed utilities<br/>[utils_moe.py]"]
  node_runtime_utils["Runtime utilities<br/>distributed runtime<br/>[utils.py]"]
  node_logger["Artifact logger<br/>logging<br/>[logger.py]"]
  node_window_extension["CUDA window extension<br/>CUDA/C++ acceleration<br/>[window_process.py]"]
end

node_configs -->|"preset"| node_config_merge
node_config_merge -->|"runtime config"| node_supervised_runner
node_supervised_runner -->|"builds loaders"| node_data_build
node_data_build -.->|"optional backend"| node_cached_images
node_data_build -.->|"22K support"| node_imagenet22k
node_supervised_runner -->|"builds model"| node_model_factory
node_model_factory -->|"Swin V1"| node_swin_backbones
node_model_factory -->|"Swin V2"| node_swinv2
node_model_factory -->|"Swin-MLP"| node_swin_mlp
node_supervised_runner -->|"builds"| node_optimizer
node_supervised_runner -->|"updates"| node_scheduler
node_supervised_runner -->|"distributed and checkpoints"| node_runtime_utils
node_runtime_utils -->|"metrics and artifacts"| node_logger
node_swin_backbones -.->|"accelerated windows"| node_window_extension
node_simmim_pretrain -->|"masked inputs"| node_simmim_data
node_simmim_pretrain -->|"trains"| node_simmim_model
node_simmim_model -->|"encoder option"| node_swinv2
node_simmim_finetune -->|"loads representations"| node_simmim_model
node_moe_runner -->|"trains"| node_moe_model
node_moe_runner -->|"distributed execution"| node_moe_runtime
node_configs -.->|"SimMIM preset"| node_simmim_pretrain
node_configs -.->|"MoE preset"| node_moe_runner

click node_configs "https://github.com/microsoft/swin-transformer/blob/main/configs/swin/swin_tiny_patch4_window7_224.yaml"
click node_config_merge "https://github.com/microsoft/swin-transformer/blob/main/config.py"
click node_supervised_runner "https://github.com/microsoft/swin-transformer/blob/main/main.py"
click node_optimizer "https://github.com/microsoft/swin-transformer/blob/main/optimizer.py"
click node_scheduler "https://github.com/microsoft/swin-transformer/blob/main/lr_scheduler.py"
click node_data_build "https://github.com/microsoft/swin-transformer/blob/main/data/build.py"
click node_cached_images "https://github.com/microsoft/swin-transformer/blob/main/data/cached_image_folder.py"
click node_imagenet22k "https://github.com/microsoft/swin-transformer/blob/main/data/imagenet22k_dataset.py"
click node_model_factory "https://github.com/microsoft/swin-transformer/blob/main/models/build.py"
click node_swin_backbones "https://github.com/microsoft/swin-transformer/blob/main/models/swin_transformer.py"
click node_swinv2 "https://github.com/microsoft/swin-transformer/blob/main/models/swin_transformer_v2.py"
click node_swin_mlp "https://github.com/microsoft/swin-transformer/blob/main/models/swin_mlp.py"
click node_simmim_pretrain "https://github.com/microsoft/swin-transformer/blob/main/main_simmim_pt.py"
click node_simmim_finetune "https://github.com/microsoft/swin-transformer/blob/main/main_simmim_ft.py"
click node_simmim_model "https://github.com/microsoft/swin-transformer/blob/main/models/simmim.py"
click node_simmim_data "https://github.com/microsoft/swin-transformer/blob/main/data/data_simmim_pt.py"
click node_moe_runner "https://github.com/microsoft/swin-transformer/blob/main/main_moe.py"
click node_moe_model "https://github.com/microsoft/swin-transformer/blob/main/models/swin_transformer_moe.py"
click node_moe_runtime "https://github.com/microsoft/swin-transformer/blob/main/utils_moe.py"
click node_runtime_utils "https://github.com/microsoft/swin-transformer/blob/main/utils.py"
click node_logger "https://github.com/microsoft/swin-transformer/blob/main/logger.py"
click node_window_extension "https://github.com/microsoft/swin-transformer/blob/main/kernels/window_process/window_process.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_configs,node_config_merge,node_supervised_runner,node_optimizer,node_scheduler,node_simmim_pretrain,node_simmim_finetune,node_moe_runner toneBlue
class node_data_build,node_cached_images,node_imagenet22k,node_simmim_data toneAmber
class node_model_factory,node_swin_backbones,node_swinv2,node_swin_mlp,node_simmim_model,node_moe_model toneMint
class node_moe_runtime,node_runtime_utils,node_logger,node_window_extension toneRose
```

## Components

### Experiment control

- [YAML presets](https://github.com/microsoft/swin-transformer/blob/main/configs/swin/swin_tiny_patch4_window7_224.yaml): experiment configs · Select task, scale, resolution, and training settings. · `configs/swin/swin_tiny_patch4_window7_224.yaml`
- [Config merge](https://github.com/microsoft/swin-transformer/blob/main/config.py): configuration · Merges presets with command-line overrides. · `config.py`
- [Supervised runner](https://github.com/microsoft/swin-transformer/blob/main/main.py): training entry point · Runs dense-model training, validation, and resume flows. · `main.py`
- [Optimizer builder](https://github.com/microsoft/swin-transformer/blob/main/optimizer.py): optimization · `optimizer.py`
- [LR scheduler](https://github.com/microsoft/swin-transformer/blob/main/lr_scheduler.py): optimization · `lr_scheduler.py`
- [SimMIM pretraining](https://github.com/microsoft/swin-transformer/blob/main/main_simmim_pt.py): training entry point · `main_simmim_pt.py`
- [SimMIM fine-tuning](https://github.com/microsoft/swin-transformer/blob/main/main_simmim_ft.py): training entry point · `main_simmim_ft.py`
- [Swin-MoE runner](https://github.com/microsoft/swin-transformer/blob/main/main_moe.py): distributed entry point · Launches expert-routed multi-GPU training. · `main_moe.py`

### Data ingestion

- [Dataset and loaders](https://github.com/microsoft/swin-transformer/blob/main/data/build.py): data builder · Builds ImageNet loaders and distributed samplers. · `data/build.py`
- [Cached/zip images](https://github.com/microsoft/swin-transformer/blob/main/data/cached_image_folder.py): dataset backend · `data/cached_image_folder.py`
- [ImageNet-22K labels](https://github.com/microsoft/swin-transformer/blob/main/data/imagenet22k_dataset.py): dataset backend · `data/imagenet22k_dataset.py`
- [SimMIM data path](https://github.com/microsoft/swin-transformer/blob/main/data/data_simmim_pt.py): pretraining data · `data/data_simmim_pt.py`

### Model variants

- [Model factory](https://github.com/microsoft/swin-transformer/blob/main/models/build.py): model builder · Dispatches configured dense backbone variants. · `models/build.py`
- [Swin dense backbones](https://github.com/microsoft/swin-transformer/blob/main/models/swin_transformer.py): vision backbone · Hierarchical patch, shifted-window, merging, and classifier stages. · `models/swin_transformer.py`
- [Swin V2 backbone](https://github.com/microsoft/swin-transformer/blob/main/models/swin_transformer_v2.py): vision backbone · `models/swin_transformer_v2.py`
- [Swin-MLP backbone](https://github.com/microsoft/swin-transformer/blob/main/models/swin_mlp.py): vision backbone · `models/swin_mlp.py`
- [SimMIM wrapper](https://github.com/microsoft/swin-transformer/blob/main/models/simmim.py): masked reconstruction model · Wraps a Swin encoder for masked-image reconstruction. · `models/simmim.py`
- [Swin-MoE model](https://github.com/microsoft/swin-transformer/blob/main/models/swin_transformer_moe.py): expert-routed backbone · Uses external Tutel MoE routing. · `models/swin_transformer_moe.py`

### Distributed runtime

- [MoE runtime utilities](https://github.com/microsoft/swin-transformer/blob/main/utils_moe.py): distributed utilities · `utils_moe.py`
- [Runtime utilities](https://github.com/microsoft/swin-transformer/blob/main/utils.py): distributed runtime · Provides distributed setup, checkpoints, metric reduction, and logging. · `utils.py`
- [Artifact logger](https://github.com/microsoft/swin-transformer/blob/main/logger.py): logging · `logger.py`
- [CUDA window extension](https://github.com/microsoft/swin-transformer/blob/main/kernels/window_process/window_process.py): CUDA/C++ acceleration · Accelerates window partition and reversal. · `kernels/window_process/window_process.py`

## Connections

- YAML presets → Config merge: preset
- Config merge → Supervised runner: runtime config
- Supervised runner → Dataset and loaders: builds loaders
- Dataset and loaders → Cached/zip images: optional backend
- Dataset and loaders → ImageNet-22K labels: 22K support
- Supervised runner → Model factory: builds model
- Model factory → Swin dense backbones: Swin V1
- Model factory → Swin V2 backbone: Swin V2
- Model factory → Swin-MLP backbone: Swin-MLP
- Supervised runner → Optimizer builder: builds
- Supervised runner → LR scheduler: updates
- Supervised runner → Runtime utilities: distributed and checkpoints
- Runtime utilities → Artifact logger: metrics and artifacts
- Swin dense backbones → CUDA window extension: accelerated windows
- SimMIM pretraining → SimMIM data path: masked inputs
- SimMIM pretraining → SimMIM wrapper: trains
- SimMIM wrapper → Swin V2 backbone: encoder option
- SimMIM fine-tuning → SimMIM wrapper: loads representations
- Swin-MoE runner → Swin-MoE model: trains
- Swin-MoE runner → MoE runtime utilities: distributed execution
- YAML presets → SimMIM pretraining: SimMIM preset
- YAML presets → Swin-MoE runner: MoE preset
