# openai/improved-diffusion architecture

> Architecture diagram of the GitHub repository openai/improved-diffusion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openai/improved-diffusion
- Repository: https://github.com/openai/improved-diffusion

## Overview

1. This repository is a research training/sampling package for diffusion models, centered on the `improved_diffusion` Python package and a small set of CLI scripts. It is not a web service; the main runtime is local or distributed Python jobs launched from the command line.

2. The top-level architecture splits into four parts: model/diffusion core (`improved_diffusion/`), dataset preparation helpers (`datasets/`), executable entrypoints (`scripts/`), and packaging metadata (`setup.py`). The README describes the intended flow: prepare image folders, train a model, then sample from saved checkpoints.

3. The core model subsystem is built around a U-Net implementation in `improved_diffusion/unet.py`. This is the neural network that predicts denoising outputs during training and generation, with configuration assembled through `script_util.py`.

4. The diffusion-process subsystem lives in `improved_diffusion/gaussian_diffusion.py`, with timestep/respacing logic in `respace.py` and `resample.py`. This layer defines the forward noising process, reverse sampling process, loss targets, and optional timestep selection strategies such as reduced-step sampling and DDIM-style spacing.

5. Training orchestration is handled by `improved_diffusion/train_util.py`. It connects the data loader, model, diffusion object, optimizer/training loop, checkpointing, and logging. The `losses.py` module holds the objective calculations used during optimization, including variants such as standard diffusion loss and VLB-related terms.

6. Data ingestion is intentionally simple and file-system based. `improved_diffusion/image_datasets.py` loads images from a directory tree, handles resizing/cropping, and optionally infers class labels from filename prefixes or subdirectories. The `datasets/` folder contains dataset-specific preparation scripts for CIFAR-10 and LSUN bedroom, plus README instructions for building these directories.

7. The executable scripts provide the user-facing boundaries of the system. `scripts/image_train.py` trains unconditional or class-conditional image models, `scripts/image_sample.py` loads a checkpoint and produces samples, `scripts/image_nll.py` evaluates likelihood-related metrics, and `scripts/super_res_train.py` / `scripts/super_res_sample.py` support conditional super-resolution workflows.

8. The super-resolution path is a distinct conditional pipeline rather than just another sampling mode. It reuses the same diffusion core and model machinery, but the inputs include a low-resolution conditioning image, so the model boundary differs from pure unconditional generation.

9. Distributed and memory-efficiency concerns are handled explicitly inside the package. `dist_util.py` supports MPI-style multi-GPU/multi-machine execution, while `fp16_util.py` provides mixed-precision helpers. The README’s `mpiexec` examples and `--microbatch` flag reflect this architecture: the training loop is designed to scale across ranks but still work on a single GPU.

10. Logging and artifact flow are file-based. `logger.py` writes metrics, checkpoints, and sample outputs to a run directory controlled by `OPENAI_LOGDIR` or a temporary directory. Training produces `.pt` checkpoints; sampling writes `.npz` batches, with the EMA checkpoint typically serving as the preferred sampling source.

11. `script_util.py` is the configuration boundary that converts CLI flags into model, diffusion, and training objects. This is important for diagramming because the scripts do not hardcode architectures; instead, they assemble experiments from a shared parameterization of image size, channel count, residual blocks, diffusion steps, noise schedule, and conditioning mode.

12. Overall data flow is: image files on disk → dataset loader/preprocessing → U-Net + Gaussian diffusion training loop → checkpoint files and logs → sampling/evaluation scripts that reload checkpoints and run the reverse diffusion process to emit images or likelihood outputs.

13. The repo is self-contained aside from standard Python dependencies and MPI support. There are no external APIs or databases; the only notable external interaction is the filesystem for datasets, checkpoints, logs, and generated sample files.

Main entry points: [`improved_diffusion/gaussian_diffusion.py`](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/gaussian_diffusion.py) (Diffusion process), [`improved_diffusion/train_util.py`](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/train_util.py) (Train loop), [`improved_diffusion/script_util.py`](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/script_util.py) (Config builder), [`improved_diffusion/logger.py`](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/logger.py) (Run logger)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_core["Diffusion core"]
  node_script_util["Config builder<br/>config assembly<br/>[script_util.py]"]
  node_unet{{"U-Net<br/>denoiser network<br/>[unet.py]"}}
  node_gaussian_diffusion["Diffusion process<br/>diffusion math"]
  node_respace["Timestep spacing<br/>sampling schedule<br/>[respace.py]"]
  node_resample["Timestep sampler<br/>timestep policy<br/>[resample.py]"]
  node_train_util["Train loop<br/>training orchestration<br/>[train_util.py]"]
  node_losses["Objectives<br/>loss functions<br/>[losses.py]"]
  node_dist_util["Distributed I/O<br/>mpi runtime<br/>[dist_util.py]"]
  node_fp16_util["FP16 helpers<br/>mixed precision<br/>[fp16_util.py]"]
  node_logger["Run logger<br/>artifact logging<br/>[logger.py]"]
  node_nn["NN utilities<br/>tensor utils<br/>[nn.py]"]
end

subgraph group_data["Data prep"]
  node_image_datasets["Image loader<br/>dataset loader<br/>[image_datasets.py]"]
  node_cifar10_prep["CIFAR-10 prep<br/>dataset script<br/>[cifar10.py]"]
  node_lsun_prep["LSUN prep<br/>dataset script<br/>[lsun_bedroom.py]"]
end

subgraph group_entry["Entry scripts"]
  node_image_train["Train images<br/>cli training<br/>[image_train.py]"]
  node_image_sample["Sample images<br/>cli sampling<br/>[image_sample.py]"]
  node_image_nll["Evaluate NLL<br/>cli evaluation<br/>[image_nll.py]"]
  node_super_res_train["Train SR<br/>cli training<br/>[super_res_train.py]"]
  node_super_res_sample["Sample SR<br/>cli sampling"]
end

node_setup["Package metadata<br/>packaging<br/>[setup.py]"]

node_image_train -->|"builds"| node_script_util
node_image_sample -->|"builds"| node_script_util
node_image_nll -->|"builds"| node_script_util
node_super_res_train -->|"builds"| node_script_util
node_super_res_sample -->|"builds"| node_script_util
node_script_util -->|"configures"| node_unet
node_script_util -->|"configures"| node_gaussian_diffusion
node_train_util -->|"loads data"| node_image_datasets
node_train_util -->|"trains"| node_unet
node_train_util -->|"optimizes"| node_gaussian_diffusion
node_train_util -->|"computes"| node_losses
node_train_util -->|"runs on"| node_dist_util
node_train_util -->|"uses"| node_fp16_util
node_train_util -->|"writes"| node_logger
node_gaussian_diffusion -->|"uses"| node_respace
node_gaussian_diffusion -->|"uses"| node_resample
node_image_sample -->|"samples"| node_gaussian_diffusion
node_image_nll -->|"evaluates"| node_gaussian_diffusion
node_super_res_sample -->|"samples"| node_gaussian_diffusion
node_super_res_train -->|"trains"| node_gaussian_diffusion
node_image_datasets -->|"expects"| node_cifar10_prep
node_image_datasets -->|"expects"| node_lsun_prep
node_cifar10_prep -.->|"packages with"| node_setup
node_lsun_prep -.->|"packages with"| node_setup
node_image_train -->|"writes"| node_logger
node_image_sample -->|"writes"| node_logger
node_super_res_sample -->|"writes"| node_logger
node_unet -->|"builds on"| node_nn
node_gaussian_diffusion -->|"builds on"| node_nn
node_setup -.->|"installs"| node_dist_util

click node_script_util "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/script_util.py"
click node_unet "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/unet.py"
click node_gaussian_diffusion "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/gaussian_diffusion.py"
click node_respace "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/respace.py"
click node_resample "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/resample.py"
click node_train_util "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/train_util.py"
click node_losses "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/losses.py"
click node_image_datasets "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/image_datasets.py"
click node_cifar10_prep "https://github.com/openai/improved-diffusion/blob/main/datasets/cifar10.py"
click node_lsun_prep "https://github.com/openai/improved-diffusion/blob/main/datasets/lsun_bedroom.py"
click node_image_train "https://github.com/openai/improved-diffusion/blob/main/scripts/image_train.py"
click node_image_sample "https://github.com/openai/improved-diffusion/blob/main/scripts/image_sample.py"
click node_image_nll "https://github.com/openai/improved-diffusion/blob/main/scripts/image_nll.py"
click node_super_res_train "https://github.com/openai/improved-diffusion/blob/main/scripts/super_res_train.py"
click node_super_res_sample "https://github.com/openai/improved-diffusion/blob/main/scripts/super_res_sample.py"
click node_dist_util "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/dist_util.py"
click node_fp16_util "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/fp16_util.py"
click node_logger "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/logger.py"
click node_nn "https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/nn.py"
click node_setup "https://github.com/openai/improved-diffusion/blob/main/setup.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_script_util,node_unet,node_gaussian_diffusion,node_respace,node_resample,node_train_util,node_losses,node_dist_util,node_fp16_util,node_logger,node_nn toneBlue
class node_image_datasets,node_cifar10_prep,node_lsun_prep toneAmber
class node_image_train,node_image_sample,node_image_nll,node_super_res_train,node_super_res_sample toneMint
class node_setup toneNeutral
```

## Components

### Diffusion core

Model, diffusion, training, and runtime utilities inside the Python package.

- [Config builder](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/script_util.py): config assembly · Converts CLI flags into model, diffusion, and training objects. · `improved_diffusion/script_util.py`
- [U-Net](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/unet.py): denoiser network · Neural network backbone used to predict denoising outputs. · `improved_diffusion/unet.py`
- [Diffusion process](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/gaussian_diffusion.py): diffusion math · Forward noising, reverse sampling, and diffusion loss targets. · `improved_diffusion/gaussian_diffusion.py`
- [Timestep spacing](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/respace.py): sampling schedule · Reduces or rearranges timesteps for faster or alternative sampling. · `improved_diffusion/respace.py`
- [Timestep sampler](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/resample.py): timestep policy · Chooses timestep selection strategies used during training or sampling. · `improved_diffusion/resample.py`
- [Train loop](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/train_util.py): training orchestration · Connects data, model, optimizer, checkpoints, and logging. · `improved_diffusion/train_util.py`
- [Objectives](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/losses.py): loss functions · Computes diffusion training objectives and VLB-related terms. · `improved_diffusion/losses.py`
- [Distributed I/O](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/dist_util.py): mpi runtime · MPI-style multi-GPU and multi-machine helpers. · `improved_diffusion/dist_util.py`
- [FP16 helpers](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/fp16_util.py): mixed precision · Mixed-precision training and model utilities. · `improved_diffusion/fp16_util.py`
- [Run logger](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/logger.py): artifact logging · Writes metrics, checkpoints, and sample outputs to the run directory. · `improved_diffusion/logger.py`
- [NN utilities](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/nn.py): tensor utils · Shared neural-network helper functions used by core modules. · `improved_diffusion/nn.py`

### Data prep

Filesystem-based dataset loading and dataset-specific preparation helpers.

- [Image loader](https://github.com/openai/improved-diffusion/blob/main/improved_diffusion/image_datasets.py): dataset loader · Loads images from directories, preprocesses them, and infers labels when available. · `improved_diffusion/image_datasets.py`
- [CIFAR-10 prep](https://github.com/openai/improved-diffusion/blob/main/datasets/cifar10.py): dataset script · Prepares CIFAR-10 data into the expected filesystem layout. · `datasets/cifar10.py`
- [LSUN prep](https://github.com/openai/improved-diffusion/blob/main/datasets/lsun_bedroom.py): dataset script · Prepares LSUN bedroom data into the expected filesystem layout. · `datasets/lsun_bedroom.py`

### Entry scripts

Command-line entrypoints for training, sampling, likelihood evaluation, and super-resolution.

- [Train images](https://github.com/openai/improved-diffusion/blob/main/scripts/image_train.py): cli training · Main entrypoint for unconditional and class-conditional training. · `scripts/image_train.py`
- [Sample images](https://github.com/openai/improved-diffusion/blob/main/scripts/image_sample.py): cli sampling · Loads a checkpoint and generates image samples. · `scripts/image_sample.py`
- [Evaluate NLL](https://github.com/openai/improved-diffusion/blob/main/scripts/image_nll.py): cli evaluation · Computes likelihood-related metrics from a trained checkpoint. · `scripts/image_nll.py`
- [Train SR](https://github.com/openai/improved-diffusion/blob/main/scripts/super_res_train.py): cli training · Trains the conditional super-resolution model path. · `scripts/super_res_train.py`
- [Sample SR](https://github.com/openai/improved-diffusion/blob/main/scripts/super_res_sample.py): cli sampling · Runs super-resolution sampling from a checkpoint and low-resolution input. · `scripts/super_res_sample.py`

### Other components

- [Package metadata](https://github.com/openai/improved-diffusion/blob/main/setup.py): packaging · Python packaging metadata for installing the repository. · `setup.py`

## Connections

- Train images → Config builder: builds · Assembles the experiment configuration from CLI flags.
- Sample images → Config builder: builds · Uses the shared config boundary to construct sampling components.
- Evaluate NLL → Config builder: builds · Constructs evaluation-time model and diffusion objects.
- Train SR → Config builder: builds · Creates the conditional super-resolution training setup.
- Sample SR → Config builder: builds · Creates the conditional super-resolution sampling setup.
- Config builder → U-Net: configures · Selects the U-Net architecture and its conditioning parameters.
- Config builder → Diffusion process: configures · Creates the diffusion process from schedule and loss settings.
- Train loop → Image loader: loads data · Consumes image batches from the filesystem dataset loader.
- Train loop → U-Net: trains · Optimizes the denoising network during training.
- Train loop → Diffusion process: optimizes · Uses diffusion objectives to compute training signals.
- Train loop → Objectives: computes · Evaluates diffusion and VLB-style objectives.
- Train loop → Distributed I/O: runs on · Supports distributed execution across ranks and devices.
- Train loop → FP16 helpers: uses · Applies mixed-precision helpers when enabled.
- Train loop → Run logger: writes · Persists checkpoints, metrics, and sample artifacts.
- Diffusion process → Timestep spacing: uses · Applies timestep spacing for reduced-step sampling.
- Diffusion process → Timestep sampler: uses · Selects timestep schedules and resampling strategies.
- Sample images → Diffusion process: samples · Runs the reverse diffusion process to generate images.
- Evaluate NLL → Diffusion process: evaluates · Uses diffusion math to compute likelihood-related outputs.
- Sample SR → Diffusion process: samples · Uses the reverse process with low-resolution conditioning.
- Train SR → Diffusion process: trains · Optimizes the conditional diffusion objective for super-resolution.
- Image loader → CIFAR-10 prep: expects · Consumes CIFAR-10 directories prepared by dataset scripts.
- Image loader → LSUN prep: expects · Consumes LSUN bedroom directories prepared by dataset scripts.
- CIFAR-10 prep → Package metadata: packages with · Dataset tooling lives alongside the installable Python package.
- LSUN prep → Package metadata: packages with · Dataset tooling lives alongside the installable Python package.
- Train images → Run logger: writes · Training entrypoint writes run artifacts through the logger.
- Sample images → Run logger: writes · Sampling entrypoint emits generated samples to the run directory.
- Sample SR → Run logger: writes · Super-resolution sampling writes outputs and metadata.
- U-Net → NN utilities: builds on · Uses shared neural-network helper functions.
- Diffusion process → NN utilities: builds on · Relies on shared tensor and module utilities.
- Package metadata → Distributed I/O: installs · Packaging exposes the distributed runtime helpers as part of the library.
