# compvis/stable-diffusion architecture

> Architecture diagram of the GitHub repository compvis/stable-diffusion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/compvis/stable-diffusion
- Repository: https://github.com/compvis/stable-diffusion

## Overview

Stable Diffusion is a latent text-to-image diffusion model, with reference workflows for text-to-image, image-to-image, and inpainting. The main sampling path takes a prompt through text conditioning, latent denoising, and autoencoder decoding to rendered images; training and dataset preparation form a separate branch. The README documents safety checking and invisible watermarking in the reference sampling script. Some implementation details were not sampled, so those documented integrations are shown separately; other edges are limited to explicit source references and documented behavior.

Main entry points: [`scripts/txt2img.py`](https://github.com/compvis/stable-diffusion/blob/main/scripts/txt2img.py) (Text sampling), [`ldm/models/diffusion/ddpm.py`](https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/ddpm.py) (Latent diffusion), [`ldm/models/diffusion/ddim.py`](https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/ddim.py) (DDIM sampler), [`main.py`](https://github.com/compvis/stable-diffusion/blob/main/main.py) (Training orchestration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_workflows["User workflows"]
  node_txt2img["Text sampling<br/>[txt2img.py]"]
  node_img2img["Image-to-image<br/>[img2img.py]"]
  node_inpaint["Inpainting<br/>[inpaint.py]"]
  node_knn["KNN image search<br/>[knn2img.py]"]
  node_safety["Safety checker"]
  node_watermark["Invisible watermark"]
  node_images["Rendered images<br/>[txt2img.py]"]
end

subgraph group_model["Latent diffusion model"]
  node_latent["Latent diffusion<br/>[ddpm.py]"]
  node_text["Text encoder<br/>[modules.py]"]
  node_unet["Diffusion UNet<br/>[openaimodel.py]"]
  node_attention["Attention layers<br/>[attention.py]"]
  node_transformer["Transformer layers<br/>[x_transformer.py]"]
  node_autoencoder["Latent autoencoder<br/>[autoencoder.py]"]
  node_gaussian["Gaussian latents<br/>[distributions.py]"]
end

subgraph group_sampling["Sampling methods"]
  node_ddim["DDIM sampler<br/>[ddim.py]"]
  node_plms["PLMS sampler<br/>[plms.py]"]
  node_dpm["DPM solver<br/>[dpm_solver.py]"]
end

subgraph group_training["Training and data"]
  node_train["Training orchestration<br/>[main.py]"]
  node_datasets["Image datasets"]
  node_degradation["Image degradation<br/>[bsrgan.py]"]
  node_ema["EMA weights<br/>[ema.py]"]
end

node_user(("Prompt user"))

node_user -->|"provides prompt"| node_txt2img
node_txt2img -->|"encodes text"| node_text
node_txt2img -->|"selects sampler"| node_ddim
node_txt2img -->|"selects sampler"| node_plms
node_txt2img -->|"selects sampler"| node_dpm
node_ddim -->|"denoises with"| node_latent
node_plms -.->|"denoises with"| node_latent
node_latent -->|"runs denoiser"| node_unet
node_latent -->|"uses first stage"| node_autoencoder
node_autoencoder -->|"models latent distribution"| node_gaussian
node_text -->|"uses transformer"| node_transformer
node_unet -->|"uses attention"| node_attention
node_txt2img -.->|"decodes samples"| node_autoencoder
node_txt2img -.->|"checks outputs"| node_safety
node_txt2img -.->|"watermarks outputs"| node_watermark
node_txt2img -->|"writes images"| node_images
node_user -->|"supplies image"| node_img2img
node_img2img -->|"samples with"| node_ddim
node_user -->|"supplies mask"| node_inpaint
node_inpaint -->|"samples with"| node_ddim
node_knn -->|"uses encoder"| node_text
node_knn -->|"samples with"| node_plms
node_train -->|"loads datasets"| node_datasets
node_train -.->|"trains model"| node_latent
node_train -.->|"tracks weights"| node_ema
node_datasets -.->|"uses degradation"| node_degradation

click node_txt2img "https://github.com/compvis/stable-diffusion/blob/main/scripts/txt2img.py"
click node_img2img "https://github.com/compvis/stable-diffusion/blob/main/scripts/img2img.py"
click node_inpaint "https://github.com/compvis/stable-diffusion/blob/main/scripts/inpaint.py"
click node_knn "https://github.com/compvis/stable-diffusion/blob/main/scripts/knn2img.py"
click node_images "https://github.com/compvis/stable-diffusion/blob/main/scripts/txt2img.py"
click node_latent "https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/ddpm.py"
click node_text "https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/encoders/modules.py"
click node_unet "https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/diffusionmodules/openaimodel.py"
click node_attention "https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/attention.py"
click node_transformer "https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/x_transformer.py"
click node_autoencoder "https://github.com/compvis/stable-diffusion/blob/main/ldm/models/autoencoder.py"
click node_gaussian "https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/distributions/distributions.py"
click node_ddim "https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/ddim.py"
click node_plms "https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/plms.py"
click node_dpm "https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/dpm_solver/dpm_solver.py"
click node_train "https://github.com/compvis/stable-diffusion/blob/main/main.py"
click node_datasets "https://github.com/compvis/stable-diffusion/tree/main/ldm/data"
click node_degradation "https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/image_degradation/bsrgan.py"
click node_ema "https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/ema.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_txt2img,node_img2img,node_inpaint,node_knn,node_safety,node_watermark,node_images,node_user toneBlue
class node_latent,node_text,node_unet,node_attention,node_transformer,node_autoencoder,node_gaussian toneAmber
class node_ddim,node_plms,node_dpm toneMint
class node_train,node_datasets,node_degradation,node_ema toneRose
```

## Components

### User workflows

- [Text sampling](https://github.com/compvis/stable-diffusion/blob/main/scripts/txt2img.py): `scripts/txt2img.py`
- [Image-to-image](https://github.com/compvis/stable-diffusion/blob/main/scripts/img2img.py): `scripts/img2img.py`
- [Inpainting](https://github.com/compvis/stable-diffusion/blob/main/scripts/inpaint.py): `scripts/inpaint.py`
- [KNN image search](https://github.com/compvis/stable-diffusion/blob/main/scripts/knn2img.py): `scripts/knn2img.py`
- **Safety checker**
- **Invisible watermark**
- [Rendered images](https://github.com/compvis/stable-diffusion/blob/main/scripts/txt2img.py): `scripts/txt2img.py`

### Latent diffusion model

- [Latent diffusion](https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/ddpm.py): `ldm/models/diffusion/ddpm.py`
- [Text encoder](https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/encoders/modules.py): `ldm/modules/encoders/modules.py`
- [Diffusion UNet](https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/diffusionmodules/openaimodel.py): `ldm/modules/diffusionmodules/openaimodel.py`
- [Attention layers](https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/attention.py): `ldm/modules/attention.py`
- [Transformer layers](https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/x_transformer.py): `ldm/modules/x_transformer.py`
- [Latent autoencoder](https://github.com/compvis/stable-diffusion/blob/main/ldm/models/autoencoder.py): `ldm/models/autoencoder.py`
- [Gaussian latents](https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/distributions/distributions.py): `ldm/modules/distributions/distributions.py`

### Sampling methods

- [DDIM sampler](https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/ddim.py): `ldm/models/diffusion/ddim.py`
- [PLMS sampler](https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/plms.py): `ldm/models/diffusion/plms.py`
- [DPM solver](https://github.com/compvis/stable-diffusion/blob/main/ldm/models/diffusion/dpm_solver/dpm_solver.py): `ldm/models/diffusion/dpm_solver/dpm_solver.py`

### Training and data

- [Training orchestration](https://github.com/compvis/stable-diffusion/blob/main/main.py): `main.py`
- [Image datasets](https://github.com/compvis/stable-diffusion/tree/main/ldm/data): `ldm/data`
- [Image degradation](https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/image_degradation/bsrgan.py): `ldm/modules/image_degradation/bsrgan.py`
- [EMA weights](https://github.com/compvis/stable-diffusion/blob/main/ldm/modules/ema.py): `ldm/modules/ema.py`

### Other components

- **Prompt user**

## Connections

- Prompt user → Text sampling: provides prompt (evidence: `README.md`)
- Text sampling → Text encoder: encodes text (evidence: `scripts/txt2img.py`)
- Text sampling → DDIM sampler: selects sampler (evidence: `scripts/txt2img.py`)
- Text sampling → PLMS sampler: selects sampler (evidence: `scripts/txt2img.py`)
- Text sampling → DPM solver: selects sampler (evidence: `scripts/txt2img.py`)
- DDIM sampler → Latent diffusion: denoises with (evidence: `ldm/models/diffusion/ddim.py`)
- PLMS sampler → Latent diffusion: denoises with
- Latent diffusion → Diffusion UNet: runs denoiser (evidence: `ldm/models/diffusion/ddpm.py`)
- Latent diffusion → Latent autoencoder: uses first stage (evidence: `ldm/models/diffusion/ddpm.py`)
- Latent autoencoder → Gaussian latents: models latent distribution (evidence: `ldm/models/autoencoder.py`)
- Text encoder → Transformer layers: uses transformer (evidence: `ldm/modules/encoders/modules.py`)
- Diffusion UNet → Attention layers: uses attention (evidence: `ldm/modules/diffusionmodules/openaimodel.py`)
- Text sampling → Latent autoencoder: decodes samples
- Text sampling → Safety checker: checks outputs (evidence: `README.md`)
- Text sampling → Invisible watermark: watermarks outputs (evidence: `README.md`)
- Text sampling → Rendered images: writes images (evidence: `README.md`)
- Prompt user → Image-to-image: supplies image (evidence: `scripts/img2img.py`)
- Image-to-image → DDIM sampler: samples with (evidence: `scripts/img2img.py`)
- Prompt user → Inpainting: supplies mask (evidence: `scripts/inpaint.py`)
- Inpainting → DDIM sampler: samples with (evidence: `scripts/inpaint.py`)
- KNN image search → Text encoder: uses encoder (evidence: `scripts/knn2img.py`)
- KNN image search → PLMS sampler: samples with (evidence: `scripts/knn2img.py`)
- Training orchestration → Image datasets: loads datasets (evidence: `main.py`)
- Training orchestration → Latent diffusion: trains model
- Training orchestration → EMA weights: tracks weights
- Image datasets → Image degradation: uses degradation
