# xhkzdepartedream/flow-matching architecture

> Architecture diagram of the GitHub repository xhkzdepartedream/flow-matching, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xhkzdepartedream/flow-matching
- Repository: https://github.com/xhkzdepartedream/flow-matching

## Overview

##### Project Type and Purpose
- This is a research‐grade, full‐pipeline machine learning framework for continuous‐time diffusion models (“Flow‐Matching”) built in PyTorch.
- It ingests high-res images (CelebA-HQ), compresses them via a VAE, and trains a lightweight DiT (Diffusion Transformer) in the latent space using flow‐matching schedulers.

##### High-Level Components
a. Configuration Manager
- Directory: configs/*.yaml
- Loads hyperparameters (model size, data paths, scheduler settings, multi-GPU flags).

b. Data Ingestion & Preprocessing
- Directory: data_processing/
- Scripts fetch/download raw datasets (CelebA-HQ, FFHQ256), build training datasets, extract VGG16 features if needed, and implement dataset utilities (e.g., tree.py).

c. VAE Compression Module
- Files: modules/autoencoderkl.py, modules/vae_modules.py
- Implements AutoencoderKL to compress 256×256 images → 32×32×4 latent codes.
- Used in a two‐stage training: pretrain/fine-tune VAE then freeze encoder for diffusion.

d. Core Diffusion Model
- Directory: models/
- DiT.py: Transformer‐based diffusion architecture operating in latent space.
- Unet.py: alternate U-Net code path if selected via config.

e. Flow-Matching Scheduler & Sampler
- Directory: fm_scheduler/
- OTScheduler & Sampler orchestrate the continuous‐time flow, sampling timesteps, computing vector fields.

f. Training Orchestration
- Directory: trainer/
- Trainer.py: generic loop over batches, optimizer steps, logging.
- AutoencoderKL_trainer.py: specialized VAE training.

g. Execution Scripts
- Directory: scripts/
- scripts/train/train.py & train_finetune_autoencoderkl.py kick off multi-GPU training with torchrun.
- batch_sample.py + variants: use trained weights to generate images.
- compute_fid.py, visualize_latent_space.py, check_recon_effect.py: evaluation and visualization.

h. Utilities & Outputs
- utils.py: helper functions (logging, model I/O, LMDB saving).
- visuals_celebahq/: sample outputs and plots.

i. External Resources
- Multi-GPU cluster (torchrun), external storage for model weights (Google Drive).

##### Relationships & Data Flows
1. Config Manager → Trainer & Scripts
- YAML → Python dict → model, scheduler, data pipeline initialization.
2. Data Ingestion → Dataset → Trainer
- Raw images → data_processing → torch Dataset → DataLoader in Trainer.
3. VAE Module
- Trainer pre‐trains AutoencoderKL → saves encoder/decoder weights.
4. Diffusion Training
- Frozen encoder produces latents → DiT + Flow-Matching Scheduler → Trainer updates DiT parameters.
5. Checkpoints & Logs
- Trainer writes model weights, training metrics (FID, loss) to disk.
6. Sampling & Evaluation
- batch_sample scripts load DiT + Scheduler → generate latents → decode via VAE decoder → output images.
- compute_fid script compares outputs to ground truth distribution.

##### Architectural Patterns & Principles
- Modular Design: clear separation of data, models, schedulers, training loops, and scripts.
- Config‐Driven: all hyperparameters, paths, model choices live in YAML.
- Two‐Stage Pipeline: VAE compression stage + diffusion stage.
- Multi-GPU Friendly: use torchrun, distributed DataParallel/Broadcast.

## Diagram (Mermaid)

```mermaid
flowchart TB
    subgraph "Configuration Manager"
        CM["configs/*.yaml"]:::conf
    end

    subgraph "Data Ingestion & Preprocessing"
        DP["data_processing/"]:::data
    end

    subgraph "VAE Compression Module"
        AE_KL["autoencoderkl.py"]:::model
        VAE_MOD["vae_modules.py"]:::model
    end

    subgraph "Core Diffusion Model"
        DiT["DiT.py"]:::model
        Unet["Unet.py"]:::model
    end

    subgraph "Flow-Matching Scheduler & Sampler"
        FB["FlowsBase.py"]:::sched
        OT["OTScheduler.py"]:::sched
        SM["Sampler.py"]:::sched
    end

    subgraph "Training Orchestration"
        TR["Trainer.py"]:::train
        AE_TR["AutoencoderKL_trainer.py"]:::train
    end

    subgraph "Execution Scripts"
        TS1["train.py"]:::script
        TS2["train_finetune_autoencoderkl.py"]:::script
        SS1["batch_sample.py"]:::script
        SS2["batch_sample_single.py"]:::script
        SS3["sample_celebahq.py"]:::script
        EV1["compute_fid.py"]:::script
        EV2["check_recon_effect.py"]:::script
        EV3["visualize_latent_space.py"]:::script
        EV4["save_latent_to_lmdb_from_pretrained.py"]:::script
    end

    subgraph "Utilities & Outputs"
        UT["utils.py"]:::util
        VO["visuals_celebahq/"]:::output
    end

    subgraph "External Resources"
        Raw["Raw Image Data"]:::external
        GPU["Multi-GPU Cluster"]:::external
        S3["External Storage"]:::external
    end

    Raw --> DP
    CM --> DP
    CM --> AE_KL
    CM --> VAE_MOD
    CM --> DiT
    CM --> Unet
    CM --> FB
    CM --> OT
    CM --> SM
    CM --> TR

    DP --> TR
    AE_KL --> AE_TR
    VAE_MOD --> AE_TR
    AE_TR --> TR

    TR --> DiT
    TR --> FB
    TR --> OT
    TR --> SM

    DiT --> TR
    FB --> SM

    TR --> S3
    TR --> UT
    TR -.-> GPU

    TS1 --> TR
    TS2 --> AE_TR

    SS1 --> SM
    SS2 --> SM
    SS3 --> SM

    SM --> VAE_MOD
    VAE_MOD --> VO

    EV1 --> VO
    EV1 --> Raw
    EV2 --> VO
    EV3 --> VO
    EV4 --> UT

    classDef conf fill:#ccffcc,stroke:#333;
    classDef data fill:#cce5ff,stroke:#333;
    classDef model fill:#ffd9b3,stroke:#333;
    classDef sched fill:#e0b3ff,stroke:#333;
    classDef train fill:#fff2b3,stroke:#333;
    classDef script fill:#e0e0e0,stroke:#333;
    classDef util fill:#d9d9d9,stroke:#333;
    classDef output fill:#f2f2f2,stroke:#333;
    classDef external fill:#ffcccc,stroke:#333,stroke-dasharray:5 5;

    click CM "https://github.com/xhkzdepartedream/flow-matching/tree/main/configs/"
    click DP "https://github.com/xhkzdepartedream/flow-matching/tree/main/data_processing/"
    click AE_KL "https://github.com/xhkzdepartedream/flow-matching/blob/main/modules/autoencoderkl.py"
    click VAE_MOD "https://github.com/xhkzdepartedream/flow-matching/blob/main/modules/vae_modules.py"
    click DiT "https://github.com/xhkzdepartedream/flow-matching/blob/main/models/DiT.py"
    click Unet "https://github.com/xhkzdepartedream/flow-matching/blob/main/models/Unet.py"
    click FB "https://github.com/xhkzdepartedream/flow-matching/blob/main/fm_scheduler/FlowsBase.py"
    click OT "https://github.com/xhkzdepartedream/flow-matching/blob/main/fm_scheduler/OTScheduler.py"
    click SM "https://github.com/xhkzdepartedream/flow-matching/blob/main/fm_scheduler/Sampler.py"
    click TR "https://github.com/xhkzdepartedream/flow-matching/blob/main/trainer/Trainer.py"
    click AE_TR "https://github.com/xhkzdepartedream/flow-matching/blob/main/trainer/AutoencoderKL_trainer.py"
    click TS1 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/train/train.py"
    click TS2 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/train/train_finetune_autoencoderkl.py"
    click SS1 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/batch_sample.py"
    click SS2 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/batch_sample_single.py"
    click SS3 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/sample_celebahq.py"
    click EV1 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/compute_fid.py"
    click EV2 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/check_recon_effect.py"
    click EV3 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/visualize_latent_space.py"
    click EV4 "https://github.com/xhkzdepartedream/flow-matching/blob/main/scripts/save_latent_to_lmdb_from_pretrained.py"
    click UT "https://github.com/xhkzdepartedream/flow-matching/blob/main/utils.py"
    click VO "https://github.com/xhkzdepartedream/flow-matching/tree/main/visuals_celebahq/"
```
