How gnobitab/rectifiedflow works
1. Repository purpose
This repo is the official implementation of Rectified Flow for image generation and reflow-based distillation. The architecture is centered on learning an ODE-based velocity field that transports noise to data, then iteratively “reflowing” trajectories to make sampling progressively straighter and faster.
2. Top-level execution boundary
The main entry point is ImageGeneration/main.py. It acts as a mode dispatcher for three workflows: training, evaluation/sampling, and reflow. Most behavior is driven by config files under ImageGeneration/configs, so the architecture is primarily configuration-driven rather than hardcoded.
3. Configuration layer
The configs/ tree defines dataset- and experiment-specific setups:
- CIFAR-10 defaults and rectified-flow variants
- high-resolution setups for LSUN Bedroom/Church, CelebA-HQ, and AFHQ-Cat
- specialized reflow configs for data generation, online reflow, and distillation
This layer determines model choice, training schedule, sampling method, checkpoint paths, and reflow behavior.
4. Training / evaluation orchestration
The run_lib.py, run_lib_pytorch.py, and run_lib_reflow.py files are the main orchestration boundary. They connect configs to concrete workflows:
- dataset construction
- model instantiation
- optimizer/training loop
- checkpointing
- sampling/evaluation
- reflow pair generation and teacher-student style distillation
The separation suggests standard training/eval logic is shared, while reflow gets its own orchestration path.
5. Data subsystem
Data handling is split between datasets.py and pytorch_datasets.py. This repo supports both the generic dataset pipeline used in the score-based framework and PyTorch-specific dataset loading. The data boundary feeds either:
- standard supervised-like image batches for training
- generated trajectory pairs (Z0, Z1) for reflow/distillation
Reflow data can be generated offline into assets/reflow_data/... or online during training.
6. Model subsystem
The models/ directory contains the neural velocity/score architectures:
- ddpm.py for DDPM++-style models
- ncsnpp.py / ncsnv2.py for score-based U-Net variants
- layers.py, layerspp.py, normalization.py, up_or_down_sampling.py, and utils.py for reusable building blocks
- ema.py for exponential moving average parameter tracking
Architecturally, these models are the learnable core that predicts the vector field used by the flow/ODE samplers.
7. Flow and SDE abstraction
sde_lib.py provides the stochastic/ODE process abstraction used by the broader score-based framework. In this project, it supports the rectified-flow formulation by defining how the model is interpreted during training and sampling. This is the conceptual boundary between the neural network and the continuous-time generative process.
8. Losses, likelihood, and sampling
- losses.py defines the training objective for the flow/score model.
- sampling.py contains ODE-based generation routines, including Euler-style sampling used for fast one-step or few-step generation.
- likelihood.py supports probability/likelihood-related evaluation paths.
- evaluation.py covers sample generation and metric computation, including FID/IS-style evaluation.
These modules form the inference side of the system: once a checkpoint exists, they determine how images are produced and scored.
9. Reflow pipeline
Reflow is a first-class workflow in this repo, not just a training trick. The architecture supports:
- generating paired trajectories from a teacher model
- saving those pairs to disk for offline training
- using the generated pairs to train a stronger student flow
- optional online generation to reduce storage cost
This creates a multi-stage teacher → data-pair generation → student refinement loop.
10. Custom CUDA / PyTorch ops
The op/ directory contains fused low-level operations (fused_act, fused_bias_act, upfirdn2d) implemented in C++/CUDA with Python bindings. These are performance-critical primitives used by the model backbones, especially for high-resolution image generation. This is the main systems-level boundary in the repo.
11. Runtime and ecosystem
The project is primarily PyTorch-based, but the README also references TensorFlow, JAX, and TensorFlow Probability because the broader score-based ecosystem and evaluation stack historically depended on them. For architecture purposes, the runtime center of gravity here is still PyTorch plus CUDA extensions for speed.
12. Artifacts and external dependencies
Important external inputs/outputs include:
- pretrained checkpoints from Google Drive
- dataset statistics files such as assets/stats/cifar10_stats.npz for FID evaluation
- generated reflow datasets stored under assets/reflow_data/...
- training/eval outputs stored under logs/...
These artifacts are part of the operational architecture because training, reflow, and evaluation all depend on them.
13. Repository structure in one line
Overall, the architecture is: config-driven experiment setup → dataset loading → model + SDE/ODE flow definition → training or reflow orchestration → sampling/evaluation → optional offline/online reflow distillation, with custom CUDA ops accelerating the model backbone.
Main entry points: run_lib.py run_lib.py · run_lib_reflow.py run_lib_reflow.py · layers.py layers*.py · main.py main.py
Components 21
Execution
Top-level dispatch and workflow entrypoints
- main.py · Dispatches training, evaluation, sampling, and reflow workflows from config.
ImageGeneration/main.py
Configs
Experiment and dataset configuration layer
- CIFAR configs · Default CIFAR-10 experiment settings reused by rectified-flow variants.
ImageGeneration/configs/default_cifar10_configs.py - LSUN configs · Default LSUN experiment settings reused by higher-resolution variants.
ImageGeneration/configs/default_lsun_configs.py - RF configs · Dataset-specific rectified-flow and reflow experiment definitions.
ImageGeneration/configs/rectified_flow
Core flow
Training, sampling, loss, and process abstractions
- run_lib.py · Shared training and evaluation orchestration for the score-based pipeline.
ImageGeneration/run_lib.py - run_lib_pytorch.py · PyTorch-specific training and evaluation orchestration layer.
ImageGeneration/run_lib_pytorch.py - run_lib_reflow.py · Coordinates teacher-generated pairs, offline/online reflow, and student refinement.
ImageGeneration/run_lib_reflow.py - losses.py · Defines the rectified-flow and score-model loss functions.
ImageGeneration/losses.py - sampling.py · Implements ODE/Euler-style sampling routines for generation.
ImageGeneration/sampling.py - evaluation.py · Runs sample generation and evaluation metrics such as FID and IS.
ImageGeneration/evaluation.py - likelihood.py · Supports probability and likelihood-related evaluation paths.
ImageGeneration/likelihood.py - sde_lib.py · Continuous-time process abstraction for training and sampling interpretation.
ImageGeneration/sde_lib.py
Data
Dataset loading and reflow data handling
- datasets.py · Generic dataset pipeline used by the score-based framework.
ImageGeneration/datasets.py - pytorch_datasets.py · PyTorch-specific dataset loading and preprocessing paths.
ImageGeneration/pytorch_datasets.py
Models
Velocity/score network architectures and shared layers
- ddpm.py · DDPM++-style network used as a velocity/score backbone.
ImageGeneration/models/ddpm.py - ncsnpp.py · NCSN++ U-Net variant for score/velocity prediction.
ImageGeneration/models/ncsnpp.py - ncsnv2.py · NCSNv2 U-Net variant for score-based modeling.
ImageGeneration/models/ncsnv2.py - layers*.py · Reusable layers, normalization, sampling, and utility blocks.
ImageGeneration/models/layers.py - ema.py · Exponential moving average helper for model parameters.
ImageGeneration/models/ema.py
Systems
Performance-critical custom ops and runtime artifacts
- op/ · Fused custom C++/CUDA primitives for model performance.
ImageGeneration/op - assets · External artifacts such as stats files, generated reflow data, and checkpoints.
Connections 22
- main.py → RF configs: loads · Selects workflow and experiment settings from configuration modules.
- main.py → run_lib.py: dispatches · Routes execution into the shared orchestration layer.
- main.py → run_lib_reflow.py: dispatches · Routes reflow-specific workflows to the reflow orchestrator.
- run_lib.py → datasets.py: uses · Builds the standard dataset pipeline for training and evaluation.
- run_lib.py → pytorch_datasets.py: uses · Can bind to PyTorch-native dataset loading paths.
- run_lib.py → losses.py: trains with · Connects configs and models to the training objective.
- run_lib.py → sampling.py: samples with · Uses ODE sampling for generation and evaluation.
- run_lib.py → evaluation.py: evaluates with · Runs metric computation and sample evaluation.
- run_lib.py → likelihood.py: evaluates with · Supports likelihood-oriented evaluation workflows.
- run_lib.py → sde_lib.py: interprets via · Uses the continuous-time flow abstraction during training and sampling.
- run_lib_pytorch.py → ddpm.py: instantiates · Constructs the PyTorch model backbone for rectified flow.
- run_lib_pytorch.py → ncsnpp.py: instantiates · Alternative backbone for score/velocity prediction.
- run_lib_pytorch.py → ncsnv2.py: instantiates · Alternative backbone for score/velocity prediction.
- ddpm.py → layers*.py: builds from · Composes reusable neural network blocks.
- ncsnpp.py → layers*.py: builds from · Composes reusable neural network blocks.
- ncsnv2.py → layers*.py: builds from · Composes reusable neural network blocks.
- layers*.py → op/: accelerated by · Uses fused custom kernels for performance-critical operations.
- run_lib_reflow.py → assets: reads/writes · Consumes and produces reflow trajectory pairs and related artifacts.
- run_lib_reflow.py → sampling.py: generates pairs with · Uses the teacher sampler to create trajectory pairs for distillation.
- run_lib_reflow.py → losses.py: optimizes with · Trains student flows on generated or online reflow pairs.
- evaluation.py → assets: reads · Consumes dataset statistics and stored checkpoints for evaluation.
- sampling.py → assets: writes · Can emit generated samples or trajectory data for later use.