# x-gengroup/flow-factory architecture

> Architecture diagram of the GitHub repository x-gengroup/flow-factory, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/x-gengroup/flow-factory
- Repository: https://github.com/x-gengroup/flow-factory

## Overview

1. **What this repo is**
   Flow-Factory is an RL fine-tuning framework for diffusion and flow-matching generative models. Its architecture is centered on decoupling three axes: **model family**, **training algorithm**, and **reward source** so they can be swapped through config rather than hard-coded paths.

2. **Main execution path**
   Training starts from the CLI/entrypoint layer in `src/flow_factory/cli.py` and `src/flow_factory/train.py`, which loads structured hyperparameters from `src/flow_factory/hparams/`. Those args are then used to assemble the training stack: model, scheduler, data pipeline, reward pipeline, trainer, logging, and distributed runtime.

3. **Trainer layer = algorithm boundary**
   `src/flow_factory/trainers/` is the core algorithm boundary. It contains separate trainers for `dpo`, `grpo`, `nft`, `awm`, `dgpo`, `crd`, and algorithm-specific variants such as GRPO-Guard. This is where policy optimization logic lives, and it is intentionally separated from model definitions so the same algorithm can operate across multiple generative backbones.

4. **Model layer = architecture boundary**
   `src/flow_factory/models/` contains model adapters for each supported family: FLUX variants, Stable Diffusion 3.5, Qwen-Image / Edit, Wan video models, Z-Image, and LTX-2 audio-video. These wrappers adapt each backbone into the framework’s common training interface, hiding model-specific tokenization, conditioning, and forward/sampling details behind a registry/loader pattern.

5. **Sampling and scheduler subsystem**
   `src/flow_factory/scheduler/` provides sampling/update rules such as `flow_match_euler_discrete` and `unipc_multistep`. These are the diffusion/flow integration steps used during rollout generation. `src/flow_factory/samples/` holds the sample lifecycle objects, which connect model outputs, trajectories, and reward evaluation during training.

6. **Data ingestion pipeline**
   `src/flow_factory/data_utils/` handles dataset loading, batching, and sampling policy. The repository supports text-only, image-conditioned, multi-image, and video-conditioned datasets, using plain text or JSONL metadata under `dataset/`. The data layer is task-aware but still generic enough to feed the same trainer stack across image, video, and audio-video tasks.

7. **Reward subsystem**
   `src/flow_factory/rewards/` is another major boundary. It includes built-in pointwise and groupwise rewards such as PickScore, CLIP, OCR, `vllm_evaluate`, and Rational Rewards variants. Reward processing is separated into loader/registry/processor modules so rewards can be local models, heuristic scorers, or remote judges without changing the trainer.

8. **Remote reward and evaluation path**
   The repo also supports off-process reward evaluation through `reward_server/example_server.py` and the helper script in `scripts/`. This lets the trainer call an OpenAI-compatible or vLLM-hosted judge model remotely, which matters for large VLM-as-judge reward models and for async reward workflows.

9. **Advantage and reward-to-update transformation**
   `src/flow_factory/advantage/` contains the advantage processor used by algorithms that need group-based normalization, ranking, or reward shaping before the policy update. This layer is the bridge between raw reward outputs and algorithm-specific optimization targets.

10. **Training support utilities**
   `src/flow_factory/utils/` provides cross-cutting infrastructure: distributed helpers, checkpointing, EMA, noise scheduling, trajectory collection, memory tracking, and modality utilities for image/audio/video handling. These modules are not algorithms themselves; they support the training loop and keep the trainer/model code thinner.

11. **Logging and experiment tracking**
   `src/flow_factory/logger/` abstracts experiment tracking backends: Weights & Biases, SwanLab, and TensorBoard. Logging is treated as a pluggable subsystem, and the README shows it is used for both scalar metrics and sampled generations, which are first-class outputs of the training loop.

12. **Distributed/runtime infrastructure**
   The repository is designed for multi-GPU and multi-node training. `config/accelerate_configs/` and `config/deepspeed/` define launch-time sharding and memory strategies, while `multinode_examples/` contains cluster launch examples. This indicates the framework expects large-model RL runs and leans on Hugging Face Accelerate plus optional DeepSpeed/FSDP for scale.

13. **Configuration-driven composition**
   The `examples/` tree is organized by algorithm, finetuning mode, and model family, showing the main composition pattern: choose a trainer, choose a model adapter, choose a reward, and bind them with YAML. This is the clearest place to infer the system’s architecture because it demonstrates the repo’s primary promise: model–algorithm decoupling.

14. **External technology boundaries**
   The implementation is built on top of `diffusers` (bundled as a submodule for some models), with optional `deepspeed`, `wandb`, and `swanlab`. Those dependencies materially affect the architecture because they provide the underlying generative backbones, distributed training behavior, and experiment tracking endpoints rather than being incidental tooling.

15. **Data and artifact boundaries**
   The `dataset/` directory stores example corpora and metadata, while `assets/`, `docker/`, and `config/` support reproducibility and deployment. In an architecture diagram, these should be shown as external inputs/operational support around the central training pipeline, not as part of the training core itself.

Main entry points: [`src/flow_factory/train.py`](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/train.py) (Train), [`src/flow_factory/models`](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models) (Models), [`src/flow_factory/rewards`](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/rewards) (Rewards), [`src/flow_factory/trainers`](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/trainers) (Trainers)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_entry["Entry & config"]
  node_node_cli["CLI<br/>entrypoint<br/>[cli.py]"]
  node_node_train["Train<br/>bootstrap<br/>[train.py]"]
  node_node_hparams["HParams<br/>config schema"]
end

subgraph group_group_core["Core training stack"]
  node_node_trainers["Trainers<br/>algorithm layer"]
  node_node_scheduler["Scheduler<br/>sampling rules"]
  node_node_samples["Samples<br/>sample lifecycle<br/>[samples.py]"]
  node_node_data["Data<br/>ingestion pipeline"]
end

subgraph group_group_models["Model adapters"]
  node_node_models["Models<br/>backbone adapters"]
  node_node_flux["Flux<br/>model family"]
  node_node_sd3["SD 3.5<br/>model family<br/>[sd3_5.py]"]
  node_node_qwen["Qwen Image<br/>model family"]
  node_node_wan["Wan<br/>model family"]
  node_node_zimage["Z-Image<br/>model family<br/>[z_image.py]"]
  node_node_ltx2["LTX-2<br/>model family"]
end

subgraph group_group_rewards["Reward pipeline"]
  node_node_rewards["Rewards<br/>reward layer"]
  node_node_rewardproc["Reward Proc"]
  node_node_remote_reward["Remote Judge<br/>off-process reward<br/>[example_server.py]"]
  node_node_advantage["Advantage<br/>update transform"]
end

subgraph group_group_runtime["Runtime & support"]
  node_node_logger["Logger<br/>tracking layer"]
  node_node_utils["Utils<br/>support libs"]
  node_node_runtime["Runtime configs<br/>scale-out config"]
end

subgraph group_group_examples["Examples & ops"]
  node_node_examples["Examples<br/>composition configs"]
end

node_node_cli -->|"invokes"| node_node_train
node_node_train -->|"loads"| node_node_hparams
node_node_train -->|"builds"| node_node_trainers
node_node_train -->|"builds"| node_node_models
node_node_train -->|"builds"| node_node_scheduler
node_node_train -->|"builds"| node_node_data
node_node_train -->|"builds"| node_node_rewards
node_node_train -->|"builds"| node_node_logger
node_node_train -->|"uses"| node_node_utils
node_node_trainers -->|"uses"| node_node_advantage
node_node_trainers -->|"consumes"| node_node_samples
node_node_models -->|"includes"| node_node_flux
node_node_models -->|"includes"| node_node_sd3
node_node_models -->|"includes"| node_node_qwen
node_node_models -->|"includes"| node_node_wan
node_node_models -->|"includes"| node_node_zimage
node_node_models -->|"includes"| node_node_ltx2
node_node_scheduler -->|"drives"| node_node_samples
node_node_data -->|"feeds"| node_node_samples
node_node_rewards -->|"processed by"| node_node_rewardproc
node_node_rewards -.->|"can delegate"| node_node_remote_reward
node_node_rewardproc -->|"supplies"| node_node_advantage
node_node_examples -->|"configures"| node_node_trainers
node_node_examples -->|"configures"| node_node_models
node_node_examples -->|"configures"| node_node_rewards
node_node_runtime -->|"supports"| node_node_train
node_node_logger -->|"uses"| node_node_utils
node_node_remote_reward -.->|"extends"| node_node_rewards

click node_node_cli "https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/cli.py"
click node_node_train "https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/train.py"
click node_node_hparams "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/hparams"
click node_node_trainers "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/trainers"
click node_node_models "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models"
click node_node_flux "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/flux"
click node_node_sd3 "https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/models/stable_diffusion/sd3_5.py"
click node_node_qwen "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/qwen_image"
click node_node_wan "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/wan"
click node_node_zimage "https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/models/z_image/z_image.py"
click node_node_ltx2 "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/ltx2"
click node_node_scheduler "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/scheduler"
click node_node_samples "https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/samples/samples.py"
click node_node_data "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/data_utils"
click node_node_rewards "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/rewards"
click node_node_rewardproc "https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/rewards/reward_processor.py"
click node_node_remote_reward "https://github.com/x-gengroup/flow-factory/blob/main/reward_server/example_server.py"
click node_node_advantage "https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/advantage/advantage_processor.py"
click node_node_logger "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/logger"
click node_node_utils "https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/utils"
click node_node_examples "https://github.com/x-gengroup/flow-factory/tree/main/examples"
click node_node_runtime "https://github.com/x-gengroup/flow-factory/tree/main/config"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_cli,node_node_train,node_node_hparams toneBlue
class node_node_trainers,node_node_scheduler,node_node_samples,node_node_data toneAmber
class node_node_models,node_node_flux,node_node_sd3,node_node_qwen,node_node_wan,node_node_zimage,node_node_ltx2 toneMint
class node_node_rewards,node_node_rewardproc,node_node_remote_reward,node_node_advantage toneRose
class node_node_logger,node_node_utils,node_node_runtime toneIndigo
class node_node_examples toneTeal
```

## Components

### Entry & config

CLI, training bootstrap, and structured hyperparameters

- [CLI](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/cli.py): entrypoint · Command-line entry into training workflows · `src/flow_factory/cli.py`
- [Train](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/train.py): bootstrap · Main training assembly and orchestration · `src/flow_factory/train.py`
- [HParams](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/hparams): config schema · Structured argument and hyperparameter definitions · `src/flow_factory/hparams`

### Core training stack

Model, trainer, data, rollout, and reward subsystems

- [Trainers](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/trainers): algorithm layer · Algorithm boundary for DPO, GRPO, NFT, AWM, DGPO, CRD, and variants · `src/flow_factory/trainers`
- [Scheduler](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/scheduler): sampling rules · Flow/diffusion update rules used during rollout generation · `src/flow_factory/scheduler`
- [Samples](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/samples/samples.py): sample lifecycle · Trajectory and sample lifecycle objects · `src/flow_factory/samples/samples.py`
- [Data](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/data_utils): ingestion pipeline · Dataset loading, batching, and sampling policy · `src/flow_factory/data_utils`

### Model adapters

Backbone-specific wrappers behind a common interface

- [Models](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models): backbone adapters · Registry of model-family wrappers for training · `src/flow_factory/models`
- [Flux](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/flux): model family · Flux adapters for multiple variants · `src/flow_factory/models/flux`
- [SD 3.5](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/models/stable_diffusion/sd3_5.py): model family · Stable Diffusion 3.5 adapter · `src/flow_factory/models/stable_diffusion/sd3_5.py`
- [Qwen Image](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/qwen_image): model family · Qwen image and edit-plus adapters · `src/flow_factory/models/qwen_image`
- [Wan](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/wan): model family · Video model adapters for i2v, t2v, and v2v · `src/flow_factory/models/wan`
- [Z-Image](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/models/z_image/z_image.py): model family · Z-Image and turbo adapters · `src/flow_factory/models/z_image/z_image.py`
- [LTX-2](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/models/ltx2): model family · Audio-video adapters for i2av and t2av · `src/flow_factory/models/ltx2`

### Reward pipeline

Local, remote, and processed reward paths

- [Rewards](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/rewards): reward layer · Built-in reward models, judges, and loaders · `src/flow_factory/rewards`
- [Reward Proc](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/rewards/reward_processor.py): reward processing · Registry-backed reward processing and composition · `src/flow_factory/rewards/reward_processor.py`
- [Remote Judge](https://github.com/x-gengroup/flow-factory/blob/main/reward_server/example_server.py): off-process reward · Remote reward evaluation via server or vLLM-compatible judge · `reward_server/example_server.py`
- [Advantage](https://github.com/x-gengroup/flow-factory/blob/main/src/flow_factory/advantage/advantage_processor.py): update transform · Converts rewards into advantage or ranking signals · `src/flow_factory/advantage/advantage_processor.py`

### Runtime & support

Logging, utilities, and scale-out infrastructure

- [Logger](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/logger): tracking layer · Pluggable experiment logging backends · `src/flow_factory/logger`
- [Utils](https://github.com/x-gengroup/flow-factory/tree/main/src/flow_factory/utils): support libs · Distributed, checkpointing, EMA, trajectory, and modality helpers · `src/flow_factory/utils`
- [Runtime configs](https://github.com/x-gengroup/flow-factory/tree/main/config): scale-out config · Accelerate and DeepSpeed launch configurations · `config`

### Examples & ops

Composition examples, deployment, and external serving

- [Examples](https://github.com/x-gengroup/flow-factory/tree/main/examples): composition configs · Algorithm and model-specific YAML compositions · `examples`

## Connections

- CLI → Train: invokes · CLI launches the training entrypoint
- Train → HParams: loads · Training bootstrapping starts from structured configuration
- Train → Trainers: builds · Selects algorithm implementation from config
- Train → Models: builds · Chooses a backbone adapter through registry/loader
- Train → Scheduler: builds · Attaches rollout/update scheduler
- Train → Data: builds · Creates dataset and batching pipeline
- Train → Rewards: builds · Attaches reward source and processor
- Train → Logger: builds · Initializes experiment logging
- Train → Utils: uses · Relies on shared runtime helpers
- Trainers → Advantage: uses · Some algorithms transform rewards before updates
- Trainers → Samples: consumes · Algorithms operate on generated samples and trajectories
- Models → Flux: includes · Flux family adapters
- Models → SD 3.5: includes · Stable Diffusion family adapter
- Models → Qwen Image: includes · Qwen image adapters
- Models → Wan: includes · Wan video adapters
- Models → Z-Image: includes · Z-Image adapters
- Models → LTX-2: includes · LTX-2 audio-video adapters
- Scheduler → Samples: drives · Sampling rules shape rollout lifecycles
- Data → Samples: feeds · Batches and modality-specific inputs become samples
- Rewards → Reward Proc: processed by · Raw reward implementations are normalized and composed
- Rewards → Remote Judge: can delegate · Supports remote judge-style evaluation
- Reward Proc → Advantage: supplies · Processed reward outputs feed advantage computation
- Examples → Trainers: configures · Examples bind trainer choice and algorithm settings
- Examples → Models: configures · Examples bind model family and modality
- Examples → Rewards: configures · Examples show reward selection and composition
- Runtime configs → Train: supports · Launch configs support multi-GPU and multi-node runs
- Logger → Utils: uses · Logging integrates shared formatting and runtime helpers
- Remote Judge → Rewards: extends · External judge path complements local reward modules
