# zhziszz/dllm architecture

> Architecture diagram of the GitHub repository zhziszz/dllm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zhziszz/dllm
- Repository: https://github.com/zhziszz/dllm

## Overview

dLLM is a library for training, sampling from, and evaluating diffusion language models. The principal workflow is to prepare training data, train a model with a reusable algorithm trainer or a model-specific pipeline, sample outputs, and evaluate them. Separate branches cover model conversion, accelerated inference, editing-based diffusion, and GRPO reasoning training. The README documents these capabilities, while source-level edges below are limited to relationships visible in excerpts or the SOURCE INDEX; unsampled internals are not assumed.

Main entry points: [`dllm/core/trainers/mdlm.py`](https://github.com/zhziszz/dllm/blob/main/dllm/core/trainers/mdlm.py) (MDLM Trainer), [`dllm/core/samplers/mdlm.py`](https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/mdlm.py) (MDLM Sampler), [`dllm/core/samplers/base.py`](https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/base.py) (Sampler Interface), [`dllm/pipelines/rl/grpo/datasets.py`](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/rl/grpo/datasets.py) (GRPO Datasets)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_shared["Shared Runtime"]
  node_trainers["MDLM Trainer<br/>[mdlm.py]"]
  node_bdtrainer["BD3LM Trainer<br/>[bd3lm.py]"]
  node_schedulers["Diffusion Schedulers<br/>[alpha.py]"]
  node_sampler_base["Sampler Interface<br/>[base.py]"]
  node_mdlm_sampler["MDLM Sampler<br/>[mdlm.py]"]
  node_bd_sampler["BD3LM Sampler<br/>[bd3lm.py]"]
  node_evaluation["Core Evaluation<br/>[base.py]"]
end

subgraph group_models["Model Pipelines"]
  node_dream["Dream Pipeline<br/>[modeling_dream.py]"]
  node_dream_generation["Dream Generation"]
  node_llada["LLaDA Models<br/>[modeling_llada.py]"]
  node_llada2["LLaDA 2 Models"]
  node_llada21["LLaDA 2.1 Models"]
  node_a2d["AR Conversion<br/>[convert.py]"]
  node_editflow["EditFlow Pipeline<br/>[trainer.py]"]
  node_fast["Fast-dLLM<br/>[sampler.py]"]
end

subgraph group_reasoning["Reasoning Training"]
  node_rl_data["GRPO Datasets<br/>[datasets.py]"]
  node_rewards["Task Rewards<br/>[math.py]"]
  node_grpo["GRPO Trainer<br/>[trainer.py]"]
end

subgraph group_utilities["Data and Utilities"]
  node_datasets["Dataset Loaders"]
  node_prep["Data Preparation<br/>[data.py]"]
  node_collators["Batch Collators<br/>[collators.py]"]
  node_configs["Training Config<br/>[configs.py]"]
  node_model_utils["Model Utilities<br/>[models.py]"]
  node_visualizers["Output Visualizers<br/>[visualizers.py]"]
end

node_user(("User"))

node_user -.->|"selects data"| node_datasets
node_datasets -.->|"prepare data"| node_prep
node_prep -.->|"supply training data"| node_trainers
node_trainers -->|"reads config"| node_configs
node_trainers -->|"processes data"| node_prep
node_bdtrainer -->|"extends trainer"| node_trainers
node_bdtrainer -->|"uses collators"| node_collators
node_sampler_base -->|"defaults scheduler"| node_schedulers
node_mdlm_sampler -->|"implements"| node_sampler_base
node_bd_sampler -->|"implements"| node_sampler_base
node_mdlm_sampler -->|"uses schedule"| node_schedulers
node_user -.->|"requests generation"| node_mdlm_sampler
node_user -.->|"requests evaluation"| node_evaluation
node_dream -->|"uses generation"| node_dream_generation
node_user -.->|"converts models"| node_a2d
node_user -.->|"trains edit models"| node_editflow
node_user -.->|"requests fast inference"| node_fast
node_user -.->|"trains or evaluates"| node_llada
node_user -.->|"requests inference"| node_llada2
node_user -.->|"requests inference"| node_llada21
node_user -.->|"starts GRPO training"| node_rl_data
node_rl_data -->|"maps task rewards"| node_rewards
node_grpo -.->|"uses task data"| node_rl_data
node_grpo -.->|"scores completions"| node_rewards

click node_datasets "https://github.com/zhziszz/dllm/tree/main/dllm/data"
click node_prep "https://github.com/zhziszz/dllm/blob/main/dllm/utils/data.py"
click node_collators "https://github.com/zhziszz/dllm/blob/main/dllm/utils/collators.py"
click node_configs "https://github.com/zhziszz/dllm/blob/main/dllm/utils/configs.py"
click node_model_utils "https://github.com/zhziszz/dllm/blob/main/dllm/utils/models.py"
click node_visualizers "https://github.com/zhziszz/dllm/blob/main/dllm/utils/visualizers.py"
click node_trainers "https://github.com/zhziszz/dllm/blob/main/dllm/core/trainers/mdlm.py"
click node_bdtrainer "https://github.com/zhziszz/dllm/blob/main/dllm/core/trainers/bd3lm.py"
click node_schedulers "https://github.com/zhziszz/dllm/blob/main/dllm/core/schedulers/alpha.py"
click node_sampler_base "https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/base.py"
click node_mdlm_sampler "https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/mdlm.py"
click node_bd_sampler "https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/bd3lm.py"
click node_evaluation "https://github.com/zhziszz/dllm/blob/main/dllm/core/eval/base.py"
click node_dream "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/dream/models/modeling_dream.py"
click node_dream_generation "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/dream/models/generation_utils.py"
click node_llada "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/llada/models/modeling_llada.py"
click node_llada2 "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/llada2/models/modeling_llada2_moe.py"
click node_llada21 "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/llada21/models/modeling_llada21_moe.py"
click node_a2d "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/a2d/convert.py"
click node_editflow "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/editflow/trainer.py"
click node_fast "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/fastdllm/llada/sampler.py"
click node_rl_data "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/rl/grpo/datasets.py"
click node_rewards "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/rl/grpo/rewards/math.py"
click node_grpo "https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/rl/grpo/trainer.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_trainers,node_bdtrainer,node_schedulers,node_sampler_base,node_mdlm_sampler,node_bd_sampler,node_evaluation,node_user toneBlue
class node_dream,node_dream_generation,node_llada,node_llada2,node_llada21,node_a2d,node_editflow,node_fast toneAmber
class node_rl_data,node_rewards,node_grpo toneMint
class node_datasets,node_prep,node_collators,node_configs,node_model_utils,node_visualizers toneRose
```

## Components

### Shared Runtime

- [MDLM Trainer](https://github.com/zhziszz/dllm/blob/main/dllm/core/trainers/mdlm.py): `dllm/core/trainers/mdlm.py`
- [BD3LM Trainer](https://github.com/zhziszz/dllm/blob/main/dllm/core/trainers/bd3lm.py): `dllm/core/trainers/bd3lm.py`
- [Diffusion Schedulers](https://github.com/zhziszz/dllm/blob/main/dllm/core/schedulers/alpha.py): `dllm/core/schedulers/alpha.py`
- [Sampler Interface](https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/base.py): `dllm/core/samplers/base.py`
- [MDLM Sampler](https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/mdlm.py): `dllm/core/samplers/mdlm.py`
- [BD3LM Sampler](https://github.com/zhziszz/dllm/blob/main/dllm/core/samplers/bd3lm.py): `dllm/core/samplers/bd3lm.py`
- [Core Evaluation](https://github.com/zhziszz/dllm/blob/main/dllm/core/eval/base.py): `dllm/core/eval/base.py`

### Model Pipelines

- [Dream Pipeline](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/dream/models/modeling_dream.py): `dllm/pipelines/dream/models/modeling_dream.py`
- [Dream Generation](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/dream/models/generation_utils.py): `dllm/pipelines/dream/models/generation_utils.py`
- [LLaDA Models](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/llada/models/modeling_llada.py): `dllm/pipelines/llada/models/modeling_llada.py`
- [LLaDA 2 Models](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/llada2/models/modeling_llada2_moe.py): `dllm/pipelines/llada2/models/modeling_llada2_moe.py`
- [LLaDA 2.1 Models](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/llada21/models/modeling_llada21_moe.py): `dllm/pipelines/llada21/models/modeling_llada21_moe.py`
- [AR Conversion](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/a2d/convert.py): `dllm/pipelines/a2d/convert.py`
- [EditFlow Pipeline](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/editflow/trainer.py): `dllm/pipelines/editflow/trainer.py`
- [Fast-dLLM](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/fastdllm/llada/sampler.py): `dllm/pipelines/fastdllm/llada/sampler.py`

### Reasoning Training

- [GRPO Datasets](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/rl/grpo/datasets.py): `dllm/pipelines/rl/grpo/datasets.py`
- [Task Rewards](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/rl/grpo/rewards/math.py): `dllm/pipelines/rl/grpo/rewards/math.py`
- [GRPO Trainer](https://github.com/zhziszz/dllm/blob/main/dllm/pipelines/rl/grpo/trainer.py): `dllm/pipelines/rl/grpo/trainer.py`

### Data and Utilities

- [Dataset Loaders](https://github.com/zhziszz/dllm/tree/main/dllm/data): `dllm/data`
- [Data Preparation](https://github.com/zhziszz/dllm/blob/main/dllm/utils/data.py): `dllm/utils/data.py`
- [Batch Collators](https://github.com/zhziszz/dllm/blob/main/dllm/utils/collators.py): `dllm/utils/collators.py`
- [Training Config](https://github.com/zhziszz/dllm/blob/main/dllm/utils/configs.py): `dllm/utils/configs.py`
- [Model Utilities](https://github.com/zhziszz/dllm/blob/main/dllm/utils/models.py): `dllm/utils/models.py`
- [Output Visualizers](https://github.com/zhziszz/dllm/blob/main/dllm/utils/visualizers.py): `dllm/utils/visualizers.py`

### Other components

- **User**

## Connections

- User → Dataset Loaders: selects data (evidence: `README.md`)
- Dataset Loaders → Data Preparation: prepare data
- Data Preparation → MDLM Trainer: supply training data (evidence: `README.md`)
- MDLM Trainer → Training Config: reads config (evidence: `dllm/core/trainers/mdlm.py`)
- MDLM Trainer → Data Preparation: processes data (evidence: `dllm/core/trainers/mdlm.py`)
- BD3LM Trainer → MDLM Trainer: extends trainer (evidence: `dllm/core/trainers/bd3lm.py`)
- BD3LM Trainer → Batch Collators: uses collators (evidence: `dllm/core/trainers/bd3lm.py`)
- Sampler Interface → Diffusion Schedulers: defaults scheduler (evidence: `dllm/core/samplers/base.py`)
- MDLM Sampler → Sampler Interface: implements (evidence: `dllm/core/samplers/mdlm.py`)
- BD3LM Sampler → Sampler Interface: implements (evidence: `dllm/core/samplers/bd3lm.py`)
- MDLM Sampler → Diffusion Schedulers: uses schedule
- User → MDLM Sampler: requests generation (evidence: `README.md`)
- User → Core Evaluation: requests evaluation (evidence: `README.md`)
- Dream Pipeline → Dream Generation: uses generation (evidence: `dllm/pipelines/dream/models/modeling_dream.py`)
- User → AR Conversion: converts models (evidence: `README.md`)
- User → EditFlow Pipeline: trains edit models (evidence: `README.md`)
- User → Fast-dLLM: requests fast inference (evidence: `README.md`)
- User → LLaDA Models: trains or evaluates (evidence: `README.md`)
- User → LLaDA 2 Models: requests inference (evidence: `README.md`)
- User → LLaDA 2.1 Models: requests inference (evidence: `README.md`)
- User → GRPO Datasets: starts GRPO training (evidence: `README.md`)
- GRPO Datasets → Task Rewards: maps task rewards (evidence: `dllm/pipelines/rl/grpo/datasets.py`)
- GRPO Trainer → GRPO Datasets: uses task data
- GRPO Trainer → Task Rewards: scores completions