# modelscope/easydistill architecture

> Architecture diagram of the GitHub repository modelscope/easydistill, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/modelscope/easydistill
- Repository: https://github.com/modelscope/easydistill

## Overview

EasyDistill 2 is a config-driven toolkit for converting black-box teacher models into training data. Users select a workflow through the CLI; the configured pipeline applies generation and, as appropriate, synthesis, evaluation/filtering, rewriting, balancing, or preference scoring, then builds SFT or DPO outputs. Separate capabilities cover text, vision-language, image/video generation, and agent trajectories. The graph preserves these documented branches, while sampled source confirms selected dispatch, generation, backend, and data-model relationships; unsampled implementations are represented at their existing directory paths, with links omitted where evidence is insufficient.

Main entry points: [`easydistill/pipeline`](https://github.com/modelscope/easydistill/tree/main/easydistill/pipeline) (Distillation pipelines), [`easydistill/cli/runners`](https://github.com/modelscope/easydistill/tree/main/easydistill/cli/runners) (Job runners), [`easydistill/cli/backend_factory.py`](https://github.com/modelscope/easydistill/blob/main/easydistill/cli/backend_factory.py) (Backend factory), [`easydistill/operators/generation.py`](https://github.com/modelscope/easydistill/blob/main/easydistill/operators/generation.py) (Generation operators)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Workflow entry"]
  node_cli["CLI dispatch<br/>[main.py]"]
  node_runner["Job runners"]
  node_loaders["Dataset loading<br/>[data_loaders.py]"]
end

subgraph group_processing["Distillation processing"]
  node_pipeline["Distillation pipelines"]
  node_basic["Basic distillation<br/>[distillation.py]"]
  node_operators["Generation operators<br/>[generation.py]"]
  node_synthesis["Synthesis and rewrite"]
  node_evaluation["Evaluation and filtering"]
  node_agent["Agent trajectories<br/>[trajectory.py]"]
  node_search["Search task evolution<br/>[task_evolver.py]"]
  node_preference["Preference construction"]
end

subgraph group_modalities["Multimodal generation"]
  node_imageops["Image generation"]
  node_videoops["Video generation<br/>[t2v_generation.py]"]
end

subgraph group_integrations["Models and services"]
  node_backendfactory["Backend factory<br/>[backend_factory.py]"]
  node_backend["Model backend API<br/>[base.py]"]
  node_imagebackend["Image backends"]
  node_videobackend["Video backends<br/>[t2v_base.py]"]
end

subgraph group_outputs["Training data"]
  node_models["Typed data models<br/>[models.py]"]
  node_sft["SFT dataset builder<br/>[sft_builder.py]"]
end

node_user(("Workflow user"))
node_teacher{{"Teacher model service"}}
node_outputs[("Training-ready datasets")]

node_user -->|"selects job"| node_cli
node_cli -->|"dispatches job"| node_runner
node_runner -->|"builds backend"| node_backendfactory
node_runner -->|"runs workflow"| node_pipeline
node_runner -.->|"runs distillation"| node_basic
node_runner -.->|"loads input data"| node_loaders
node_pipeline -->|"composes generation"| node_operators
node_pipeline -->|"composes synthesis"| node_synthesis
node_pipeline -->|"composes evaluation"| node_evaluation
node_pipeline -->|"builds SFT"| node_sft
node_pipeline -->|"builds DPO pairs"| node_preference
node_pipeline -->|"rolls trajectories"| node_agent
node_pipeline -->|"evolves search tasks"| node_search
node_pipeline -->|"supports image jobs"| node_imageops
node_pipeline -.->|"supports video jobs"| node_videoops
node_backendfactory -->|"creates backend"| node_backend
node_operators -->|"requests generation"| node_backend
node_backend -->|"calls model service"| node_teacher
node_backendfactory -->|"creates image backend"| node_imagebackend
node_imageops -.->|"requests images"| node_imagebackend
node_backendfactory -.->|"creates video backend"| node_videobackend
node_videoops -->|"requests videos"| node_videobackend
node_operators -->|"uses request models"| node_models
node_sft -.->|"builds samples"| node_models
node_preference -.->|"builds preference samples"| node_models
node_sft -->|"exports datasets"| node_outputs
node_preference -->|"exports preference pairs"| node_outputs

click node_cli "https://github.com/modelscope/easydistill/blob/main/easydistill/cli/main.py"
click node_runner "https://github.com/modelscope/easydistill/tree/main/easydistill/cli/runners"
click node_loaders "https://github.com/modelscope/easydistill/blob/main/easydistill/cli/data_loaders.py"
click node_pipeline "https://github.com/modelscope/easydistill/tree/main/easydistill/pipeline"
click node_basic "https://github.com/modelscope/easydistill/blob/main/easydistill/basic/distillation.py"
click node_operators "https://github.com/modelscope/easydistill/blob/main/easydistill/operators/generation.py"
click node_synthesis "https://github.com/modelscope/easydistill/tree/main/easydistill/rewrite"
click node_evaluation "https://github.com/modelscope/easydistill/tree/main/easydistill/eval"
click node_agent "https://github.com/modelscope/easydistill/blob/main/easydistill/operators/agent/trajectory.py"
click node_search "https://github.com/modelscope/easydistill/blob/main/easydistill/operators/search_agent/task_evolver.py"
click node_preference "https://github.com/modelscope/easydistill/tree/main/easydistill/operators/preference"
click node_imageops "https://github.com/modelscope/easydistill/tree/main/easydistill/operators/t2i"
click node_videoops "https://github.com/modelscope/easydistill/blob/main/easydistill/operators/t2v/t2v_generation.py"
click node_backendfactory "https://github.com/modelscope/easydistill/blob/main/easydistill/cli/backend_factory.py"
click node_backend "https://github.com/modelscope/easydistill/blob/main/easydistill/backends/base.py"
click node_imagebackend "https://github.com/modelscope/easydistill/tree/main/easydistill/backends"
click node_videobackend "https://github.com/modelscope/easydistill/blob/main/easydistill/backends/t2v_base.py"
click node_models "https://github.com/modelscope/easydistill/blob/main/easydistill/data/models.py"
click node_sft "https://github.com/modelscope/easydistill/blob/main/easydistill/operators/sft_builder.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_runner,node_loaders,node_user toneBlue
class node_pipeline,node_basic,node_operators,node_synthesis,node_evaluation,node_agent,node_search,node_preference,node_outputs toneAmber
class node_imageops,node_videoops toneMint
class node_backendfactory,node_backend,node_imagebackend,node_videobackend toneRose
class node_models,node_sft,node_teacher toneIndigo
```

## Components

### Workflow entry

- [CLI dispatch](https://github.com/modelscope/easydistill/blob/main/easydistill/cli/main.py): `easydistill/cli/main.py`
- [Job runners](https://github.com/modelscope/easydistill/tree/main/easydistill/cli/runners): `easydistill/cli/runners`
- [Dataset loading](https://github.com/modelscope/easydistill/blob/main/easydistill/cli/data_loaders.py): `easydistill/cli/data_loaders.py`

### Distillation processing

- [Distillation pipelines](https://github.com/modelscope/easydistill/tree/main/easydistill/pipeline): `easydistill/pipeline`
- [Basic distillation](https://github.com/modelscope/easydistill/blob/main/easydistill/basic/distillation.py): `easydistill/basic/distillation.py`
- [Generation operators](https://github.com/modelscope/easydistill/blob/main/easydistill/operators/generation.py): `easydistill/operators/generation.py`
- [Synthesis and rewrite](https://github.com/modelscope/easydistill/tree/main/easydistill/rewrite): `easydistill/rewrite`
- [Evaluation and filtering](https://github.com/modelscope/easydistill/tree/main/easydistill/eval): `easydistill/eval`
- [Agent trajectories](https://github.com/modelscope/easydistill/blob/main/easydistill/operators/agent/trajectory.py): `easydistill/operators/agent/trajectory.py`
- [Search task evolution](https://github.com/modelscope/easydistill/blob/main/easydistill/operators/search_agent/task_evolver.py): `easydistill/operators/search_agent/task_evolver.py`
- [Preference construction](https://github.com/modelscope/easydistill/tree/main/easydistill/operators/preference): `easydistill/operators/preference`

### Multimodal generation

- [Image generation](https://github.com/modelscope/easydistill/tree/main/easydistill/operators/t2i): `easydistill/operators/t2i`
- [Video generation](https://github.com/modelscope/easydistill/blob/main/easydistill/operators/t2v/t2v_generation.py): `easydistill/operators/t2v/t2v_generation.py`

### Models and services

- [Backend factory](https://github.com/modelscope/easydistill/blob/main/easydistill/cli/backend_factory.py): `easydistill/cli/backend_factory.py`
- [Model backend API](https://github.com/modelscope/easydistill/blob/main/easydistill/backends/base.py): `easydistill/backends/base.py`
- [Image backends](https://github.com/modelscope/easydistill/tree/main/easydistill/backends): `easydistill/backends`
- [Video backends](https://github.com/modelscope/easydistill/blob/main/easydistill/backends/t2v_base.py): `easydistill/backends/t2v_base.py`

### Training data

- [Typed data models](https://github.com/modelscope/easydistill/blob/main/easydistill/data/models.py): `easydistill/data/models.py`
- [SFT dataset builder](https://github.com/modelscope/easydistill/blob/main/easydistill/operators/sft_builder.py): `easydistill/operators/sft_builder.py`

### Other components

- **Workflow user**
- **Teacher model service**
- **Training-ready datasets**

## Connections

- Workflow user → CLI dispatch: selects job
- CLI dispatch → Job runners: dispatches job (evidence: `easydistill/cli/main.py`)
- Job runners → Backend factory: builds backend (evidence: `easydistill/cli/runners/pipeline.py`)
- Job runners → Distillation pipelines: runs workflow (evidence: `easydistill/cli/runners/pipeline.py`)
- Job runners → Basic distillation: runs distillation
- Job runners → Dataset loading: loads input data
- Distillation pipelines → Generation operators: composes generation
- Distillation pipelines → Synthesis and rewrite: composes synthesis (evidence: `easydistill/pipeline/common.py`)
- Distillation pipelines → Evaluation and filtering: composes evaluation
- Distillation pipelines → SFT dataset builder: builds SFT
- Distillation pipelines → Preference construction: builds DPO pairs (evidence: `easydistill/pipeline/common.py`)
- Distillation pipelines → Agent trajectories: rolls trajectories
- Distillation pipelines → Search task evolution: evolves search tasks
- Distillation pipelines → Image generation: supports image jobs (evidence: `easydistill/pipeline/common.py`)
- Distillation pipelines → Video generation: supports video jobs
- Backend factory → Model backend API: creates backend (evidence: `easydistill/cli/backend_factory.py`)
- Generation operators → Model backend API: requests generation (evidence: `easydistill/operators/generation.py`)
- Model backend API → Teacher model service: calls model service
- Backend factory → Image backends: creates image backend (evidence: `easydistill/cli/backend_factory.py`)
- Image generation → Image backends: requests images
- Backend factory → Video backends: creates video backend
- Video generation → Video backends: requests videos (evidence: `easydistill/operators/t2v/t2v_generation.py`)
- Generation operators → Typed data models: uses request models (evidence: `easydistill/operators/generation.py`)
- SFT dataset builder → Typed data models: builds samples
- Preference construction → Typed data models: builds preference samples
- SFT dataset builder → Training-ready datasets: exports datasets
- Preference construction → Training-ready datasets: exports preference pairs
