# deepseek-ai/esft architecture

> Architecture diagram of the GitHub repository deepseek-ai/esft, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepseek-ai/esft
- Repository: https://github.com/deepseek-ai/esft

## Overview

ESFT specializes a sparse MoE language model by measuring expert activity, selecting task-relevant experts, fine-tuning those components, and evaluating the resulting adapter. Supporting capabilities include standard and expert-parallel training, single- and multi-GPU evaluation, and task-specific scoring for intent, law, summary, and translation. The source excerpts show the core orchestration and model adaptation; some subsystem links are documented in the README rather than verified in unsampled implementation details.

Main entry points: [`train.py`](https://github.com/deepseek-ai/esft/blob/main/train.py) (Standard training), [`train_ep.py`](https://github.com/deepseek-ai/esft/blob/main/train_ep.py) (Expert-parallel training), [`scripts/eval.py`](https://github.com/deepseek-ai/esft/blob/main/scripts/eval.py) (Single-GPU evaluation), [`eval_multigpu.py`](https://github.com/deepseek-ai/esft/blob/main/eval_multigpu.py) (Multi-GPU evaluation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_analysis["Expert analysis"]
  node_train_corpus[("Training corpus")]
  node_score_runner["Expert scoring"]
  node_expert_scores[("Expert score files")]
  node_config_generator["Expert selection"]
  node_expert_config["Expert configuration"]
end

subgraph group_training["Specialized training"]
  node_training_data["Training preparation<br/>[utils.py]"]
  node_trainer["Standard training<br/>[train.py]"]
  node_ep_trainer["Expert-parallel training<br/>[train_ep.py]"]
  node_adapter_logic["ESFT adaptation<br/>[esft.py]"]
  node_adapter_output[("Trained adapters")]
end

subgraph group_model["MoE model"]
  node_model_config["DeepSeek configuration"]
  node_model_runtime["DeepSeek MoE model"]
end

subgraph group_evaluation["Evaluation"]
  node_eval_corpus[("Evaluation datasets")]
  node_evaluator["Task evaluators<br/>[benchmarks.py]"]
  node_single_eval["Single-GPU evaluation<br/>[eval.py]"]
  node_multi_eval["Multi-GPU evaluation<br/>[eval_multigpu.py]"]
  node_completions[("Evaluation results")]
end

node_user(("Researcher"))
node_base_model["Pretrained model"]
node_openai["OpenAI GPT-4"]

node_user -->|"runs scoring"| node_score_runner
node_train_corpus -->|"samples tokens"| node_score_runner
node_score_runner -->|"logs expert use"| node_expert_scores
node_expert_scores -->|"aggregates scores"| node_config_generator
node_config_generator -->|"writes selection"| node_expert_config
node_user -->|"runs training"| node_trainer
node_user -->|"runs parallel training"| node_ep_trainer
node_expert_config -->|"selects trainable experts"| node_trainer
node_expert_config -->|"selects trainable experts"| node_ep_trainer
node_train_corpus -->|"provides messages"| node_training_data
node_training_data -->|"formats examples"| node_trainer
node_training_data -->|"formats examples"| node_ep_trainer
node_trainer -->|"loads model"| node_model_runtime
node_ep_trainer -->|"loads model"| node_model_runtime
node_trainer -->|"configures adaptation"| node_adapter_logic
node_ep_trainer -->|"configures adaptation"| node_adapter_logic
node_trainer -->|"saves checkpoints"| node_adapter_output
node_ep_trainer -->|"saves checkpoints"| node_adapter_output
node_model_runtime -->|"uses configuration"| node_model_config
node_user -->|"runs evaluation"| node_single_eval
node_user -->|"runs evaluation"| node_multi_eval
node_base_model -->|"loads base weights"| node_single_eval
node_base_model -->|"loads base weights"| node_multi_eval
node_adapter_output -->|"loads adapter"| node_single_eval
node_adapter_output -->|"loads adapter"| node_multi_eval
node_eval_corpus -->|"supplies examples"| node_single_eval
node_eval_corpus -->|"supplies examples"| node_multi_eval
node_single_eval -->|"evaluates predictions"| node_evaluator
node_multi_eval -->|"evaluates predictions"| node_evaluator
node_evaluator -.->|"requests GPT-4 scores"| node_openai
node_single_eval -->|"writes scores"| node_completions
node_multi_eval -->|"writes rank results"| node_completions

click node_train_corpus "https://github.com/deepseek-ai/esft/tree/main/datasets/train"
click node_score_runner "https://github.com/deepseek-ai/esft/blob/main/scripts/expert/get_expert_scores.py"
click node_expert_scores "https://github.com/deepseek-ai/esft/tree/main/results/expert_scores"
click node_config_generator "https://github.com/deepseek-ai/esft/blob/main/scripts/expert/generate_expert_config.py"
click node_expert_config "https://github.com/deepseek-ai/esft/tree/main/results/expert_configs"
click node_training_data "https://github.com/deepseek-ai/esft/blob/main/utils.py"
click node_trainer "https://github.com/deepseek-ai/esft/blob/main/train.py"
click node_ep_trainer "https://github.com/deepseek-ai/esft/blob/main/train_ep.py"
click node_adapter_logic "https://github.com/deepseek-ai/esft/blob/main/esft.py"
click node_adapter_output "https://github.com/deepseek-ai/esft/tree/main/results"
click node_model_config "https://github.com/deepseek-ai/esft/blob/main/deepseek/configuration_deepseek.py"
click node_model_runtime "https://github.com/deepseek-ai/esft/blob/main/deepseek/modeling_deepseek.py"
click node_eval_corpus "https://github.com/deepseek-ai/esft/tree/main/datasets/eval"
click node_evaluator "https://github.com/deepseek-ai/esft/blob/main/benchmarks.py"
click node_single_eval "https://github.com/deepseek-ai/esft/blob/main/scripts/eval.py"
click node_multi_eval "https://github.com/deepseek-ai/esft/blob/main/eval_multigpu.py"
click node_completions "https://github.com/deepseek-ai/esft/tree/main/results/completions"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_corpus,node_score_runner,node_expert_scores,node_config_generator,node_expert_config toneBlue
class node_training_data,node_trainer,node_ep_trainer,node_adapter_logic,node_adapter_output toneAmber
class node_model_config,node_model_runtime toneMint
class node_eval_corpus,node_evaluator,node_single_eval,node_multi_eval,node_completions toneRose
class node_user,node_base_model,node_openai toneIndigo
```

## Components

### Expert analysis

- [Training corpus](https://github.com/deepseek-ai/esft/tree/main/datasets/train): `datasets/train`
- [Expert scoring](https://github.com/deepseek-ai/esft/blob/main/scripts/expert/get_expert_scores.py): `scripts/expert/get_expert_scores.py`
- [Expert score files](https://github.com/deepseek-ai/esft/tree/main/results/expert_scores): `results/expert_scores`
- [Expert selection](https://github.com/deepseek-ai/esft/blob/main/scripts/expert/generate_expert_config.py): `scripts/expert/generate_expert_config.py`
- [Expert configuration](https://github.com/deepseek-ai/esft/tree/main/results/expert_configs): `results/expert_configs`

### Specialized training

- [Training preparation](https://github.com/deepseek-ai/esft/blob/main/utils.py): `utils.py`
- [Standard training](https://github.com/deepseek-ai/esft/blob/main/train.py): `train.py`
- [Expert-parallel training](https://github.com/deepseek-ai/esft/blob/main/train_ep.py): `train_ep.py`
- [ESFT adaptation](https://github.com/deepseek-ai/esft/blob/main/esft.py): `esft.py`
- [Trained adapters](https://github.com/deepseek-ai/esft/tree/main/results): `results`

### MoE model

- [DeepSeek configuration](https://github.com/deepseek-ai/esft/blob/main/deepseek/configuration_deepseek.py): `deepseek/configuration_deepseek.py`
- [DeepSeek MoE model](https://github.com/deepseek-ai/esft/blob/main/deepseek/modeling_deepseek.py): `deepseek/modeling_deepseek.py`

### Evaluation

- [Evaluation datasets](https://github.com/deepseek-ai/esft/tree/main/datasets/eval): `datasets/eval`
- [Task evaluators](https://github.com/deepseek-ai/esft/blob/main/benchmarks.py): `benchmarks.py`
- [Single-GPU evaluation](https://github.com/deepseek-ai/esft/blob/main/scripts/eval.py): `scripts/eval.py`
- [Multi-GPU evaluation](https://github.com/deepseek-ai/esft/blob/main/eval_multigpu.py): `eval_multigpu.py`
- [Evaluation results](https://github.com/deepseek-ai/esft/tree/main/results/completions): `results/completions`

### Other components

- **Researcher**
- **Pretrained model**
- **OpenAI GPT-4**

## Connections

- Researcher → Expert scoring: runs scoring (evidence: `README.md`)
- Training corpus → Expert scoring: samples tokens (evidence: `scripts/expert/get_expert_scores.py`)
- Expert scoring → Expert score files: logs expert use (evidence: `scripts/expert/get_expert_scores.py`)
- Expert score files → Expert selection: aggregates scores (evidence: `scripts/expert/generate_expert_config.py`)
- Expert selection → Expert configuration: writes selection (evidence: `scripts/expert/generate_expert_config.py`)
- Researcher → Standard training: runs training (evidence: `README.md`)
- Researcher → Expert-parallel training: runs parallel training (evidence: `README.md`)
- Expert configuration → Standard training: selects trainable experts (evidence: `README.md`)
- Expert configuration → Expert-parallel training: selects trainable experts (evidence: `README.md`)
- Training corpus → Training preparation: provides messages
- Training preparation → Standard training: formats examples (evidence: `train.py`)
- Training preparation → Expert-parallel training: formats examples (evidence: `train_ep.py`)
- Standard training → DeepSeek MoE model: loads model (evidence: `train.py`)
- Expert-parallel training → DeepSeek MoE model: loads model (evidence: `train_ep.py`)
- Standard training → ESFT adaptation: configures adaptation (evidence: `train.py`)
- Expert-parallel training → ESFT adaptation: configures adaptation (evidence: `train_ep.py`)
- Standard training → Trained adapters: saves checkpoints (evidence: `README.md`)
- Expert-parallel training → Trained adapters: saves checkpoints (evidence: `README.md`)
- DeepSeek MoE model → DeepSeek configuration: uses configuration (evidence: `deepseek/modeling_deepseek.py`)
- Researcher → Single-GPU evaluation: runs evaluation (evidence: `README.md`)
- Researcher → Multi-GPU evaluation: runs evaluation (evidence: `README.md`)
- Pretrained model → Single-GPU evaluation: loads base weights (evidence: `scripts/eval.py`)
- Pretrained model → Multi-GPU evaluation: loads base weights (evidence: `eval_multigpu.py`)
- Trained adapters → Single-GPU evaluation: loads adapter (evidence: `scripts/eval.py`)
- Trained adapters → Multi-GPU evaluation: loads adapter (evidence: `eval_multigpu.py`)
- Evaluation datasets → Single-GPU evaluation: supplies examples (evidence: `scripts/eval.py`)
- Evaluation datasets → Multi-GPU evaluation: supplies examples (evidence: `eval_multigpu.py`)
- Single-GPU evaluation → Task evaluators: evaluates predictions (evidence: `scripts/eval.py`)
- Multi-GPU evaluation → Task evaluators: evaluates predictions (evidence: `eval_multigpu.py`)
- Task evaluators → OpenAI GPT-4: requests GPT-4 scores (evidence: `benchmarks.py`)
- Single-GPU evaluation → Evaluation results: writes scores (evidence: `scripts/eval.py`)
- Multi-GPU evaluation → Evaluation results: writes rank results (evidence: `eval_multigpu.py`)