# deepseek-ai/deepseek-coder architecture

> Architecture diagram of the GitHub repository deepseek-ai/deepseek-coder, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepseek-ai/deepseek-coder
- Repository: https://github.com/deepseek-ai/deepseek-coder

## Overview

This repository is a **model application and evaluation toolkit** for DeepSeek Coder, not the model pretraining code itself. The top-level architecture splits into three main concerns: **benchmark evaluation**, **fine-tuning**, and a small **interactive demo**. The README also documents how the released models are consumed through Hugging Face Transformers and vLLM.

##### 1. Core boundary of the repo
The repo assumes pretrained DeepSeek Coder checkpoints are downloaded externally from Hugging Face. Most code here is about:
- loading those models,
- formatting prompts for different tasks,
- running generation,
- grading outputs,
- and comparing results across benchmarks.

##### 2. Evaluation subsystem as the main product
The `Evaluation/` directory is the largest part of the repo and is organized by benchmark family:
- `HumanEval`
- `MBPP`
- `LeetCode`
- `DS-1000`
- `PAL-Math`

Each benchmark folder contains its own dataset files, prompt/eval scripts, and helper code. The common architectural pattern is: **dataset → prompt construction → model inference → execution/grading → metric report**.

##### 3. HumanEval and MBPP share a common execution model
`Evaluation/HumanEval` and `Evaluation/MBPP` both vendor a `human_eval` package with modules like `data.py`, `execution.py`, and `evaluate_functional_correctness.py`. This layer is the sandboxed code-execution core that:
- loads tasks from JSONL datasets,
- runs generated Python solutions,
- measures functional correctness using pass/fail outcomes,
- and computes benchmark metrics such as pass@k.

These folders also include `eval_instruct.py`, `eval_pal.py`, and `eval.sh`, showing support for multiple prompting styles:
- instruction-style generation,
- program-aided/“PAL” style reasoning,
- and shell-driven batch runs.

##### 4. Benchmark-specific data and prompt layers
The benchmark folders keep their own data formats and configs close to the evaluator:
- `HumanEval/data/*` contains multilingual HumanEval variants.
- `MBPP/data/*` contains MBPP train/test JSONL files.
- `test_config.yaml` files define run settings for each benchmark.

This keeps benchmark semantics local, so the evaluator logic can stay generic while the prompts and test suites remain benchmark-specific.

##### 5. LeetCode evaluation is a separate flow
`Evaluation/LeetCode` is structurally different from HumanEval/MBPP:
- `evaluate_leetcode.py` is the benchmark driver,
- `vllm_inference.py` indicates high-throughput batched inference via vLLM,
- and the local `human_eval` package provides shared execution/evaluation utilities.

This implies a flow optimized for larger-scale generation throughput, likely with multilingual task data in `data/*.jsonl`.

##### 6. PAL-Math is a program-execution reasoning pipeline
`Evaluation/PAL-Math` is a distinct subsystem for math reasoning with executable intermediate steps. Its architecture is:
- task datasets under `datasets/`,
- prompt templates under `prompts/`,
- orchestration in `run.py`,
- grading/parsing/execution in `utils/`.

The important boundary here is that the model output is not only text; it is parsed into program-like reasoning, executed by `python_executor.py`, then checked by `grader.py` and `parser.py`. This makes PAL-Math an execution-backed reasoning benchmark rather than a pure text-generation benchmark.

##### 7. Fine-tuning is isolated from evaluation
`finetune/` is a separate training subsystem. It includes:
- `finetune_deepseekcoder.py` as the trainer entry point,
- a DeepSpeed ZeRO-3 config in `configs/ds_config_zero3.json`,
- and its own requirements file.

The data contract for fine-tuning is simple and externalized: instruction-following examples with `instruction` and `output` fields. The training path therefore is:
**prepared instruction dataset → DeepSpeed training script → fine-tuned model checkpoint**.

##### 8. Inference modes are documented but not deeply integrated
The README describes three inference patterns that act like consumer interfaces to the model:
- plain completion with Transformers,
- chat completion using `apply_chat_template`,
- repository-level completion/infill,
- and high-throughput generation via vLLM.

Architecturally, these are not separate services in the repo; they are usage modes for the same underlying model checkpoints.

##### 9. Demo is a thin interactive front end
`demo/app.py` plus `style.css` and `requirement.txt` form a small local UI layer. It likely wraps model inference behind a simple interactive interface for trying prompts and completions. This layer sits at the edge of the system and depends on the model runtime rather than owning any evaluation or training logic.

##### 10. Shared utilities are duplicated per benchmark family
Several folders have `utils/` or `human_eval/` packages. These are not global abstractions; they are **per-benchmark support libraries** for:
- dataset loading,
- prompt formatting,
- code execution,
- grading,
- and result aggregation.

That duplication is intentional because each benchmark has slightly different task formats and scoring rules.

##### 11. External runtime and infrastructure dependencies matter
The repo relies on a few major external technologies that shape the architecture:
- **PyTorch / Transformers** for model loading and generation,
- **vLLM** for scalable batched inference,
- **DeepSpeed** for distributed fine-tuning,
- and benchmark-specific execution of generated code in Python sandboxes.

These are not incidental libraries; they define the repo’s execution model and scalability boundaries.

##### 12. Data assets are static and benchmark-local
The `data/`, `datasets/`, and `prompts/` directories are mostly static assets. They are inputs to the evaluation pipeline, not runtime-generated outputs. The repo also includes model and code licenses separately, reinforcing the split between:
- repository code,
- benchmark assets,
- and external model weights/licenses.

##### 13. How the pieces fit together
A good architecture diagram for this repo would show three parallel lanes:
1. **Evaluation lane**: benchmark data → prompt builder → model inference → code execution/grading → metrics.
2. **Fine-tuning lane**: instruction dataset → DeepSpeed trainer → adapted checkpoint.
3. **Demo/inference lane**: user prompt → Transformers or vLLM → generated completion/chat response.

The key architectural idea is that DeepSeek Coder is treated as an external pretrained model asset, while this repository provides the surrounding tooling to **evaluate, adapt, and demo** it across code and reasoning tasks.

Main entry points: [`Evaluation`](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation) (Eval hub), [`Evaluation/LeetCode`](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/LeetCode) (LeetCode eval), [`Evaluation/PAL-Math/run.py`](https://github.com/deepseek-ai/deepseek-coder/blob/main/Evaluation/PAL-Math/run.py) (PAL-Math run), [`finetune/finetune_deepseekcoder.py`](https://github.com/deepseek-ai/deepseek-coder/blob/main/finetune/finetune_deepseekcoder.py) (Fine-tune script)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g_eval["Evaluation"]
  node_n_evalhub["Eval hub<br/>benchmark suite"]
  node_n_he_data["HumanEval data<br/>datasets + prompts"]
  node_n_he_runner["HumanEval runner<br/>eval scripts<br/>[eval_instruct.py]"]
  node_n_he_exec["HumanEval core<br/>functional eval"]
  node_n_mbpp_data["MBPP data<br/>datasets + prompts"]
  node_n_mbpp_runner["MBPP runner<br/>eval scripts<br/>[eval_instruct.py]"]
  node_n_mbpp_exec["MBPP core<br/>functional eval"]
  node_n_leetcode["LeetCode eval<br/>batched benchmark"]
  node_n_pal_data["PAL-Math assets<br/>datasets + prompts"]
  node_n_pal_run["PAL-Math run<br/>orchestrator<br/>[run.py]"]
  node_n_pal_exec["PAL-Math exec<br/>parse-grade-exec"]
  node_n_ds1000["DS-1000<br/>benchmark docs"]
end

subgraph group_g_finetune["Fine-tuning"]
  node_n_train_data["Train data<br/>instruction dataset"]
  node_n_train["Fine-tune script<br/>trainer"]
  node_n_train_cfg["ZeRO-3 config<br/>ds config"]
end

subgraph group_g_demo["Demo"]
  node_n_demo["Interactive demo<br/>ui wrapper<br/>[app.py]"]
end

subgraph group_g_external["External runtime"]
  node_n_model[("DeepSeek Coder<br/>external checkpoint")]
  node_n_transformers["Transformers<br/>inference runtime"]
  node_n_vllm["vLLM<br/>batch inference"]
  node_n_deepspeed["DeepSpeed<br/>training runtime"]
end

node_n_model -->|"loads"| node_n_transformers
node_n_model -->|"serves"| node_n_vllm
node_n_model -->|"fine-tuned by"| node_n_deepspeed
node_n_evalhub -->|"uses"| node_n_he_data
node_n_evalhub -->|"runs"| node_n_he_runner
node_n_he_runner -->|"invokes"| node_n_he_exec
node_n_he_runner -->|"infers with"| node_n_transformers
node_n_evalhub -->|"uses"| node_n_mbpp_data
node_n_evalhub -->|"runs"| node_n_mbpp_runner
node_n_mbpp_runner -->|"invokes"| node_n_mbpp_exec
node_n_mbpp_runner -->|"infers with"| node_n_transformers
node_n_leetcode -->|"batched infer"| node_n_vllm
node_n_leetcode -->|"fallback infer"| node_n_transformers
node_n_leetcode -->|"reuses"| node_n_he_exec
node_n_pal_run -->|"loads"| node_n_pal_data
node_n_pal_run -->|"executes"| node_n_pal_exec
node_n_pal_run -->|"queries"| node_n_transformers
node_n_train -->|"trains on"| node_n_train_data
node_n_train -->|"configured by"| node_n_train_cfg
node_n_train -->|"uses"| node_n_deepspeed
node_n_demo -->|"queries"| node_n_transformers
node_n_demo -->|"shows"| node_n_model
node_n_evalhub -->|"documents"| node_n_ds1000

click node_n_evalhub "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation"
click node_n_he_data "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/HumanEval/data"
click node_n_he_runner "https://github.com/deepseek-ai/deepseek-coder/blob/main/Evaluation/HumanEval/eval_instruct.py"
click node_n_he_exec "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/HumanEval/human_eval"
click node_n_mbpp_data "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/MBPP/data"
click node_n_mbpp_runner "https://github.com/deepseek-ai/deepseek-coder/blob/main/Evaluation/MBPP/eval_instruct.py"
click node_n_mbpp_exec "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/MBPP/human_eval"
click node_n_leetcode "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/LeetCode"
click node_n_pal_data "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/PAL-Math/datasets"
click node_n_pal_run "https://github.com/deepseek-ai/deepseek-coder/blob/main/Evaluation/PAL-Math/run.py"
click node_n_pal_exec "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/PAL-Math/utils"
click node_n_ds1000 "https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/DS-1000"
click node_n_train "https://github.com/deepseek-ai/deepseek-coder/blob/main/finetune/finetune_deepseekcoder.py"
click node_n_train_cfg "https://github.com/deepseek-ai/deepseek-coder/blob/main/finetune/configs/ds_config_zero3.json"
click node_n_demo "https://github.com/deepseek-ai/deepseek-coder/blob/main/demo/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n_evalhub,node_n_he_data,node_n_he_runner,node_n_he_exec,node_n_mbpp_data,node_n_mbpp_runner,node_n_mbpp_exec,node_n_leetcode,node_n_pal_data,node_n_pal_run,node_n_pal_exec,node_n_ds1000 toneBlue
class node_n_train_data,node_n_train,node_n_train_cfg toneAmber
class node_n_demo toneMint
class node_n_model,node_n_transformers,node_n_vllm,node_n_deepspeed toneRose
```

## Components

### Evaluation

Benchmarking and scoring pipelines for code and math tasks.

- [Eval hub](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation): benchmark suite · Top-level evaluation area that organizes benchmark-specific pipelines. · `Evaluation`
- [HumanEval data](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/HumanEval/data): datasets + prompts · Multilingual HumanEval task files and benchmark-local assets. · `Evaluation/HumanEval/data`
- [HumanEval runner](https://github.com/deepseek-ai/deepseek-coder/blob/main/Evaluation/HumanEval/eval_instruct.py): eval scripts · Instruction and PAL evaluation entrypoints plus shell batch execution. · `Evaluation/HumanEval/eval_instruct.py`
- [HumanEval core](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/HumanEval/human_eval): functional eval · Shared execution and correctness checking for generated programs. · `Evaluation/HumanEval/human_eval`
- [MBPP data](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/MBPP/data): datasets + prompts · MBPP training and test JSONL data kept close to the evaluator. · `Evaluation/MBPP/data`
- [MBPP runner](https://github.com/deepseek-ai/deepseek-coder/blob/main/Evaluation/MBPP/eval_instruct.py): eval scripts · Instruction and PAL evaluation entrypoints plus shell batch execution. · `Evaluation/MBPP/eval_instruct.py`
- [MBPP core](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/MBPP/human_eval): functional eval · Shared execution and correctness checking for MBPP generations. · `Evaluation/MBPP/human_eval`
- [LeetCode eval](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/LeetCode): batched benchmark · Separate evaluation flow optimized for larger-scale code generation. · `Evaluation/LeetCode`
- [PAL-Math assets](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/PAL-Math/datasets): datasets + prompts · Math reasoning datasets and prompt templates. · `Evaluation/PAL-Math/datasets`
- [PAL-Math run](https://github.com/deepseek-ai/deepseek-coder/blob/main/Evaluation/PAL-Math/run.py): orchestrator · Main control flow for prompt execution and benchmark runs. · `Evaluation/PAL-Math/run.py`
- [PAL-Math exec](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/PAL-Math/utils): parse-grade-exec · Parsing, execution, and grading of program-like model outputs. · `Evaluation/PAL-Math/utils`
- [DS-1000](https://github.com/deepseek-ai/deepseek-coder/tree/main/Evaluation/DS-1000): benchmark docs · Standalone benchmark area documented separately from the code-heavy suites. · `Evaluation/DS-1000`

### Fine-tuning

Instruction tuning pipeline for adapted checkpoints.

- **Train data**: instruction dataset · External instruction-following examples with instruction/output fields.
- [Fine-tune script](https://github.com/deepseek-ai/deepseek-coder/blob/main/finetune/finetune_deepseekcoder.py): trainer · Main supervised fine-tuning entrypoint for DeepSeek Coder. · `finetune/finetune_deepseekcoder.py`
- [ZeRO-3 config](https://github.com/deepseek-ai/deepseek-coder/blob/main/finetune/configs/ds_config_zero3.json): ds config · DeepSpeed configuration used to scale training. · `finetune/configs/ds_config_zero3.json`

### Demo

Small interactive front end for trying the model.

- [Interactive demo](https://github.com/deepseek-ai/deepseek-coder/blob/main/demo/app.py): ui wrapper · Thin local front end for interactive prompt and completion testing. · `demo/app.py`

### External runtime

Libraries and model assets used by the repo but not owned by it.

- **DeepSeek Coder**: external checkpoint · Pretrained model weights consumed by the repo, downloaded separately.
- **Transformers**: inference runtime · Primary Hugging Face model loading and generation path.
- **vLLM**: batch inference · High-throughput generation backend used by the LeetCode flow.
- **DeepSpeed**: training runtime · Distributed fine-tuning backend, including ZeRO-3 support.

## Connections

- DeepSeek Coder → Transformers: loads · Model weights are consumed through Hugging Face Transformers.
- DeepSeek Coder → vLLM: serves · The same checkpoints can be generated through vLLM.
- DeepSeek Coder → DeepSpeed: fine-tuned by · Training adapts the pretrained checkpoint using DeepSpeed.
- Eval hub → HumanEval data: uses · HumanEval evaluation starts from benchmark-local task files.
- Eval hub → HumanEval runner: runs · Runner scripts format prompts and launch evaluation jobs.
- HumanEval runner → HumanEval core: invokes · Generated solutions are checked through the shared execution core.
- HumanEval runner → Transformers: infers with · HumanEval uses standard model generation for completions.
- Eval hub → MBPP data: uses · MBPP evaluation starts from benchmark-local task files.
- Eval hub → MBPP runner: runs · Runner scripts format prompts and launch evaluation jobs.
- MBPP runner → MBPP core: invokes · Generated solutions are checked through the shared execution core.
- MBPP runner → Transformers: infers with · MBPP uses standard model generation for completions.
- LeetCode eval → vLLM: batched infer · LeetCode evaluation is optimized for high-throughput generation.
- LeetCode eval → Transformers: fallback infer · The benchmark flow still depends on model inference runtime.
- LeetCode eval → HumanEval core: reuses · LeetCode shares execution and evaluation utilities with code tasks.
- PAL-Math run → PAL-Math assets: loads · PAL-Math orchestrator consumes benchmark datasets and prompts.
- PAL-Math run → PAL-Math exec: executes · Program-like answers are parsed, executed, and graded.
- PAL-Math run → Transformers: queries · PAL-Math generates reasoning traces through the model runtime.
- Fine-tune script → Train data: trains on · Fine-tuning expects prepared instruction/output examples.
- Fine-tune script → ZeRO-3 config: configured by · Training scale and sharding are set by the ZeRO-3 config.
- Fine-tune script → DeepSpeed: uses · The trainer is built around DeepSpeed distributed training.
- Interactive demo → Transformers: queries · The demo is a thin UI over model inference.
- Interactive demo → DeepSeek Coder: shows · Interactive prompting uses the externally supplied checkpoint.
- Eval hub → DS-1000: documents · DS-1000 is present as a benchmark area with its own README.
