# deepseek-ai/deepseek-v3 architecture

> Architecture diagram of the GitHub repository deepseek-ai/deepseek-v3, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepseek-ai/deepseek-v3
- Repository: https://github.com/deepseek-ai/deepseek-v3

## Overview

This repository provides a local inference and checkpoint-preparation toolkit for DeepSeek-V3. The main workflow loads a configured Transformer and checkpoint, tokenizes interactive or batch prompts, generates and decodes completions, and returns them to the user. Separate utilities shard Hugging Face checkpoints for model-parallel inference and convert FP8 weights to BF16. The excerpts show the generator-to-model and conversion-to-kernel dependencies; other model internals and README-described capabilities are represented at a higher level where implementation details were not sampled.

Main entry points: [`inference/generate.py`](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/generate.py) (Text generation), [`inference/fp8_cast_bf16.py`](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/fp8_cast_bf16.py) (FP8 to BF16), [`inference/model.py`](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/model.py) (Transformer model), [`inference/convert.py`](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/convert.py) (Checkpoint sharding)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Inference runtime"]
  node_generation["Text generation<br/>[generate.py]"]
  node_tokenizer["Tokenizer"]
  node_config["Model configs"]
end

subgraph group_model["Model execution"]
  node_transformer["Transformer model<br/>[model.py]"]
  node_kernels["Triton kernels<br/>[kernel.py]"]
end

subgraph group_weights["Weight preparation"]
  node_checkpoint_sharding["Checkpoint sharding<br/>[convert.py]"]
  node_fp8_conversion["FP8 to BF16<br/>[fp8_cast_bf16.py]"]
end

node_user(("Prompt user"))
node_checkpoint[("Model checkpoint")]
node_completion["Generated text"]
node_source_weights[("Hugging Face weights")]
node_prepared_weights[("Prepared weights")]

node_user -->|"submits prompts"| node_generation
node_generation -->|"reads settings"| node_config
node_generation -->|"tokenizes prompts"| node_tokenizer
node_generation -->|"loads weights"| node_checkpoint
node_generation -->|"runs inference"| node_transformer
node_transformer -->|"uses kernels"| node_kernels
node_generation -->|"produces text"| node_completion
node_completion -->|"returns result"| node_user
node_source_weights -->|"provides weights"| node_checkpoint_sharding
node_checkpoint_sharding -->|"writes shards"| node_prepared_weights
node_source_weights -->|"provides FP8 weights"| node_fp8_conversion
node_fp8_conversion -->|"dequantizes with"| node_kernels
node_fp8_conversion -->|"writes BF16 weights"| node_prepared_weights

click node_generation "https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/generate.py"
click node_config "https://github.com/deepseek-ai/deepseek-v3/tree/main/inference/configs"
click node_transformer "https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/model.py"
click node_kernels "https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/kernel.py"
click node_checkpoint_sharding "https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/convert.py"
click node_fp8_conversion "https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/fp8_cast_bf16.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_generation,node_tokenizer,node_config,node_user toneBlue
class node_transformer,node_kernels,node_checkpoint,node_source_weights,node_prepared_weights toneAmber
class node_checkpoint_sharding,node_fp8_conversion toneMint
class node_completion toneIndigo
```

## Components

### Inference runtime

- [Text generation](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/generate.py): `inference/generate.py`
- **Tokenizer**
- [Model configs](https://github.com/deepseek-ai/deepseek-v3/tree/main/inference/configs): `inference/configs`

### Model execution

- [Transformer model](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/model.py): `inference/model.py`
- [Triton kernels](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/kernel.py): `inference/kernel.py`

### Weight preparation

- [Checkpoint sharding](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/convert.py): `inference/convert.py`
- [FP8 to BF16](https://github.com/deepseek-ai/deepseek-v3/blob/main/inference/fp8_cast_bf16.py): `inference/fp8_cast_bf16.py`

### Other components

- **Prompt user**
- **Model checkpoint**
- **Generated text**
- **Hugging Face weights**
- **Prepared weights**

## Connections

- Prompt user → Text generation: submits prompts (evidence: `inference/generate.py`)
- Text generation → Model configs: reads settings (evidence: `inference/generate.py`)
- Text generation → Tokenizer: tokenizes prompts (evidence: `inference/generate.py`)
- Text generation → Model checkpoint: loads weights (evidence: `inference/generate.py`)
- Text generation → Transformer model: runs inference (evidence: `inference/generate.py`)
- Transformer model → Triton kernels: uses kernels (evidence: `inference/model.py`)
- Text generation → Generated text: produces text (evidence: `inference/generate.py`)
- Generated text → Prompt user: returns result
- Hugging Face weights → Checkpoint sharding: provides weights (evidence: `inference/convert.py`)
- Checkpoint sharding → Prepared weights: writes shards (evidence: `inference/convert.py`)
- Hugging Face weights → FP8 to BF16: provides FP8 weights (evidence: `inference/fp8_cast_bf16.py`)
- FP8 to BF16 → Triton kernels: dequantizes with (evidence: `inference/fp8_cast_bf16.py`)
- FP8 to BF16 → Prepared weights: writes BF16 weights (evidence: `inference/fp8_cast_bf16.py`)
