# deepseek-ai/deepseek-v2 architecture

> Architecture diagram of the GitHub repository deepseek-ai/deepseek-v2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepseek-ai/deepseek-v2
- Repository: https://github.com/deepseek-ai/deepseek-v2

## Overview

##### Project type and purpose
- DeepSeek-V2 is an open-source mixture-of-experts language model repository.
- It provides pretrained model weights, inference code (via Huggingface, SGLang, vLLM), an OpenAI-compatible API platform, and a web-based chat interface.

##### File structure
- Top-level files:
– README.md: central documentation
– LICENSE-CODE, LICENSE-MODEL: code and model licenses
– deepseek-v2-tech-report.pdf: technical paper
- No typical “src” folder: code is provided via remote Huggingface trust-remote-code, SGLang packages, vLLM integrations.

##### README for architectural insights
- Sections on “Model Architecture,” “Chat Website,” “API Platform,” and “How to run locally.”
- Highlights two key subsystems: training pipeline (pretraining, SFT, RL) and inference pipeline (Huggingface Transformers, SGLang, vLLM).
- External integrations: Huggingface repo, DeepSeek API platform, chat.deepseek.com website.

##### Main components
a. Model Development Pipeline (offline)
– Data storage (8.1T token corpus)
– Pretraining cluster (GPU/TPU farm)
– Supervised fine-tuning (SFT) job
– Reinforcement learning (RLHF) job
– Output: DeepSeek-V2 Base & Chat model artifacts
b. Model Repository
– Huggingface Model Hub (deepseek-ai/DeepSeek-V2)
– License files (CODE & MODEL)
– Figures & Tech-report PDF
c. Inference Engines (online)
– Huggingface Transformers path
– SGLang service path (tokenizer, MLA-optimized runtime, FP8 options)
– vLLM service path (tensor parallels, sampling params)
d. API Platform
– DeepSeek RESTful API server (OpenAI-compatible endpoints)
– Backend inference cluster (serving via SGLang or vLLM)
– Authentication & rate limiting
e. Chat Website
– Frontend React/Vue.js client (chat.deepseek.com)
– Calls DeepSeek API Platform
f. Clients & Integrations
– Python SDK & code snippets (HuggingFace transformers example)
– LangChain integration
– Direct gRPC/HTTP clients

##### Relationships and data flows
- Offline training pipeline produces model artifacts and pushes them to the Model Hub.
- Inference engines pull models from the Model Hub into GPU clusters.
- API Platform routes client requests (web, SDK, LangChain) to inference engines.
- Chat Website interacts with the API Platform.
- Clients can bypass website and call the API Platform directly.

##### Architectural patterns and principles
- Microservice separation:
– Training jobs vs. inference services vs. web frontend vs. API gateway
- Layered architecture:
– Data layer (model storage), service layer (inference engines), presentation layer (web, SDK)
- Pluggable inference runtimes: interchangeable Huggingface / SGLang / vLLM

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Model Development Pipeline
    subgraph "Model Development Pipeline"
        direction TB
        DataLake["Data Lake\n(8.1T token corpus)"]:::offline
        Pretrain["Pretraining Cluster\n(GPU/TPU farm)"]:::offline
        SFT["Supervised Fine-Tuning Job"]:::offline
        RLHF["Reinforcement Learning (RLHF) Job"]:::offline
        Artifacts["DeepSeek-V2 Model Artifacts\n(Base & Chat)"]:::offline
        DataLake -->|ingest data| Pretrain -->|produce checkpoint| SFT -->|produce SFT model| RLHF -->|produce chat model| Artifacts
    end

    %% Model Repository
    subgraph "Model Repository"
        direction TB
        ModelHub["Huggingface Model Hub\n(deepseek-ai/DeepSeek-V2)"]:::storage
        LICENSECODE["LICENSE-CODE"]:::storage
        LICENSEMODEL["LICENSE-MODEL"]:::storage
        TechReport["deepseek-v2-tech-report.pdf"]:::storage
        Figures["figures/"]:::storage
        README["README.md"]:::storage
        Artifacts -->|push models| ModelHub
        ModelHub --> LICENSECODE
        ModelHub --> LICENSEMODEL
        ModelHub --> TechReport
        ModelHub --> Figures
        ModelHub --> README
    end

    %% Inference Engines
    subgraph "Inference Engines"
        direction TB
        HF["Huggingface Transformers Service"]:::inference
        SGLang["SGLang Service\n(tokenizer, MLA runtime)"]:::inference
        vLLM["vLLM Service\n(tensor parallel, sampling)"]:::inference
        ModelHub -->|download weights| HF
        ModelHub -->|download weights| SGLang
        ModelHub -->|download weights| vLLM
    end

    %% API & Web Layer
    subgraph "API & Web Layer"
        direction TB
        APIServer["DeepSeek RESTful API Server\n(OpenAI-compatible)"]:::frontend
        Auth["Authentication & Rate Limiting"]:::frontend
        InferenceCluster["Backend Inference Cluster"]:::frontend
        ChatUI["Chat Website\n(chat.deepseek.com)"]:::frontend
        APIServer --> Auth --> InferenceCluster
        InferenceCluster --> HF
        InferenceCluster --> SGLang
        InferenceCluster --> vLLM
        ChatUI -->|HTTP REST call| APIServer
    end

    %% Clients & Integrations
    subgraph "Clients & Integrations"
        direction TB
        PythonSDK["Python SDK & Code Snippets"]:::client
        LangChain["LangChain Integration"]:::client
        DirectClient["Direct gRPC/HTTP Client"]:::client
        PythonSDK -->|REST/gRPC| APIServer
        LangChain -->|REST/gRPC| APIServer
        DirectClient -->|REST/gRPC| APIServer
    end

    %% Click Events
    click README "https://github.com/deepseek-ai/deepseek-v2/blob/main/README.md"
    click LICENSECODE "https://github.com/deepseek-ai/deepseek-v2/tree/main/LICENSE-CODE"
    click LICENSEMODEL "https://github.com/deepseek-ai/deepseek-v2/tree/main/LICENSE-MODEL"
    click TechReport "https://github.com/deepseek-ai/deepseek-v2/blob/main/deepseek-v2-tech-report.pdf"
    click Figures "https://github.com/deepseek-ai/deepseek-v2/tree/main/figures/"

    %% Styles
    classDef offline fill:#AEDFF7,stroke:#000,stroke-width:1px
    classDef inference fill:#C8E6C9,stroke:#000,stroke-width:1px
    classDef frontend fill:#FFE0B2,stroke:#000,stroke-width:1px
    classDef storage fill:#EEEEEE,stroke:#000,stroke-width:1px
    classDef client fill:#F0F4C3,stroke:#000,stroke-width:1px
```
