# deepseek-ai/deepep architecture

> Architecture diagram of the GitHub repository deepseek-ai/deepep, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/deepseek-ai/deepep
- Repository: https://github.com/deepseek-ai/deepep

## Overview

DeepEP is a GPU communication library for distributed ML, centered on MoE expert-parallel dispatch and combine. A model or training runtime calls the Python buffer API, which coordinates native buffers and JIT-compiled communication kernels to exchange tokens and return expert results. Other branches provide weight/gradient exchange, bucket collectives, Engram storage, and pipeline-parallel transfers. The graph also shows buffer planning and shared communication/runtime support. Some subsystem paths and capabilities are documented but their implementations were not excerpted; those relationships are marked as documented rather than inferred from unsampled code.

Main entry points: [`csrc/python_api.cpp`](https://github.com/deepseek-ai/deepep/blob/main/csrc/python_api.cpp) (Python bindings), [`deep_ep/__init__.py`](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/__init__.py) (Public API), [`deep_ep/buffers/ep.py`](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/ep.py) (EP buffer), [`csrc/buffers/ep.hpp`](https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/ep.hpp) (EP native buffer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Python API"]
  node_public_api["Public API<br/>[__init__.py]"]
  node_allocator["Buffer allocator<br/>[allocator.py]"]
  node_buffer_base["Buffer lifecycle<br/>[base.py]"]
end

subgraph group_ep["Expert parallel"]
  node_ep_buffer["EP buffer<br/>[ep.py]"]
  node_ep_native["EP native buffer<br/>[ep.hpp]"]
  node_ep_kernels["EP kernels"]
  node_lb_kernels["Load-balance kernels"]
end

subgraph group_parallel["Other collectives"]
  node_bucket_buffer["Bucket buffer<br/>[bucket.py]"]
  node_bucket_native["Bucket collectives<br/>[bucket.hpp]"]
  node_bucket_kernels["Bucket kernels"]
  node_engram_buffer["Engram buffer<br/>[engram.py]"]
  node_engram_native["Engram operations<br/>[engram.hpp]"]
  node_pp_buffer["PP buffer<br/>[pp.py]"]
  node_pp_native["PP transfers<br/>[pp.hpp]"]
end

subgraph group_runtime["Communication runtime"]
  node_python_binding["Python bindings<br/>[python_api.cpp]"]
  node_comm_context["Comm context<br/>[context.cpp]"]
  node_comm_api["Comm primitives<br/>[barrier.hpp]"]
  node_jit["JIT runtime<br/>[jit.hpp]"]
  node_events["Stream events<br/>[event.hpp]"]
end

node_model(("ML model runtime"))

node_model -->|"calls"| node_public_api
node_public_api -->|"exports"| node_ep_buffer
node_public_api -->|"exports"| node_bucket_buffer
node_public_api -->|"exports"| node_engram_buffer
node_public_api -->|"exports"| node_pp_buffer
node_ep_buffer -->|"uses plans"| node_allocator
node_ep_buffer -->|"inherits lifecycle"| node_buffer_base
node_ep_buffer -->|"calls native API"| node_python_binding
node_python_binding -->|"registers"| node_ep_native
node_ep_native -->|"launches"| node_ep_kernels
node_ep_native -.->|"supports weight exchange"| node_lb_kernels
node_ep_buffer -->|"coordinates waits"| node_events
node_bucket_buffer -->|"inherits lifecycle"| node_buffer_base
node_bucket_buffer -.->|"calls native API"| node_python_binding
node_python_binding -->|"registers"| node_bucket_native
node_bucket_native -->|"launches collectives"| node_bucket_kernels
node_engram_buffer -->|"inherits lifecycle"| node_buffer_base
node_engram_buffer -.->|"calls native API"| node_python_binding
node_python_binding -->|"registers"| node_engram_native
node_engram_native -->|"uses JIT"| node_jit
node_pp_buffer -->|"inherits lifecycle"| node_buffer_base
node_pp_buffer -.->|"calls native API"| node_python_binding
node_python_binding -->|"registers"| node_pp_native
node_pp_native -->|"uses barriers"| node_comm_api
node_ep_native -->|"uses context"| node_comm_context
node_bucket_native -->|"uses barriers"| node_comm_api
node_comm_api -->|"orders streams"| node_events
node_python_binding -->|"registers JIT API"| node_jit
node_jit -->|"initializes runtime"| node_comm_context

click node_public_api "https://github.com/deepseek-ai/deepep/blob/main/deep_ep/__init__.py"
click node_allocator "https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/allocator.py"
click node_buffer_base "https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/base.py"
click node_ep_buffer "https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/ep.py"
click node_ep_native "https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/ep.hpp"
click node_ep_kernels "https://github.com/deepseek-ai/deepep/tree/main/csrc/kernels/ep"
click node_lb_kernels "https://github.com/deepseek-ai/deepep/tree/main/csrc/kernels/ep"
click node_bucket_buffer "https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/bucket.py"
click node_bucket_native "https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/bucket.hpp"
click node_bucket_kernels "https://github.com/deepseek-ai/deepep/tree/main/csrc/kernels/bucket"
click node_engram_buffer "https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/engram.py"
click node_engram_native "https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/engram.hpp"
click node_pp_buffer "https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/pp.py"
click node_pp_native "https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/pp.hpp"
click node_python_binding "https://github.com/deepseek-ai/deepep/blob/main/csrc/python_api.cpp"
click node_comm_context "https://github.com/deepseek-ai/deepep/blob/main/csrc/kernels/comm/context.cpp"
click node_comm_api "https://github.com/deepseek-ai/deepep/blob/main/csrc/comm/barrier.hpp"
click node_jit "https://github.com/deepseek-ai/deepep/blob/main/csrc/runtime/jit.hpp"
click node_events "https://github.com/deepseek-ai/deepep/blob/main/csrc/utils/event.hpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_public_api,node_allocator,node_buffer_base toneBlue
class node_ep_buffer,node_ep_native,node_ep_kernels,node_lb_kernels toneAmber
class node_bucket_buffer,node_bucket_native,node_bucket_kernels,node_engram_buffer,node_engram_native,node_pp_buffer,node_pp_native toneMint
class node_python_binding,node_comm_context,node_comm_api,node_jit,node_events toneRose
class node_model toneIndigo
```

## Components

### Python API

- [Public API](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/__init__.py): `deep_ep/__init__.py`
- [Buffer allocator](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/allocator.py): `deep_ep/buffers/allocator.py`
- [Buffer lifecycle](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/base.py): `deep_ep/buffers/base.py`

### Expert parallel

- [EP buffer](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/ep.py): `deep_ep/buffers/ep.py`
- [EP native buffer](https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/ep.hpp): `csrc/buffers/ep.hpp`
- [EP kernels](https://github.com/deepseek-ai/deepep/tree/main/csrc/kernels/ep): `csrc/kernels/ep`
- [Load-balance kernels](https://github.com/deepseek-ai/deepep/tree/main/csrc/kernels/ep): `csrc/kernels/ep`

### Other collectives

- [Bucket buffer](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/bucket.py): `deep_ep/buffers/bucket.py`
- [Bucket collectives](https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/bucket.hpp): `csrc/buffers/bucket.hpp`
- [Bucket kernels](https://github.com/deepseek-ai/deepep/tree/main/csrc/kernels/bucket): `csrc/kernels/bucket`
- [Engram buffer](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/engram.py): `deep_ep/buffers/engram.py`
- [Engram operations](https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/engram.hpp): `csrc/buffers/engram.hpp`
- [PP buffer](https://github.com/deepseek-ai/deepep/blob/main/deep_ep/buffers/pp.py): `deep_ep/buffers/pp.py`
- [PP transfers](https://github.com/deepseek-ai/deepep/blob/main/csrc/buffers/pp.hpp): `csrc/buffers/pp.hpp`

### Communication runtime

- [Python bindings](https://github.com/deepseek-ai/deepep/blob/main/csrc/python_api.cpp): `csrc/python_api.cpp`
- [Comm context](https://github.com/deepseek-ai/deepep/blob/main/csrc/kernels/comm/context.cpp): `csrc/kernels/comm/context.cpp`
- [Comm primitives](https://github.com/deepseek-ai/deepep/blob/main/csrc/comm/barrier.hpp): `csrc/comm/barrier.hpp`
- [JIT runtime](https://github.com/deepseek-ai/deepep/blob/main/csrc/runtime/jit.hpp): `csrc/runtime/jit.hpp`
- [Stream events](https://github.com/deepseek-ai/deepep/blob/main/csrc/utils/event.hpp): `csrc/utils/event.hpp`

### Other components

- **ML model runtime**

## Connections

- ML model runtime → Public API: calls
- Public API → EP buffer: exports (evidence: `deep_ep/__init__.py`)
- Public API → Bucket buffer: exports (evidence: `deep_ep/__init__.py`)
- Public API → Engram buffer: exports (evidence: `deep_ep/__init__.py`)
- Public API → PP buffer: exports (evidence: `deep_ep/__init__.py`)
- EP buffer → Buffer allocator: uses plans (evidence: `deep_ep/buffers/ep.py`)
- EP buffer → Buffer lifecycle: inherits lifecycle (evidence: `deep_ep/buffers/ep.py`)
- EP buffer → Python bindings: calls native API (evidence: `deep_ep/buffers/ep.py`)
- Python bindings → EP native buffer: registers (evidence: `csrc/python_api.cpp`)
- EP native buffer → EP kernels: launches (evidence: `csrc/buffers/ep.hpp`)
- EP native buffer → Load-balance kernels: supports weight exchange (evidence: `csrc/buffers/ep.hpp`)
- EP buffer → Stream events: coordinates waits (evidence: `deep_ep/buffers/ep.py`)
- Bucket buffer → Buffer lifecycle: inherits lifecycle (evidence: `deep_ep/buffers/bucket.py`)
- Bucket buffer → Python bindings: calls native API
- Python bindings → Bucket collectives: registers (evidence: `csrc/python_api.cpp`)
- Bucket collectives → Bucket kernels: launches collectives (evidence: `csrc/buffers/bucket.hpp`)
- Engram buffer → Buffer lifecycle: inherits lifecycle (evidence: `deep_ep/buffers/engram.py`)
- Engram buffer → Python bindings: calls native API
- Python bindings → Engram operations: registers (evidence: `csrc/python_api.cpp`)
- Engram operations → JIT runtime: uses JIT (evidence: `csrc/buffers/engram.hpp`)
- PP buffer → Buffer lifecycle: inherits lifecycle (evidence: `deep_ep/buffers/pp.py`)
- PP buffer → Python bindings: calls native API
- Python bindings → PP transfers: registers (evidence: `csrc/python_api.cpp`)
- PP transfers → Comm primitives: uses barriers (evidence: `csrc/buffers/pp.hpp`)
- EP native buffer → Comm context: uses context (evidence: `csrc/buffers/ep.hpp`)
- Bucket collectives → Comm primitives: uses barriers (evidence: `csrc/buffers/bucket.hpp`)
- Comm primitives → Stream events: orders streams (evidence: `csrc/comm/barrier.hpp`)
- Python bindings → JIT runtime: registers JIT API (evidence: `csrc/python_api.cpp`)
- JIT runtime → Comm context: initializes runtime (evidence: `csrc/kernels/comm/context.cpp`)
