How deepseek-ai/deepep works
DeepEP is a GPU communication library for distributed ML, centered on MoE expert-parallel dispatch and combine. A model or training runtime calls the Python buffer API, which coordinates native buffers and JIT-compiled communication kernels to exchange tokens and return expert results. Other branches provide weight/gradient exchange, bucket collectives, Engram storage, and pipeline-parallel transfers. The graph also shows buffer planning and shared communication/runtime support. Some subsystem paths and capabilities are documented but their implementations were not excerpted; those relationships are marked as documented rather than inferred from unsampled code.
Main entry points: python_api.cpp Python bindings · __init__.py Public API · ep.py EP buffer · ep.hpp EP native buffer
Components 20
Python API
- Public API
deep_ep/__init__.py - Buffer allocator
deep_ep/buffers/allocator.py - Buffer lifecycle
deep_ep/buffers/base.py
Expert parallel
- EP buffer
deep_ep/buffers/ep.py - EP native buffer
csrc/buffers/ep.hpp - EP kernels
csrc/kernels/ep - Load-balance kernels
csrc/kernels/ep
Other collectives
- Bucket buffer
deep_ep/buffers/bucket.py - Bucket collectives
csrc/buffers/bucket.hpp - Bucket kernels
csrc/kernels/bucket - Engram buffer
deep_ep/buffers/engram.py - Engram operations
csrc/buffers/engram.hpp - PP buffer
deep_ep/buffers/pp.py - PP transfers
csrc/buffers/pp.hpp
Communication runtime
- Python bindings
csrc/python_api.cpp - Comm context
csrc/kernels/comm/context.cpp - Comm primitives
csrc/comm/barrier.hpp - JIT runtime
csrc/runtime/jit.hpp - Stream events
csrc/utils/event.hpp
Other components
- ML model runtime
Connections 29
- ML model runtime → Public API: calls
- Public API → EP buffer: exports
deep_ep/__init__.py - Public API → Bucket buffer: exports
deep_ep/__init__.py - Public API → Engram buffer: exports
deep_ep/__init__.py - Public API → PP buffer: exports
deep_ep/__init__.py - EP buffer → Buffer allocator: uses plans
deep_ep/buffers/ep.py - EP buffer → Buffer lifecycle: inherits lifecycle
deep_ep/buffers/ep.py - EP buffer → Python bindings: calls native API
deep_ep/buffers/ep.py - Python bindings → EP native buffer: registers
csrc/python_api.cpp - EP native buffer → EP kernels: launches
csrc/buffers/ep.hpp - EP native buffer → Load-balance kernels: supports weight exchange
csrc/buffers/ep.hpp - EP buffer → Stream events: coordinates waits
deep_ep/buffers/ep.py - Bucket buffer → Buffer lifecycle: inherits lifecycle
deep_ep/buffers/bucket.py - Bucket buffer → Python bindings: calls native API
- Python bindings → Bucket collectives: registers
csrc/python_api.cpp - Bucket collectives → Bucket kernels: launches collectives
csrc/buffers/bucket.hpp - Engram buffer → Buffer lifecycle: inherits lifecycle
deep_ep/buffers/engram.py - Engram buffer → Python bindings: calls native API
- Python bindings → Engram operations: registers
csrc/python_api.cpp - Engram operations → JIT runtime: uses JIT
csrc/buffers/engram.hpp - PP buffer → Buffer lifecycle: inherits lifecycle
deep_ep/buffers/pp.py - PP buffer → Python bindings: calls native API
- Python bindings → PP transfers: registers
csrc/python_api.cpp - PP transfers → Comm primitives: uses barriers
csrc/buffers/pp.hpp - EP native buffer → Comm context: uses context
csrc/buffers/ep.hpp - Bucket collectives → Comm primitives: uses barriers
csrc/buffers/bucket.hpp - Comm primitives → Stream events: orders streams
csrc/comm/barrier.hpp - Python bindings → JIT runtime: registers JIT API
csrc/python_api.cpp - JIT runtime → Comm context: initializes runtime
csrc/kernels/comm/context.cpp