How deepseek-ai/deepseek-v2 works
Project type and purpose
• DeepSeek-V2 is an open-source mixture-of-experts language model repository.
• It provides pretrained model weights, inference code (via Huggingface, SGLang, vLLM), an OpenAI-compatible API platform, and a web-based chat interface.
File structure
• Top-level files:
– README.md: central documentation
– LICENSE-CODE, LICENSE-MODEL: code and model licenses
– deepseek-v2-tech-report.pdf: technical paper
• No typical “src” folder: code is provided via remote Huggingface trust-remote-code, SGLang packages, vLLM integrations.
README for architectural insights
• Sections on “Model Architecture,” “Chat Website,” “API Platform,” and “How to run locally.”
• Highlights two key subsystems: training pipeline (pretraining, SFT, RL) and inference pipeline (Huggingface Transformers, SGLang, vLLM).
• External integrations: Huggingface repo, DeepSeek API platform, chat.deepseek.com website.
Main components
a. Model Development Pipeline (offline)
– Data storage (8.1T token corpus)
– Pretraining cluster (GPU/TPU farm)
– Supervised fine-tuning (SFT) job
– Reinforcement learning (RLHF) job
– Output: DeepSeek-V2 Base & Chat model artifacts
b. Model Repository
– Huggingface Model Hub (deepseek-ai/DeepSeek-V2)
– License files (CODE & MODEL)
– Figures & Tech-report PDF
c. Inference Engines (online)
– Huggingface Transformers path
– SGLang service path (tokenizer, MLA-optimized runtime, FP8 options)
– vLLM service path (tensor parallels, sampling params)
d. API Platform
– DeepSeek RESTful API server (OpenAI-compatible endpoints)
– Backend inference cluster (serving via SGLang or vLLM)
– Authentication & rate limiting
e. Chat Website
– Frontend React/Vue.js client (chat.deepseek.com)
– Calls DeepSeek API Platform
f. Clients & Integrations
– Python SDK & code snippets (HuggingFace transformers example)
– LangChain integration
– Direct gRPC/HTTP clients
Relationships and data flows
• Offline training pipeline produces model artifacts and pushes them to the Model Hub.
• Inference engines pull models from the Model Hub into GPU clusters.
• API Platform routes client requests (web, SDK, LangChain) to inference engines.
• Chat Website interacts with the API Platform.
• Clients can bypass website and call the API Platform directly.
Architectural patterns and principles
• Microservice separation:
– Training jobs vs. inference services vs. web frontend vs. API gateway
• Layered architecture:
– Data layer (model storage), service layer (inference engines), presentation layer (web, SDK)
• Pluggable inference runtimes: interchangeable Huggingface / SGLang / vLLM