How scienceone-ai/deepseek-671b-sft-guide works
Project Type & Purpose
• This project is a specialized deep-learning training and deployment framework focused on full-parameter fine-tuning for a massive language model (DeepSeek-V3/R1 671B).
• Its purposes include: preparing and converting training data (with extended reasoning formats), executing distributed training with both data parallel (via DeepSpeed ZeRO) and sequence parallel (SP) strategies, and converting/tracing the final model weights into a Huggingface-compatible format for deployment (using vLLM over a Ray cluster).
Main Components to Represent (Each as a module
a. Data Preparation & Preprocessing Module
• Components: “data” folder (e.g., train_example.json sample), dataset conversion logic in code/xtuner/dataset, map functions (alpaca, arxiv, code_alpaca, etc.), and custom/preprocess scripts.
• Flow: Raw training data → Data converters/formatters → Training-ready tensor formats.
b. Model & Configuration Layer
• Components: "model" directory (DeepSeek-V3/R1 modeling files, custom modifications from DeepSeek-V2/V3 papers), configuration files under “code/xtuner/configs” for various run modes (e.g., zero/offload, parallel strategies), and modified “xtuner” core code.
• Flow: Configurations, custom modeling modules, datasets, and parameter settings feed into model instantiation.
c. Training Engine & Distributed Training Infrastructure
• Components: “code/xtuner/engine” (runner, hooks, parallel setups), integration with DeepSpeed (ZeRO configurations in code/xtuner/configs/deepspeed), and support for both data parallelism and sequence parallelism.
• Connection: Training scripts (e.g., sft_deepseek.py and corresponding shell scripts) activate the engine which uses the configured parallel strategies. Include scheduler integration (slurm, pdsh) and environment variable setups.
• Additional Details: Model modules in “code/xtuner/model” define the training logic; engine hooks (e.g., dataset info, evaluation, checkpointing) manage runtime behavior and logging.
d. Checkpointing & Intermediate Storage
• Components: Intermediate model weights in “pth” format (noted in training – each checkpoint ~7.4TB), conversion scripts under “code/scripts/convert_pth_to_hf.sh”.
• Flow: Conversion of raw pth weights to Huggingface model format to support subsequent deployment.
e. Deployment & Inference Layer
• Components: vLLM deployment scripts, Ray cluster orchestration (ray start commands in vllm_deploy_slurm.sh), and integration with Huggingface Transformers.
• Flow: Final converted model → vLLM serving (using ray cluster with tensor-/pipeline-parallel size settings) → Client APIs (curl example).
Relationships and Data Flow
which then feeds training-ready inputs into the Training Engine.
• The Training Engine interacts with model instantiation (from the Model & Configuration layer) and uses distributed parallel strategies.
• After model training, checkpoint outputs pass to the Checkpoint Conversion module.
• The final model is deployed by the Inference engine (vLLM deployment), and client requests flow from external API calls into the serving system.
• That underlying libraries (DeepSpeed, Huggingface transformers, xtuner framework, and Ray) form the technology backbone in various layers.
Architectural Design Patterns & Technologies
• The architecture follows a layered design with clear separation between data preprocessing, training (with distributed strategies), and deployment.
• It employs modularity (each “config” folder for various parallel and fine-tuning setups) and plug-in patterns (for dataset mapping functions, training hooks, and model converters).
• The design also leverages distributed computing patterns (data parallelism via DeepSpeed ZeRO offload and sequence parallelism) along with inter-node communication using SLURM/PDSH for orchestration.
• Technologies and frameworks: Python v3.10, DeepSpeed, Ray, vLLM, Huggingface Transformers, and xtuner.