How miaai-lab/deepseek-v4.1-flash-dgx-sparks works
This repository is a deployment-specific DeepSeek-V4.1-Flash serving profile: it adapts SGLang to run across three or four DGX Sparks, optimizing weight loading, prefill, speculative decode, interconnect use, and NVMe-backed Engram storage. The main user path is an OpenAI-compatible chat request through the serving engine to a generated response; quality-evaluation scripts exercise that endpoint. The README documents several TP4 capabilities whose implementation is not fully sampled, so those are represented at subsystem level. Edges are limited to README-documented flows and installation calls visible in the excerpts.
Main entry points: sitecustomize.py Adapter installer · engram_backend.py Engram row backend · verify_cap.py Speculative verification · qeval.py Endpoint quality gate
Components 22
Serving runtime
- Chat endpoint
- SGLang engine
- Runtime bootstrap
boot.py - Adapter installer
adapter/sitecustomize.py
Model execution
- Fast weight loader
adapter/fast_load.py - TP3 shard padding
adapter/tp3_pad.py - Prefill sequence parallel
adapter/prefill_sp.py - Chunked prefill indexer
adapter/indexer_chunked_v3.py - Dense prefill kernels
adapter/spark_prefill_dense.py - Routed MoE kernels
adapter/moe_b12x_next.py - MXFP8 kernels
adapter/mxfp8_b12x.py - Shared expert padding
adapter/shared_pad_k.py - Speculative verification
adapter/verify_cap.py - Live router kernel
adapter/router_live.py - Draft head adapters
adapter/draft_head_fp8.py
Memory and storage
- Engram row backend
adapter/engram_backend.py - Engram prefetch
adapter/engram_prefetch.py - Native row store
adapter/row_store.cpp - Engram shard packing
scripts/pack_engram.py
Quality evaluation
- Endpoint quality gate
scripts/qeval.py - Scored task set
scripts/qeval_tasks.py
Other components
- API client
Connections 23
- API client → Chat endpoint: sends requests
- Chat endpoint → SGLang engine: serves through
- Runtime bootstrap → SGLang engine: starts engine
- Adapter installer → Engram row backend: installs backend
adapter/sitecustomize.py - Adapter installer → MXFP8 kernels: installs kernels
adapter/sitecustomize.py - Adapter installer → Shared expert padding: installs padding
adapter/sitecustomize.py - Adapter installer → Prefill sequence parallel: installs parallelism
adapter/sitecustomize.py - Adapter installer → Chunked prefill indexer: installs indexer
adapter/sitecustomize.py - Adapter installer → Speculative verification: installs verification
adapter/sitecustomize.py - Adapter installer → Fast weight loader: installs loader
adapter/sitecustomize.py - Adapter installer → Routed MoE kernels: installs MoE
adapter/sitecustomize.py - Adapter installer → Live router kernel: installs router
- SGLang engine → Fast weight loader: loads weights
- SGLang engine → TP3 shard padding: pads TP shards
- SGLang engine → Dense prefill kernels: runs prefill
- SGLang engine → Routed MoE kernels: runs routed experts
- SGLang engine → Speculative verification: verifies drafts
- Speculative verification → Live router kernel: uses live routing
adapter/verify_cap.py - Engram row backend → Native row store: reads rows
- Engram prefetch → Engram row backend: prefetches rows
adapter/engram_prefetch.py - Engram shard packing → Engram row backend: packs shards
- Endpoint quality gate → Chat endpoint: scores responses
scripts/qeval.py - Endpoint quality gate → Scored task set: loads tasks
scripts/qeval.py