# miaai-lab/deepseek-v4.1-flash-dgx-sparks architecture

> Architecture diagram of the GitHub repository miaai-lab/deepseek-v4.1-flash-dgx-sparks, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks
- Repository: https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks

## Overview

This repository is a deployment-specific DeepSeek-V4.1-Flash serving profile: it adapts SGLang to run across three or four DGX Sparks, optimizing weight loading, prefill, speculative decode, interconnect use, and NVMe-backed Engram storage. The main user path is an OpenAI-compatible chat request through the serving engine to a generated response; quality-evaluation scripts exercise that endpoint. The README documents several TP4 capabilities whose implementation is not fully sampled, so those are represented at subsystem level. Edges are limited to README-documented flows and installation calls visible in the excerpts.

Main entry points: [`adapter/sitecustomize.py`](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/sitecustomize.py) (Adapter installer), [`adapter/engram_backend.py`](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/engram_backend.py) (Engram row backend), [`adapter/verify_cap.py`](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/verify_cap.py) (Speculative verification), [`scripts/qeval.py`](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/scripts/qeval.py) (Endpoint quality gate)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serving["Serving runtime"]
  node_endpoint["Chat endpoint"]
  node_sglang["SGLang engine"]
  node_boot["Runtime bootstrap<br/>[boot.py]"]
  node_patch_install["Adapter installer<br/>[sitecustomize.py]"]
end

subgraph group_model["Model execution"]
  node_weight_loader["Fast weight loader<br/>[fast_load.py]"]
  node_tp_padding["TP3 shard padding<br/>[tp3_pad.py]"]
  node_prefill_parallel["Prefill sequence parallel<br/>[prefill_sp.py]"]
  node_indexer["Chunked prefill indexer"]
  node_dense_prefill["Dense prefill kernels"]
  node_moe["Routed MoE kernels<br/>[moe_b12x_next.py]"]
  node_mxfp8["MXFP8 kernels<br/>[mxfp8_b12x.py]"]
  node_shared_pad["Shared expert padding<br/>[shared_pad_k.py]"]
  node_spec_verify["Speculative verification<br/>[verify_cap.py]"]
  node_router["Live router kernel<br/>[router_live.py]"]
  node_draft_heads["Draft head adapters<br/>[draft_head_fp8.py]"]
end

subgraph group_memory["Memory and storage"]
  node_engram[("Engram row backend<br/>[engram_backend.py]")]
  node_engram_prefetch["Engram prefetch<br/>[engram_prefetch.py]"]
  node_row_store["Native row store<br/>[row_store.cpp]"]
  node_pack_engram["Engram shard packing<br/>[pack_engram.py]"]
end

subgraph group_evaluation["Quality evaluation"]
  node_qeval["Endpoint quality gate<br/>[qeval.py]"]
  node_tasks["Scored task set<br/>[qeval_tasks.py]"]
end

node_client(("API client"))

node_client -->|"sends requests"| node_endpoint
node_endpoint -->|"serves through"| node_sglang
node_boot -.->|"starts engine"| node_sglang
node_patch_install -->|"installs backend"| node_engram
node_patch_install -->|"installs kernels"| node_mxfp8
node_patch_install -->|"installs padding"| node_shared_pad
node_patch_install -->|"installs parallelism"| node_prefill_parallel
node_patch_install -->|"installs indexer"| node_indexer
node_patch_install -->|"installs verification"| node_spec_verify
node_patch_install -.->|"installs loader"| node_weight_loader
node_patch_install -.->|"installs MoE"| node_moe
node_patch_install -.->|"installs router"| node_router
node_sglang -.->|"loads weights"| node_weight_loader
node_sglang -.->|"pads TP shards"| node_tp_padding
node_sglang -.->|"runs prefill"| node_dense_prefill
node_sglang -.->|"runs routed experts"| node_moe
node_sglang -.->|"verifies drafts"| node_spec_verify
node_spec_verify -->|"uses live routing"| node_router
node_engram -.->|"reads rows"| node_row_store
node_engram_prefetch -.->|"prefetches rows"| node_engram
node_pack_engram -.->|"packs shards"| node_engram
node_qeval -->|"scores responses"| node_endpoint
node_qeval -.->|"loads tasks"| node_tasks

click node_boot "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/boot.py"
click node_patch_install "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/sitecustomize.py"
click node_weight_loader "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/fast_load.py"
click node_tp_padding "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/tp3_pad.py"
click node_prefill_parallel "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/prefill_sp.py"
click node_indexer "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/indexer_chunked_v3.py"
click node_dense_prefill "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/spark_prefill_dense.py"
click node_moe "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/moe_b12x_next.py"
click node_mxfp8 "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/mxfp8_b12x.py"
click node_shared_pad "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/shared_pad_k.py"
click node_spec_verify "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/verify_cap.py"
click node_router "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/router_live.py"
click node_draft_heads "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/draft_head_fp8.py"
click node_engram "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/engram_backend.py"
click node_engram_prefetch "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/engram_prefetch.py"
click node_row_store "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/row_store.cpp"
click node_pack_engram "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/scripts/pack_engram.py"
click node_qeval "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/scripts/qeval.py"
click node_tasks "https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/scripts/qeval_tasks.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_endpoint,node_sglang,node_boot,node_patch_install,node_client toneBlue
class node_weight_loader,node_tp_padding,node_prefill_parallel,node_indexer,node_dense_prefill,node_moe,node_mxfp8,node_shared_pad,node_spec_verify,node_router,node_draft_heads toneAmber
class node_engram,node_engram_prefetch,node_row_store,node_pack_engram toneMint
class node_qeval,node_tasks toneRose
```

## Components

### Serving runtime

- **Chat endpoint**
- **SGLang engine**
- [Runtime bootstrap](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/boot.py): `boot.py`
- [Adapter installer](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/sitecustomize.py): `adapter/sitecustomize.py`

### Model execution

- [Fast weight loader](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/fast_load.py): `adapter/fast_load.py`
- [TP3 shard padding](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/tp3_pad.py): `adapter/tp3_pad.py`
- [Prefill sequence parallel](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/prefill_sp.py): `adapter/prefill_sp.py`
- [Chunked prefill indexer](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/indexer_chunked_v3.py): `adapter/indexer_chunked_v3.py`
- [Dense prefill kernels](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/spark_prefill_dense.py): `adapter/spark_prefill_dense.py`
- [Routed MoE kernels](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/moe_b12x_next.py): `adapter/moe_b12x_next.py`
- [MXFP8 kernels](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/mxfp8_b12x.py): `adapter/mxfp8_b12x.py`
- [Shared expert padding](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/shared_pad_k.py): `adapter/shared_pad_k.py`
- [Speculative verification](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/verify_cap.py): `adapter/verify_cap.py`
- [Live router kernel](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/router_live.py): `adapter/router_live.py`
- [Draft head adapters](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/draft_head_fp8.py): `adapter/draft_head_fp8.py`

### Memory and storage

- [Engram row backend](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/engram_backend.py): `adapter/engram_backend.py`
- [Engram prefetch](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/engram_prefetch.py): `adapter/engram_prefetch.py`
- [Native row store](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/adapter/row_store.cpp): `adapter/row_store.cpp`
- [Engram shard packing](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/scripts/pack_engram.py): `scripts/pack_engram.py`

### Quality evaluation

- [Endpoint quality gate](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/scripts/qeval.py): `scripts/qeval.py`
- [Scored task set](https://github.com/miaai-lab/deepseek-v4.1-flash-dgx-sparks/blob/main/scripts/qeval_tasks.py): `scripts/qeval_tasks.py`

### Other components

- **API client**

## Connections

- API client → Chat endpoint: sends requests
- Chat endpoint → SGLang engine: serves through
- Runtime bootstrap → SGLang engine: starts engine
- Adapter installer → Engram row backend: installs backend (evidence: `adapter/sitecustomize.py`)
- Adapter installer → MXFP8 kernels: installs kernels (evidence: `adapter/sitecustomize.py`)
- Adapter installer → Shared expert padding: installs padding (evidence: `adapter/sitecustomize.py`)
- Adapter installer → Prefill sequence parallel: installs parallelism (evidence: `adapter/sitecustomize.py`)
- Adapter installer → Chunked prefill indexer: installs indexer (evidence: `adapter/sitecustomize.py`)
- Adapter installer → Speculative verification: installs verification (evidence: `adapter/sitecustomize.py`)
- Adapter installer → Fast weight loader: installs loader (evidence: `adapter/sitecustomize.py`)
- Adapter installer → Routed MoE kernels: installs MoE (evidence: `adapter/sitecustomize.py`)
- Adapter installer → Live router kernel: installs router
- SGLang engine → Fast weight loader: loads weights
- SGLang engine → TP3 shard padding: pads TP shards
- SGLang engine → Dense prefill kernels: runs prefill
- SGLang engine → Routed MoE kernels: runs routed experts
- SGLang engine → Speculative verification: verifies drafts
- Speculative verification → Live router kernel: uses live routing (evidence: `adapter/verify_cap.py`)
- Engram row backend → Native row store: reads rows
- Engram prefetch → Engram row backend: prefetches rows (evidence: `adapter/engram_prefetch.py`)
- Engram shard packing → Engram row backend: packs shards
- Endpoint quality gate → Chat endpoint: scores responses (evidence: `scripts/qeval.py`)
- Endpoint quality gate → Scored task set: loads tasks (evidence: `scripts/qeval.py`)