# slybase/vllm-sly-radiance architecture

> Architecture diagram of the GitHub repository slybase/vllm-sly-radiance, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/slybase/vllm-sly-radiance
- Repository: https://github.com/slybase/vllm-sly-radiance

## Overview

This repository is a gfx1201-focused vLLM fork’s runtime optimization layer, intended to serve quantized Qwen models with fast attention, GEMM and speculative decoding. The principal workflow is a user request entering vLLM’s serving API, running target-model inference with optional draft-and-verify decoding and specialized kernels, then returning generated tokens. Quantization support extends beyond the primary Quark MXFP4 checkpoint to NVFP4, INT4, ParoQuant, AutoRound and escha. The README documents these capabilities; unsampled modules are represented at their exact tree paths, without inferring unshown call relationships.

Main entry points: [`dflash2/speculator.py`](https://github.com/slybase/vllm-sly-radiance/blob/main/dflash2/speculator.py) (DFlash2 drafter), [`paroquant/radiance_paroquant_mxfp4.py`](https://github.com/slybase/vllm-sly-radiance/blob/main/paroquant/radiance_paroquant_mxfp4.py) (ParoQuant MXFP4), [`sly/quant/radiance_quant_plugins.py`](https://github.com/slybase/vllm-sly-radiance/blob/main/sly/quant/radiance_quant_plugins.py) (Quant registration), [`radiance_mxfp4.py`](https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_mxfp4.py) (MXFP4 dispatch)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serving["Serving and decoding"]
  node_vllm["vLLM serving"]
  node_dflash["DFlash2 drafter<br/>[speculator.py]"]
  node_drafthead["Draft head"]
  node_lookup["Prompt lookup"]
  node_verify["Draft verification<br/>[qwen3_dflash2.py]"]
end

subgraph group_quant["Quantization support"]
  node_mxfp4["MXFP4 dispatch<br/>[radiance_mxfp4.py]"]
  node_nvfp4["NVFP4 support<br/>[radiance_nvfp4.py]"]
  node_paroquant["ParoQuant int4"]
  node_pq_mxfp4["ParoQuant MXFP4"]
  node_plugins["Quant registration"]
end

subgraph group_compute["GPU compute kernels"]
  node_autoround["AutoRound kernels"]
  node_w4a16["Drafter W4A16"]
  node_escha["Escha kernels<br/>[radiance_escha.py]"]
end

subgraph group_attention["Attention and state"]
  node_attn_decode["Decode attention"]
  node_attn_draft["Drafter attention"]
  node_gdn["Gated delta net<br/>[radiance_gdn.py]"]
  node_kv[("KV cache")]
end

node_user(("API user"))

node_user -->|"sends request"| node_vllm
node_vllm -->|"returns tokens"| node_user
node_vllm -->|"drafts tokens"| node_dflash
node_vllm -->|"verifies draft"| node_verify
node_vllm -->|"looks up prompts"| node_lookup
node_dflash -.->|"uses drafter"| node_drafthead
node_vllm -->|"dispatches GEMM"| node_mxfp4
node_vllm -->|"loads checkpoint"| node_nvfp4
node_vllm -->|"loads checkpoint"| node_paroquant
node_vllm -->|"loads checkpoint"| node_pq_mxfp4
node_plugins -->|"registers config"| node_paroquant
node_plugins -->|"registers config"| node_pq_mxfp4
node_pq_mxfp4 -->|"uses GEMM"| node_mxfp4
node_dflash -->|"uses tiled GEMM"| node_w4a16
node_vllm -.->|"runs quantized GEMM"| node_autoround
node_vllm -->|"loads checkpoint"| node_escha
node_vllm -->|"tunes verification"| node_attn_decode
node_vllm -->|"tunes draft attention"| node_attn_draft
node_vllm -->|"runs GDN kernels"| node_gdn
node_vllm -->|"reads and writes"| node_kv

click node_dflash "https://github.com/slybase/vllm-sly-radiance/blob/main/dflash2/speculator.py"
click node_drafthead "https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_drafthead.py"
click node_lookup "https://github.com/slybase/vllm-sly-radiance/blob/main/sly/radiance_lookup_draft.py"
click node_verify "https://github.com/slybase/vllm-sly-radiance/blob/main/dflash2/qwen3_dflash2.py"
click node_mxfp4 "https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_mxfp4.py"
click node_nvfp4 "https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_nvfp4.py"
click node_paroquant "https://github.com/slybase/vllm-sly-radiance/blob/main/paroquant/radiance_paroquant.py"
click node_pq_mxfp4 "https://github.com/slybase/vllm-sly-radiance/blob/main/paroquant/radiance_paroquant_mxfp4.py"
click node_plugins "https://github.com/slybase/vllm-sly-radiance/blob/main/sly/quant/radiance_quant_plugins.py"
click node_autoround "https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_autoround_kernels.h"
click node_w4a16 "https://github.com/slybase/vllm-sly-radiance/blob/main/sly/patch_w4a16_tiles.py"
click node_escha "https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_escha.py"
click node_attn_decode "https://github.com/slybase/vllm-sly-radiance/blob/main/sly/radiance_attn_decode.py"
click node_attn_draft "https://github.com/slybase/vllm-sly-radiance/blob/main/sly/radiance_attn_drafter.py"
click node_gdn "https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_gdn.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_vllm,node_dflash,node_drafthead,node_lookup,node_verify,node_user toneBlue
class node_mxfp4,node_nvfp4,node_paroquant,node_pq_mxfp4,node_plugins toneAmber
class node_autoround,node_w4a16,node_escha toneMint
class node_attn_decode,node_attn_draft,node_gdn,node_kv toneRose
```

## Components

### Serving and decoding

- **vLLM serving**
- [DFlash2 drafter](https://github.com/slybase/vllm-sly-radiance/blob/main/dflash2/speculator.py): `dflash2/speculator.py`
- [Draft head](https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_drafthead.py): `radiance_drafthead.py`
- [Prompt lookup](https://github.com/slybase/vllm-sly-radiance/blob/main/sly/radiance_lookup_draft.py): `sly/radiance_lookup_draft.py`
- [Draft verification](https://github.com/slybase/vllm-sly-radiance/blob/main/dflash2/qwen3_dflash2.py): `dflash2/qwen3_dflash2.py`

### Quantization support

- [MXFP4 dispatch](https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_mxfp4.py): `radiance_mxfp4.py`
- [NVFP4 support](https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_nvfp4.py): `radiance_nvfp4.py`
- [ParoQuant int4](https://github.com/slybase/vllm-sly-radiance/blob/main/paroquant/radiance_paroquant.py): `paroquant/radiance_paroquant.py`
- [ParoQuant MXFP4](https://github.com/slybase/vllm-sly-radiance/blob/main/paroquant/radiance_paroquant_mxfp4.py): `paroquant/radiance_paroquant_mxfp4.py`
- [Quant registration](https://github.com/slybase/vllm-sly-radiance/blob/main/sly/quant/radiance_quant_plugins.py): `sly/quant/radiance_quant_plugins.py`

### GPU compute kernels

- [AutoRound kernels](https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_autoround_kernels.h): `radiance_autoround_kernels.h`
- [Drafter W4A16](https://github.com/slybase/vllm-sly-radiance/blob/main/sly/patch_w4a16_tiles.py): `sly/patch_w4a16_tiles.py`
- [Escha kernels](https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_escha.py): `radiance_escha.py`

### Attention and state

- [Decode attention](https://github.com/slybase/vllm-sly-radiance/blob/main/sly/radiance_attn_decode.py): `sly/radiance_attn_decode.py`
- [Drafter attention](https://github.com/slybase/vllm-sly-radiance/blob/main/sly/radiance_attn_drafter.py): `sly/radiance_attn_drafter.py`
- [Gated delta net](https://github.com/slybase/vllm-sly-radiance/blob/main/radiance_gdn.py): `radiance_gdn.py`
- **KV cache**

### Other components

- **API user**

## Connections

- API user → vLLM serving: sends request (evidence: `README.md`)
- vLLM serving → API user: returns tokens (evidence: `README.md`)
- vLLM serving → DFlash2 drafter: drafts tokens (evidence: `README.md`)
- vLLM serving → Draft verification: verifies draft (evidence: `README.md`)
- vLLM serving → Prompt lookup: looks up prompts (evidence: `README.md`)
- DFlash2 drafter → Draft head: uses drafter
- vLLM serving → MXFP4 dispatch: dispatches GEMM (evidence: `README.md`)
- vLLM serving → NVFP4 support: loads checkpoint (evidence: `README.md`)
- vLLM serving → ParoQuant int4: loads checkpoint (evidence: `README.md`)
- vLLM serving → ParoQuant MXFP4: loads checkpoint (evidence: `README.md`)
- Quant registration → ParoQuant int4: registers config (evidence: `sly/quant/radiance_quant_plugins.py`)
- Quant registration → ParoQuant MXFP4: registers config (evidence: `sly/quant/radiance_quant_plugins.py`)
- ParoQuant MXFP4 → MXFP4 dispatch: uses GEMM (evidence: `paroquant/radiance_paroquant_mxfp4.py`)
- DFlash2 drafter → Drafter W4A16: uses tiled GEMM (evidence: `sly/patch_w4a16_tiles.py`)
- vLLM serving → AutoRound kernels: runs quantized GEMM
- vLLM serving → Escha kernels: loads checkpoint (evidence: `README.md`)
- vLLM serving → Decode attention: tunes verification (evidence: `README.md`)
- vLLM serving → Drafter attention: tunes draft attention (evidence: `README.md`)
- vLLM serving → Gated delta net: runs GDN kernels (evidence: `README.md`)
- vLLM serving → KV cache: reads and writes (evidence: `README.md`)
