# syv-ai/hyperqwen architecture

> Architecture diagram of the GitHub repository syv-ai/hyperqwen, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/syv-ai/hyperqwen
- Repository: https://github.com/syv-ai/hyperqwen

## Overview

HyperQwen packages a patched vLLM serving setup for large Qwen models, plus model-preparation tools. The main workflow is choosing single-user or batch operation, preparing model weights when needed, then sending requests to an OpenAI-compatible API. The product also offers speculative decoding and KVarN long-context KV-cache compression; preparation includes draft-vocabulary and MTP quantization. The README documents these capabilities, while sampled source gives detail on selected preparation and KVarN internals, not the full serving implementation.

Main entry points: [`PATCHES.md`](https://github.com/syv-ai/hyperqwen/blob/main/PATCHES.md) (Patched vLLM), [`kvarn/files/vllm/v1/attention/backends/kvarn_attn.py`](https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/v1/attention/backends/kvarn_attn.py) (KVarN backend), [`patches`](https://github.com/syv-ai/hyperqwen/tree/main/patches) (Speculative decoding), [`drafter/export_mtp.py`](https://github.com/syv-ai/hyperqwen/blob/main/drafter/export_mtp.py) (MTP export)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serve["Serving modes"]
  node_single["Single mode"]
  node_batch["Batch mode"]
end

subgraph group_runtime["Inference runtime"]
  node_api["OpenAI API"]
  node_vllm["Patched vLLM<br/>[PATCHES.md]"]
  node_spec["Speculative decoding"]
  node_mtp["MTP drafter<br/>[quant_mtp.py]"]
  node_dflash["DFlash2 drafter"]
end

subgraph group_prepare["Model preparation"]
  node_prep["Model preparation<br/>[README.md]"]
  node_quant["Head quantization"]
  node_vocab["Draft vocabulary"]
  node_mtp_export["MTP export<br/>[export_mtp.py]"]
  node_mtp_train["MTP training<br/>[train_mtp.py]"]
end

subgraph group_cache["Long-context cache"]
  node_kvarn["KVarN backend<br/>[kvarn_attn.py]"]
  node_sinkhorn["Variance normalization<br/>[sinkhorn.py]"]
  node_store["KV tile storage<br/>[kvarn_store.py]"]
  node_decode["Fused decode"]
end

node_user(("API client"))
node_source_model["Model checkpoint"]

node_user -->|"selects mode"| node_single
node_user -->|"selects mode"| node_batch
node_single -->|"starts runtime"| node_vllm
node_batch -->|"starts runtime"| node_vllm
node_user -->|"sends requests"| node_api
node_api -->|"dispatches requests"| node_vllm
node_prep -->|"reads weights"| node_source_model
node_prep -->|"runs quantization"| node_quant
node_quant -->|"enables draft head"| node_vocab
node_mtp_train -->|"provides trained weights"| node_mtp_export
node_mtp_export -->|"exports drafter"| node_mtp
node_mtp_export -->|"builds draft head"| node_vocab
node_vllm -->|"uses patches"| node_spec
node_spec -->|"configures speculation"| node_mtp
node_spec -->|"configures speculation"| node_dflash
node_vllm -.->|"selects backend"| node_kvarn
node_kvarn -->|"normalizes tiles"| node_sinkhorn
node_kvarn -->|"stores KV tiles"| node_store
node_kvarn -->|"decodes cache"| node_decode

click node_single "https://github.com/syv-ai/hyperqwen/tree/main/single-user"
click node_batch "https://github.com/syv-ai/hyperqwen/tree/main/batch"
click node_api "https://github.com/syv-ai/hyperqwen/blob/main/patches/serve-404-served-names.patch"
click node_vllm "https://github.com/syv-ai/hyperqwen/blob/main/PATCHES.md"
click node_spec "https://github.com/syv-ai/hyperqwen/tree/main/patches"
click node_mtp "https://github.com/syv-ai/hyperqwen/blob/main/prepare/quant_mtp.py"
click node_dflash "https://github.com/syv-ai/hyperqwen/tree/main/drafter"
click node_prep "https://github.com/syv-ai/hyperqwen/blob/main/prepare/README.md"
click node_quant "https://github.com/syv-ai/hyperqwen/blob/main/prepare/quant_heads_stream.py"
click node_vocab "https://github.com/syv-ai/hyperqwen/blob/main/prepare/build_draft_vocab.py"
click node_mtp_export "https://github.com/syv-ai/hyperqwen/blob/main/drafter/export_mtp.py"
click node_mtp_train "https://github.com/syv-ai/hyperqwen/blob/main/drafter/train_mtp.py"
click node_kvarn "https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/v1/attention/backends/kvarn_attn.py"
click node_sinkhorn "https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/model_executor/layers/quantization/kvarn/sinkhorn.py"
click node_store "https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/v1/attention/ops/kvarn_store.py"
click node_decode "https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/v1/attention/ops/triton_kvarn_decode.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_single,node_batch,node_user toneBlue
class node_api,node_vllm,node_spec,node_mtp,node_dflash toneAmber
class node_prep,node_quant,node_vocab,node_mtp_export,node_mtp_train toneMint
class node_kvarn,node_sinkhorn,node_store,node_decode toneRose
class node_source_model toneIndigo
```

## Components

### Serving modes

- [Single mode](https://github.com/syv-ai/hyperqwen/tree/main/single-user): `single-user`
- [Batch mode](https://github.com/syv-ai/hyperqwen/tree/main/batch): `batch`

### Inference runtime

- [OpenAI API](https://github.com/syv-ai/hyperqwen/blob/main/patches/serve-404-served-names.patch): `patches/serve-404-served-names.patch`
- [Patched vLLM](https://github.com/syv-ai/hyperqwen/blob/main/PATCHES.md): `PATCHES.md`
- [Speculative decoding](https://github.com/syv-ai/hyperqwen/tree/main/patches): `patches`
- [MTP drafter](https://github.com/syv-ai/hyperqwen/blob/main/prepare/quant_mtp.py): `prepare/quant_mtp.py`
- [DFlash2 drafter](https://github.com/syv-ai/hyperqwen/tree/main/drafter): `drafter`

### Model preparation

- [Model preparation](https://github.com/syv-ai/hyperqwen/blob/main/prepare/README.md): `prepare/README.md`
- [Head quantization](https://github.com/syv-ai/hyperqwen/blob/main/prepare/quant_heads_stream.py): `prepare/quant_heads_stream.py`
- [Draft vocabulary](https://github.com/syv-ai/hyperqwen/blob/main/prepare/build_draft_vocab.py): `prepare/build_draft_vocab.py`
- [MTP export](https://github.com/syv-ai/hyperqwen/blob/main/drafter/export_mtp.py): `drafter/export_mtp.py`
- [MTP training](https://github.com/syv-ai/hyperqwen/blob/main/drafter/train_mtp.py): `drafter/train_mtp.py`

### Long-context cache

- [KVarN backend](https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/v1/attention/backends/kvarn_attn.py): `kvarn/files/vllm/v1/attention/backends/kvarn_attn.py`
- [Variance normalization](https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/model_executor/layers/quantization/kvarn/sinkhorn.py): `kvarn/files/vllm/model_executor/layers/quantization/kvarn/sinkhorn.py`
- [KV tile storage](https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/v1/attention/ops/kvarn_store.py): `kvarn/files/vllm/v1/attention/ops/kvarn_store.py`
- [Fused decode](https://github.com/syv-ai/hyperqwen/blob/main/kvarn/files/vllm/v1/attention/ops/triton_kvarn_decode.py): `kvarn/files/vllm/v1/attention/ops/triton_kvarn_decode.py`

### Other components

- **API client**
- **Model checkpoint**

## Connections

- API client → Single mode: selects mode
- API client → Batch mode: selects mode
- Single mode → Patched vLLM: starts runtime
- Batch mode → Patched vLLM: starts runtime
- API client → OpenAI API: sends requests
- OpenAI API → Patched vLLM: dispatches requests
- Model preparation → Model checkpoint: reads weights
- Model preparation → Head quantization: runs quantization
- Head quantization → Draft vocabulary: enables draft head
- MTP training → MTP export: provides trained weights
- MTP export → MTP drafter: exports drafter
- MTP export → Draft vocabulary: builds draft head
- Patched vLLM → Speculative decoding: uses patches
- Speculative decoding → MTP drafter: configures speculation
- Speculative decoding → DFlash2 drafter: configures speculation
- Patched vLLM → KVarN backend: selects backend
- KVarN backend → Variance normalization: normalizes tiles
- KVarN backend → KV tile storage: stores KV tiles
- KVarN backend → Fused decode: decodes cache