# wtdcode/vllm-backport architecture

> Architecture diagram of the GitHub repository wtdcode/vllm-backport, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/wtdcode/vllm-backport
- Repository: https://github.com/wtdcode/vllm-backport

## Overview

vLLM Backport is an LLM inference server fork aimed at running frontier models and quantization formats on older GPUs. The primary workflow is a client request entering the HTTP API, being processed by the engine, and producing generated output; the README also highlights model-specific decoding, caching, multimodal support, and distributed execution. The graph preserves those product branches. Source excerpts are partial, so some subsystem links reflect documented capabilities rather than inspected call sites.

Main entry points: [`vllm/v1/engine`](https://github.com/wtdcode/vllm-backport/tree/master/vllm/v1/engine) (Engine core), [`rust/src/server/src/routes`](https://github.com/wtdcode/vllm-backport/tree/master/rust/src/server/src/routes) (HTTP API), [`vllm/v1/engine/llm_engine.py`](https://github.com/wtdcode/vllm-backport/blob/master/vllm/v1/engine/llm_engine.py) (LLM engine), [`vllm/inputs/engine.py`](https://github.com/wtdcode/vllm-backport/blob/master/vllm/inputs/engine.py) (Engine inputs)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_serving["Serving"]
  node_api["HTTP API"]
  node_inputs["Engine inputs<br/>[engine.py]"]
  node_engine["LLM engine<br/>[llm_engine.py]"]
  node_metrics["Metrics routes<br/>[metrics.rs]"]
end

subgraph group_inference["Inference"]
  node_enginecore["Engine core"]
  node_parser["Streaming parsers"]
  node_toolparsers["Model tool parsers"]
  node_tokenizer["Tokenization<br/>[lib.rs]"]
  node_multimodal["Multimodal inputs<br/>[multimodal.py]"]
  node_kvconfig["KV cache config<br/>[cache.py]"]
  node_speculative["Speculative decoding<br/>[speculative.py]"]
end

subgraph group_acceleration["Acceleration"]
  node_quantization["Quantized kernels"]
  node_compilation["Graph compilation"]
  node_distributed["Distributed execution"]
  node_weighttransfer["Weight transfer"]
end

subgraph group_extensions["Extensions"]
  node_lmcache[("LMCache service")]
end

node_client(("Client"))
node_response["Generated response"]

node_client -->|"sends request"| node_api
node_api -->|"normalizes input"| node_inputs
node_api -->|"submits request"| node_engine
node_engine -->|"dispatches work"| node_enginecore
node_enginecore -->|"returns output"| node_response
node_engine -->|"parses streamed output"| node_parser
node_parser -->|"selects format parser"| node_toolparsers
node_inputs -->|"uses token IDs"| node_tokenizer
node_inputs -->|"carries media data"| node_multimodal
node_enginecore -->|"configures KV cache"| node_kvconfig
node_enginecore -->|"configures decoding"| node_speculative
node_enginecore -->|"uses kernels"| node_quantization
node_enginecore -->|"compiles execution"| node_compilation
node_enginecore -->|"executes across workers"| node_distributed
node_engine -.->|"updates weights"| node_weighttransfer
node_enginecore -.->|"transfers KV blocks"| node_lmcache
node_api -->|"exposes metrics"| node_metrics

click node_api "https://github.com/wtdcode/vllm-backport/tree/master/rust/src/server/src/routes"
click node_inputs "https://github.com/wtdcode/vllm-backport/blob/master/vllm/inputs/engine.py"
click node_engine "https://github.com/wtdcode/vllm-backport/blob/master/vllm/v1/engine/llm_engine.py"
click node_enginecore "https://github.com/wtdcode/vllm-backport/tree/master/vllm/v1/engine"
click node_parser "https://github.com/wtdcode/vllm-backport/blob/master/vllm/parser/engine/streaming_parser_engine.py"
click node_toolparsers "https://github.com/wtdcode/vllm-backport/tree/master/vllm/tool_parsers"
click node_tokenizer "https://github.com/wtdcode/vllm-backport/blob/master/rust/src/tokenizer/src/lib.rs"
click node_multimodal "https://github.com/wtdcode/vllm-backport/blob/master/vllm/config/multimodal.py"
click node_kvconfig "https://github.com/wtdcode/vllm-backport/blob/master/vllm/config/cache.py"
click node_speculative "https://github.com/wtdcode/vllm-backport/blob/master/vllm/config/speculative.py"
click node_quantization "https://github.com/wtdcode/vllm-backport/tree/master/csrc/libtorch_stable/quantization"
click node_compilation "https://github.com/wtdcode/vllm-backport/tree/master/vllm/compilation"
click node_distributed "https://github.com/wtdcode/vllm-backport/tree/master/vllm/distributed"
click node_weighttransfer "https://github.com/wtdcode/vllm-backport/blob/master/vllm/distributed/weight_transfer/sharded_rdt_engine.py"
click node_metrics "https://github.com/wtdcode/vllm-backport/blob/master/rust/src/server/src/routes/metrics.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_inputs,node_engine,node_metrics,node_client toneBlue
class node_enginecore,node_parser,node_toolparsers,node_tokenizer,node_multimodal,node_kvconfig,node_speculative toneAmber
class node_quantization,node_compilation,node_distributed,node_weighttransfer toneMint
class node_lmcache toneRose
class node_response toneIndigo
```

## Components

### Serving

- [HTTP API](https://github.com/wtdcode/vllm-backport/tree/master/rust/src/server/src/routes): `rust/src/server/src/routes`
- [Engine inputs](https://github.com/wtdcode/vllm-backport/blob/master/vllm/inputs/engine.py): `vllm/inputs/engine.py`
- [LLM engine](https://github.com/wtdcode/vllm-backport/blob/master/vllm/v1/engine/llm_engine.py): `vllm/v1/engine/llm_engine.py`
- [Metrics routes](https://github.com/wtdcode/vllm-backport/blob/master/rust/src/server/src/routes/metrics.rs): `rust/src/server/src/routes/metrics.rs`

### Inference

- [Engine core](https://github.com/wtdcode/vllm-backport/tree/master/vllm/v1/engine): `vllm/v1/engine`
- [Streaming parsers](https://github.com/wtdcode/vllm-backport/blob/master/vllm/parser/engine/streaming_parser_engine.py): `vllm/parser/engine/streaming_parser_engine.py`
- [Model tool parsers](https://github.com/wtdcode/vllm-backport/tree/master/vllm/tool_parsers): `vllm/tool_parsers`
- [Tokenization](https://github.com/wtdcode/vllm-backport/blob/master/rust/src/tokenizer/src/lib.rs): `rust/src/tokenizer/src/lib.rs`
- [Multimodal inputs](https://github.com/wtdcode/vllm-backport/blob/master/vllm/config/multimodal.py): `vllm/config/multimodal.py`
- [KV cache config](https://github.com/wtdcode/vllm-backport/blob/master/vllm/config/cache.py): `vllm/config/cache.py`
- [Speculative decoding](https://github.com/wtdcode/vllm-backport/blob/master/vllm/config/speculative.py): `vllm/config/speculative.py`

### Acceleration

- [Quantized kernels](https://github.com/wtdcode/vllm-backport/tree/master/csrc/libtorch_stable/quantization): `csrc/libtorch_stable/quantization`
- [Graph compilation](https://github.com/wtdcode/vllm-backport/tree/master/vllm/compilation): `vllm/compilation`
- [Distributed execution](https://github.com/wtdcode/vllm-backport/tree/master/vllm/distributed): `vllm/distributed`
- [Weight transfer](https://github.com/wtdcode/vllm-backport/blob/master/vllm/distributed/weight_transfer/sharded_rdt_engine.py): `vllm/distributed/weight_transfer/sharded_rdt_engine.py`

### Extensions

- **LMCache service**

### Other components

- **Client**
- **Generated response**

## Connections

- Client → HTTP API: sends request
- HTTP API → Engine inputs: normalizes input
- HTTP API → LLM engine: submits request
- LLM engine → Engine core: dispatches work
- Engine core → Generated response: returns output
- LLM engine → Streaming parsers: parses streamed output
- Streaming parsers → Model tool parsers: selects format parser
- Engine inputs → Tokenization: uses token IDs
- Engine inputs → Multimodal inputs: carries media data
- Engine core → KV cache config: configures KV cache
- Engine core → Speculative decoding: configures decoding
- Engine core → Quantized kernels: uses kernels
- Engine core → Graph compilation: compiles execution
- Engine core → Distributed execution: executes across workers
- LLM engine → Weight transfer: updates weights
- Engine core → LMCache service: transfers KV blocks
- HTTP API → Metrics routes: exposes metrics