# vllm-project/afd-plugin architecture

> Architecture diagram of the GitHub repository vllm-project/afd-plugin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vllm-project/afd-plugin
- Repository: https://github.com/vllm-project/afd-plugin

## Overview

afd-plugin extends vLLM with Attention-FFN Disaggregation for GPU and Ascend NPU inference. The documented runtime flow is vLLM configuration and plugin registration, role-specific Attention and FFN workers, then connector-mediated transfer of work between the roles. Model wrappers and backend-specific execution support that flow; compatibility patches, asynchronous scheduling, profiling, graph execution, and NPU kernels are supporting capabilities. The README documents these branches, but sampled source excerpts do not establish every runtime call path, so uncertain wiring is omitted.

Main entry points: [`afd_plugin/connectors/factory.py`](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/factory.py) (Connector factory), [`afd_plugin/v1/worker/attention_model_runner.py`](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/attention_model_runner.py) (Attention runners), [`afd_plugin/v1/worker/ffn_model_runner.py`](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/ffn_model_runner.py) (FFN runners), [`afd_plugin/__init__.py`](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/__init__.py) (Plugin registration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_plugin["Plugin setup"]
  node_registration["Plugin registration<br/>[__init__.py]"]
  node_config["AFD configuration<br/>[config.py]"]
  node_validation["Runtime validation<br/>[validation.py]"]
end

subgraph group_runtime["AFD workers"]
  node_attention["Attention workers"]
  node_attention_runner["Attention runners"]
  node_ffn["FFN workers<br/>[ffn_worker.py]"]
  node_ffn_runner["FFN runners"]
end

subgraph group_connectors["Connectors"]
  node_metadata["Connector metadata<br/>[metadata.py]"]
  node_factory["Connector factory<br/>[factory.py]"]
  node_gpu_connector["GPU P2P connector<br/>[p2p.py]"]
  node_npu_sync["NPU CAMP2P connector<br/>[camp2p.py]"]
  node_npu_async["NPU async CAM<br/>[async_cam.py]"]
end

subgraph group_models["Models and execution"]
  node_models["Model wrappers"]
  node_routing["MoE routing"]
  node_ubatching["NPU ubatching"]
end

subgraph group_support["Platform support"]
  node_compat["vLLM compatibility<br/>[vllm.py]"]
  node_patches["Runtime patches<br/>[engine_core.py]"]
  node_npu_ops{{"Ascend custom ops"}}
end

node_vllm(("vLLM runtime"))
node_gpu(("CUDA platform"))
node_npu(("Ascend NPU"))

node_vllm -->|"loads plugin"| node_registration
node_vllm -->|"provides config"| node_config
node_registration -->|"validates runtime"| node_validation
node_validation -->|"selects role workers"| node_attention
node_validation -->|"selects role workers"| node_ffn
node_attention -->|"initializes runner"| node_attention_runner
node_ffn -->|"initializes runner"| node_ffn_runner
node_attention_runner -->|"creates connector"| node_factory
node_ffn_runner -->|"creates connector"| node_factory
node_factory -->|"selects backend"| node_gpu_connector
node_factory -->|"selects backend"| node_npu_sync
node_factory -->|"selects backend"| node_npu_async
node_attention_runner -->|"produces work metadata"| node_metadata
node_metadata -->|"drives FFN work"| node_ffn_runner
node_gpu_connector -->|"transfers tensors"| node_gpu
node_npu_sync -->|"transfers tensors"| node_npu
node_npu_async -->|"transfers asynchronously"| node_npu
node_attention_runner -->|"executes model role"| node_models
node_ffn_runner -->|"executes model role"| node_models
node_models -.->|"uses routing strategy"| node_routing
node_npu_async -->|"supports two-stage ubatching"| node_ubatching
node_registration -->|"checks version"| node_compat
node_patches -->|"patches engine behavior"| node_vllm
node_npu -.->|"runs custom kernels"| node_npu_ops

click node_registration "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/__init__.py"
click node_config "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/config.py"
click node_validation "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/validation.py"
click node_attention "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/attention_worker.py"
click node_attention_runner "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/attention_model_runner.py"
click node_ffn "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/ffn_worker.py"
click node_ffn_runner "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/ffn_model_runner.py"
click node_metadata "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/metadata.py"
click node_factory "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/factory.py"
click node_gpu_connector "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/gpu/p2p.py"
click node_npu_sync "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/npu/camp2p.py"
click node_npu_async "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/npu/async_cam.py"
click node_models "https://github.com/vllm-project/afd-plugin/tree/main/afd_plugin/model_executor/models"
click node_routing "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/model_executor/routing_simulator.py"
click node_ubatching "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/model_executor/npu/async_cam_ubatching.py"
click node_compat "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/compat/vllm.py"
click node_patches "https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/compat/patches/engine_core.py"
click node_npu_ops "https://github.com/vllm-project/afd-plugin/tree/main/csrc/npu/ascend_kernels"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_registration,node_config,node_validation toneBlue
class node_attention,node_attention_runner,node_ffn,node_ffn_runner toneAmber
class node_metadata,node_factory,node_gpu_connector,node_npu_sync,node_npu_async toneMint
class node_models,node_routing,node_ubatching toneRose
class node_compat,node_patches,node_npu_ops,node_vllm,node_gpu,node_npu toneIndigo
```

## Components

### Plugin setup

- [Plugin registration](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/__init__.py): `afd_plugin/__init__.py`
- [AFD configuration](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/config.py): `afd_plugin/config.py`
- [Runtime validation](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/validation.py): `afd_plugin/validation.py`

### AFD workers

- [Attention workers](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/attention_worker.py): `afd_plugin/v1/worker/attention_worker.py`
- [Attention runners](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/attention_model_runner.py): `afd_plugin/v1/worker/attention_model_runner.py`
- [FFN workers](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/ffn_worker.py): `afd_plugin/v1/worker/ffn_worker.py`
- [FFN runners](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/v1/worker/ffn_model_runner.py): `afd_plugin/v1/worker/ffn_model_runner.py`

### Connectors

- [Connector metadata](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/metadata.py): `afd_plugin/connectors/metadata.py`
- [Connector factory](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/factory.py): `afd_plugin/connectors/factory.py`
- [GPU P2P connector](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/gpu/p2p.py): `afd_plugin/connectors/gpu/p2p.py`
- [NPU CAMP2P connector](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/npu/camp2p.py): `afd_plugin/connectors/npu/camp2p.py`
- [NPU async CAM](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/connectors/npu/async_cam.py): `afd_plugin/connectors/npu/async_cam.py`

### Models and execution

- [Model wrappers](https://github.com/vllm-project/afd-plugin/tree/main/afd_plugin/model_executor/models): `afd_plugin/model_executor/models`
- [MoE routing](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/model_executor/routing_simulator.py): `afd_plugin/model_executor/routing_simulator.py`
- [NPU ubatching](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/model_executor/npu/async_cam_ubatching.py): `afd_plugin/model_executor/npu/async_cam_ubatching.py`

### Platform support

- [vLLM compatibility](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/compat/vllm.py): `afd_plugin/compat/vllm.py`
- [Runtime patches](https://github.com/vllm-project/afd-plugin/blob/main/afd_plugin/compat/patches/engine_core.py): `afd_plugin/compat/patches/engine_core.py`
- [Ascend custom ops](https://github.com/vllm-project/afd-plugin/tree/main/csrc/npu/ascend_kernels): `csrc/npu/ascend_kernels`

### Other components

- **vLLM runtime**
- **CUDA platform**
- **Ascend NPU**

## Connections

- vLLM runtime → Plugin registration: loads plugin
- vLLM runtime → AFD configuration: provides config
- Plugin registration → Runtime validation: validates runtime
- Runtime validation → Attention workers: selects role workers
- Runtime validation → FFN workers: selects role workers
- Attention workers → Attention runners: initializes runner
- FFN workers → FFN runners: initializes runner
- Attention runners → Connector factory: creates connector
- FFN runners → Connector factory: creates connector
- Connector factory → GPU P2P connector: selects backend
- Connector factory → NPU CAMP2P connector: selects backend
- Connector factory → NPU async CAM: selects backend
- Attention runners → Connector metadata: produces work metadata
- Connector metadata → FFN runners: drives FFN work
- GPU P2P connector → CUDA platform: transfers tensors
- NPU CAMP2P connector → Ascend NPU: transfers tensors
- NPU async CAM → Ascend NPU: transfers asynchronously
- Attention runners → Model wrappers: executes model role
- FFN runners → Model wrappers: executes model role
- Model wrappers → MoE routing: uses routing strategy
- NPU async CAM → NPU ubatching: supports two-stage ubatching
- Plugin registration → vLLM compatibility: checks version
- Runtime patches → vLLM runtime: patches engine behavior
- Ascend NPU → Ascend custom ops: runs custom kernels