How vllm-project/afd-plugin works
afd-plugin extends vLLM with Attention-FFN Disaggregation for GPU and Ascend NPU inference. The documented runtime flow is vLLM configuration and plugin registration, role-specific Attention and FFN workers, then connector-mediated transfer of work between the roles. Model wrappers and backend-specific execution support that flow; compatibility patches, asynchronous scheduling, profiling, graph execution, and NPU kernels are supporting capabilities. The README documents these branches, but sampled source excerpts do not establish every runtime call path, so uncertain wiring is omitted.
Main entry points: factory.py Connector factory · attention_model_runner.py Attention runners · ffn_model_runner.py FFN runners · __init__.py Plugin registration
Components 21
Plugin setup
- Plugin registration
afd_plugin/__init__.py - AFD configuration
afd_plugin/config.py - Runtime validation
afd_plugin/validation.py
AFD workers
- Attention workers
afd_plugin/v1/worker/attention_worker.py - Attention runners
afd_plugin/v1/worker/attention_model_runner.py - FFN workers
afd_plugin/v1/worker/ffn_worker.py - FFN runners
afd_plugin/v1/worker/ffn_model_runner.py
Connectors
- Connector metadata
afd_plugin/connectors/metadata.py - Connector factory
afd_plugin/connectors/factory.py - GPU P2P connector
afd_plugin/connectors/gpu/p2p.py - NPU CAMP2P connector
afd_plugin/connectors/npu/camp2p.py - NPU async CAM
afd_plugin/connectors/npu/async_cam.py
Models and execution
- Model wrappers
afd_plugin/model_executor/models - MoE routing
afd_plugin/model_executor/routing_simulator.py - NPU ubatching
afd_plugin/model_executor/npu/async_cam_ubatching.py
Platform support
- vLLM compatibility
afd_plugin/compat/vllm.py - Runtime patches
afd_plugin/compat/patches/engine_core.py - Ascend custom ops
csrc/npu/ascend_kernels
Other components
- vLLM runtime
- CUDA platform
- Ascend NPU
Connections 24
- vLLM runtime → Plugin registration: loads plugin
- vLLM runtime → AFD configuration: provides config
- Plugin registration → Runtime validation: validates runtime
- Runtime validation → Attention workers: selects role workers
- Runtime validation → FFN workers: selects role workers
- Attention workers → Attention runners: initializes runner
- FFN workers → FFN runners: initializes runner
- Attention runners → Connector factory: creates connector
- FFN runners → Connector factory: creates connector
- Connector factory → GPU P2P connector: selects backend
- Connector factory → NPU CAMP2P connector: selects backend
- Connector factory → NPU async CAM: selects backend
- Attention runners → Connector metadata: produces work metadata
- Connector metadata → FFN runners: drives FFN work
- GPU P2P connector → CUDA platform: transfers tensors
- NPU CAMP2P connector → Ascend NPU: transfers tensors
- NPU async CAM → Ascend NPU: transfers asynchronously
- Attention runners → Model wrappers: executes model role
- FFN runners → Model wrappers: executes model role
- Model wrappers → MoE routing: uses routing strategy
- NPU async CAM → NPU ubatching: supports two-stage ubatching
- Plugin registration → vLLM compatibility: checks version
- Runtime patches → vLLM runtime: patches engine behavior
- Ascend NPU → Ascend custom ops: runs custom kernels