# modelengine-group/unified-cache-management architecture

> Architecture diagram of the GitHub repository modelengine-group/unified-cache-management, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/modelengine-group/unified-cache-management
- Repository: https://github.com/modelengine-group/unified-cache-management

## Overview

Unified Cache Management (UCM) is a plugin-style library for LLM inference runtimes, not a standalone server. Its architecture is centered on three extensibility boundaries:

1. integration hooks into external model-serving frameworks,
2. algorithm modules for sparse KV reuse/retrieval,
3. storage backends for persisting and reloading KV cache blocks.

The repository is split so that Python coordinates control flow, while performance-critical pieces live in C++/CUDA/Ascend extensions.

Main architectural layers

- Runtime integration layer: `ucm/integration`
  - This is where UCM connects to external inference stacks.
  - It contains adapters and patches for:
    - vLLM, including separate patch trees for v0.9.2 and v0.11.0
    - SGLang
    - MindIE LLM
  - These modules hook into scheduler, worker, attention, KV connector, and cache-manager code in the upstream runtime.
  - The purpose is to inject UCM behavior without replacing the host engine.

- Sparse KV algorithm layer: `ucm/sparse`
  - This is the core algorithmic layer for non-prefix cache reuse.
  - It includes several algorithm families:
    - `blend` for cache blend / blockwise RoPE-style handling
    - `esa` for retrieval-based sparse attention
    - `gsa` for generalized sparse attention, with offload and prefetch submodules
    - `gsa_on_device` for device-side sparse retrieval and cache operations
    - `kvstar` for multistep retrieval workflows
    - `rerope` for window extrapolation / RoPE-related attention modifications
  - The README’s “UcmSparseBase” concept is reflected here: algorithms are designed to plug into the same hook points in the model runtime.

- Storage and cache persistence layer: `ucm/store`
  - This layer abstracts KV cache persistence and retrieval.
  - It is not tied to one storage medium; the repo includes multiple backends:
    - `posix` for local file and AIO-based storage
    - `nfsstore` for POSIX/NFS-mounted shared storage
    - `ds3fs` for a file-system-backed distributed store
    - `pcstore` for prefix-cache-oriented storage
    - `pipeline` for pipeline-oriented store integration
    - `fake` and `empty` for testing/no-op behavior
    - `mooncakestore` as another connector
  - Factories such as `ucm/store/factory.py` and `factory_v1.py` select the backend.
  - The important boundary here is that sparse algorithms do not talk directly to filesystems or devices; they target a KV store interface.

- Shared native infrastructure: `ucm/shared`
  - This is the cross-cutting foundation used by both sparse and store code.
  - It contains:
    - logging
    - metrics
    - thread primitives
    - timing and status utilities
    - device/data transport abstractions under `trans`
  - The transport layer has device-specific implementations for CUDA, Ascend, Simu, and Maca, which matters because the same cache operations need to move data across different accelerators and host memory.

- PD disaggregation support: `ucm/pd`
  - This is a smaller but important subsystem for storage-compute separation.
  - The repo includes a `toy_proxy_server.py` and supporting docs/examples, indicating that UCM can split prefill/decode responsibilities across machines or roles.

Data flow through the system

1. A user launches a supported inference stack or example.
   - Examples in the repo show offline inference, benchmarking, and deployment scripts.
   - Supported runtimes include vLLM, SGLang, and MindIE LLM.

2. UCM is injected into the runtime via patch/adaptor modules.
   - The integration layer modifies scheduler and attention paths.
   - For vLLM, there are patched implementations for model runner, worker, KV connector, scheduler, and attention layers.
   - For Ascend/NPU paths, there are separate patch trees and device-specific workers/runners.

3. During prefill or decode, UCM decides whether to:
   - reuse prefix KV,
   - retrieve cached KV blocks from external storage,
   - dump KV blocks out of device memory,
   - or apply a sparse attention strategy instead of full dense attention.
   - This decision is driven by the active sparse algorithm or KV connector.

4. The selected sparse module computes block selection or retrieval strategy.
   - Some algorithms are pure Python orchestration.
   - Others delegate to native code for speed, especially when performing top-k selection, hashing, hamming-distance retrieval, or cache reshaping.

5. Cache data is moved through a backend store.
   - The storage boundary handles persistence to local files, shared filesystems, or distributed cache layers.
   - The same abstraction serves both prefix caching and sparse cache reuse.

6. Device-specific kernels and transport code handle high-throughput movement and compute.
   - CUDA and Ascend implementations exist where performance matters.
   - This is especially visible in `gsa_on_device` and the native code under `ucm/shared/trans` and `ucm/store`.

Key subsystems and their responsibilities

- `ucm/integration/vllm`
  - The largest integration surface.
  - Contains patch files and runtime-adapted modules for vLLM internals.
  - Separate branches of code exist for dense prefix caching, sparse attention, and Ascend adaptations.
  - This is the main place where UCM becomes “active” inside the model server.

- `ucm/integration/sglang`
  - A lighter integration path with patching and connector logic for SGLang.

- `ucm/integration/mindie`
  - MindIE-specific integration with patching, unified cache memory pool, and a native hash extension.

- `ucm/store/*`
  - Concrete persistence and transfer backends.
  - `pcstore`, `nfsstore`, and `posix` are especially central because they represent different deployment topologies: local single-node, shared filesystem, or distributed/cache-separated environments.

- `ucm/sparse/gsa_on_device`
  - Contains the most hardware-specific sparse retrieval path.
  - Includes Python orchestration, config files for different models, and large native subtrees for CUDA and Ascend.
  - This subsystem is where sparse retrieval can be pushed onto the accelerator instead of being handled purely on CPU or host-side Python.

- `ucm/sparse/kvstar`
  - A retrieval pipeline with explicit task management and a native core.
  - The architecture here is task-oriented: query/request preparation, task queueing, worker execution, result waiting, and SIMD compute kernels.

- `ucm/shared/metrics` and `ucm/logger`
  - These are supporting observability layers.
  - They provide the stats needed to understand cache hit/miss behavior, offload performance, and retrieval overhead.

- `benchmarks`, `examples`, and `test`
  - These are not production runtime paths, but they matter architecturally because they show how the system is validated.
  - Benchmarks cover logging and trace replay.
  - Example scripts demonstrate offline inference, deployments, and metrics integration.
  - Tests include end-to-end scenarios for offline/online inference, environment precheck, evaluator flows, sparse variants, and connector save/load behavior.

Technology boundaries that matter

- Python is the orchestration and extension layer.
  - It owns configuration, integration logic, algorithm selection, and runtime hooks.

- C++ is used for native task engines, storage engines, retrieval engines, and low-level infrastructure.
  - Built with CMake, with many subprojects and pybind-style bindings.

- CUDA and Ascend/NPU code exist side by side.
  - This is important architecturally because the same sparse-cache ideas are implemented on multiple accelerator families.

- The repo assumes external LLM runtimes as hosts.
  - UCM augments vLLM/SGLang/MindIE rather than replacing them.
  - That means the diagram should show UCM as a middleware/plugin layer between the application and the model runtime internals.

How to draw the architecture

A clear diagram would show:

- At the top: applications, benchmarks, or inference scripts.
- In the middle: supported runtimes such as vLLM, SGLang, and MindIE.
- Overlaid into those runtimes: UCM integration hooks for scheduler, worker, attention, and KV connector paths.
- Beneath that: UCM’s sparse algorithm modules, with separate branches for blend, ESA, GSA, KVStar, and RoPE-related behavior.
- Side-by-side with algorithms: storage backends for cache persistence and retrieval.
- Under everything: shared C++/CUDA/Ascend transport, logging, and metrics infrastructure.
- Off to one side: PD disaggregation support and on-device sparse retrieval as specialized deployments.

In short, UCM is a modular cache-management and sparse-attention framework that sits inside existing LLM inference engines, routes KV cache through pluggable storage backends, and uses native device-aware components to make prefix caching, sparse attention, and disaggregated inference practical across GPU and Ascend environments.

Main entry points: [`ucm/store/factory.py`](https://github.com/modelengine-group/unified-cache-management/blob/HEAD/ucm/store/factory.py) (Store Factory), [`ucm/sparse/base.py`](https://github.com/modelengine-group/unified-cache-management/blob/HEAD/ucm/sparse/base.py) (Sparse Core), [`ucm/shared/trans`](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/shared/trans) (Transport), [`ucm/integration/vllm/ucm_connector.py`](https://github.com/modelengine-group/unified-cache-management/blob/HEAD/ucm/integration/vllm/ucm_connector.py) (UCM Connectors)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph "Integration"
  n_vllm["vLLM<br/>host runtime"]
  n_sglang["SGLang<br/>host runtime"]
  n_mindie["MindIE<br/>host runtime"]
  n_connector["UCM Connectors<br/>integration glue<br/>[ucm_connector.py]"]
end

subgraph "Sparse KV"
  n_sparse_base["Sparse Core<br/>algorithm API<br/>[base.py]"]
  n_blend["Blend<br/>sparse algo"]
  n_esa["ESA<br/>sparse algo"]
  n_gsa["GSA<br/>sparse algo"]
  n_gsa_device["GSA On-Device<br/>device sparse"]
  n_kvstar["KVStar<br/>retrieval pipeline"]
  n_rerope["ReRoPE<br/>attention mod"]
end

subgraph "Storage"
  n_store_factory["Store Factory<br/>backend selector<br/>[factory.py]"]
  n_posix["Posix Store<br/>local backend"]
  n_nfs["NFS Store<br/>shared backend"]
  n_pcstore["PC Store<br/>prefix backend"]
  n_ds3fs["DS3FS<br/>distributed backend"]
  n_pipeline["Pipeline Store<br/>pipeline backend"]
  n_fake_empty["Fake/Empty<br/>null backend"]
end

subgraph "Shared Infra"
  n_transport["Transport<br/>device io"]
  n_metrics["Metrics<br/>observability"]
  n_logging["Logging"]
end

subgraph "Usage"
  n_examples["Examples<br/>usage scripts"]
  n_benchmarks["Benchmarks<br/>perf harness"]
  n_tests["Tests<br/>validation suite"]
end

subgraph "PD Support"
  n_pd["PD Disagg<br/>disaggregation"]
end

n_examples -->|"launches"| n_vllm
n_examples -->|"launches"| n_sglang
n_examples -->|"launches"| n_mindie
n_benchmarks -->|"measures"| n_metrics
n_tests -->|"validates"| n_store_factory
n_tests -->|"validates"| n_gsa_device
n_vllm -->|"hooks"| n_connector
n_sglang -->|"hooks"| n_connector
n_mindie -->|"hooks"| n_connector
n_connector -->|"dispatches"| n_sparse_base
n_sparse_base -->|"selects"| n_blend
n_sparse_base -->|"selects"| n_esa
n_sparse_base -->|"selects"| n_gsa
n_sparse_base -->|"selects"| n_gsa_device
n_sparse_base -->|"selects"| n_kvstar
n_sparse_base -->|"selects"| n_rerope
n_gsa -->|"loads from"| n_store_factory
n_esa -->|"retrieves from"| n_store_factory
n_kvstar -->|"retrieves from"| n_store_factory
n_store_factory -->|"chooses"| n_posix
n_store_factory -->|"chooses"| n_nfs
n_store_factory -->|"chooses"| n_pcstore
n_store_factory -->|"chooses"| n_ds3fs
n_store_factory -->|"chooses"| n_pipeline
n_store_factory -->|"chooses"| n_fake_empty
n_posix -->|"moves data via"| n_transport
n_nfs -->|"moves data via"| n_transport
n_pcstore -->|"moves data via"| n_transport
n_ds3fs -->|"moves data via"| n_transport
n_pipeline -->|"moves data via"| n_transport
n_gsa_device -->|"executes on"| n_transport
n_vllm -->|"emits"| n_metrics
n_store_factory -->|"emits"| n_metrics
n_vllm -->|"uses"| n_logging
n_store_factory -->|"uses"| n_logging
n_pd -->|"separates with"| n_store_factory
n_pd -->|"coordinates"| n_connector

click n_examples "https://github.com/modelengine-group/unified-cache-management/tree/develop/examples"
click n_benchmarks "https://github.com/modelengine-group/unified-cache-management/tree/develop/benchmarks"
click n_tests "https://github.com/modelengine-group/unified-cache-management/tree/develop/test"
click n_vllm "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/integration/vllm"
click n_sglang "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/integration/sglang"
click n_mindie "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/integration/mindie"
click n_connector "https://github.com/modelengine-group/unified-cache-management/blob/develop/ucm/integration/vllm/ucm_connector.py"
click n_sparse_base "https://github.com/modelengine-group/unified-cache-management/blob/develop/ucm/sparse/base.py"
click n_blend "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/sparse/blend"
click n_esa "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/sparse/esa"
click n_gsa "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/sparse/gsa"
click n_gsa_device "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/sparse/gsa_on_device"
click n_kvstar "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/sparse/kvstar"
click n_rerope "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/sparse/rerope"
click n_store_factory "https://github.com/modelengine-group/unified-cache-management/blob/develop/ucm/store/factory.py"
click n_posix "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/store/posix"
click n_nfs "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/store/nfsstore"
click n_pcstore "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/store/pcstore"
click n_ds3fs "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/store/ds3fs"
click n_pipeline "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/store/pipeline"
click n_fake_empty "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/store/fake"
click n_transport "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/shared/trans"
click n_metrics "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/shared/metrics"
click n_logging "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/shared/infra/logger"
click n_pd "https://github.com/modelengine-group/unified-cache-management/tree/develop/ucm/pd"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class n_vllm,n_sglang,n_mindie,n_connector toneBlue
class n_sparse_base,n_blend,n_esa,n_gsa,n_gsa_device,n_kvstar,n_rerope toneAmber
class n_store_factory,n_posix,n_nfs,n_pcstore,n_ds3fs,n_pipeline,n_fake_empty toneMint
class n_transport,n_metrics,n_logging toneRose
class n_examples,n_benchmarks,n_tests toneIndigo
class n_pd toneTeal
```

## Components

### Integration

Adapters and patch trees that inject UCM into external runtimes.

- [vLLM](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/integration/vllm): host runtime · Primary runtime integration surface with patch trees for multiple versions and Ascend variants. · `ucm/integration/vllm`
- [SGLang](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/integration/sglang): host runtime · Integration and connector layer for SGLang. · `ucm/integration/sglang`
- [MindIE](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/integration/mindie): host runtime · Integration and unified-cache support for MindIE LLM. · `ucm/integration/mindie`
- [UCM Connectors](https://github.com/modelengine-group/unified-cache-management/blob/HEAD/ucm/integration/vllm/ucm_connector.py): integration glue · Common connector entrypoints that bridge host runtime cache hooks to UCM. · `ucm/integration/vllm/ucm_connector.py`

### Sparse KV

Algorithm modules for cache reuse, sparse attention, and retrieval.

- [Sparse Core](https://github.com/modelengine-group/unified-cache-management/blob/HEAD/ucm/sparse/base.py): algorithm API · Shared sparse-algorithm interface and state used by multiple cache strategies. · `ucm/sparse/base.py`
- [Blend](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/sparse/blend): sparse algo · Cache blend and blockwise RoPE-style handling. · `ucm/sparse/blend`
- [ESA](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/sparse/esa): sparse algo · Retrieval-based sparse attention module. · `ucm/sparse/esa`
- [GSA](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/sparse/gsa): sparse algo · Generalized sparse attention with offload and prefetch subsystems. · `ucm/sparse/gsa`
- [GSA On-Device](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/sparse/gsa_on_device): device sparse · Hardware-specific retrieval and cache reshape path for CUDA and Ascend. · `ucm/sparse/gsa_on_device`
- [KVStar](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/sparse/kvstar): retrieval pipeline · Multistep retrieval workflow with task queueing, waiting, and SIMD compute. · `ucm/sparse/kvstar`
- [ReRoPE](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/sparse/rerope): attention mod · RoPE/window extrapolation utilities for modified attention paths. · `ucm/sparse/rerope`

### Storage

Pluggable KV cache persistence and reload backends.

- [Store Factory](https://github.com/modelengine-group/unified-cache-management/blob/HEAD/ucm/store/factory.py): backend selector · Selects concrete KV store implementations and versioned APIs. · `ucm/store/factory.py`
- [Posix Store](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/store/posix): local backend · Local file and AIO-backed persistence backend. · `ucm/store/posix`
- [NFS Store](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/store/nfsstore): shared backend · Shared filesystem backend with device-aware transfer paths. · `ucm/store/nfsstore`
- [PC Store](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/store/pcstore): prefix backend · Prefix-cache-oriented backend for shared-buffer style cache movement. · `ucm/store/pcstore`
- [DS3FS](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/store/ds3fs): distributed backend · Distributed filesystem-style store implementation. · `ucm/store/ds3fs`
- [Pipeline Store](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/store/pipeline): pipeline backend · Pipeline-oriented store connector and loader integration. · `ucm/store/pipeline`
- [Fake/Empty](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/store/fake): null backend · No-op and testing stores used for validation and fallback behavior. · `ucm/store/fake`

### Shared Infra

Cross-cutting native logging, metrics, and transport foundations.

- [Transport](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/shared/trans): device io · Shared device/host buffer transport and stream abstractions across CUDA, Ascend, Simu, and Maca. · `ucm/shared/trans`
- [Metrics](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/shared/metrics): observability · Metrics collection and export used to track cache and retrieval performance. · `ucm/shared/metrics`
- [Logging](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/shared/infra/logger): infra logging · Shared logger foundation for native and Python-facing components. · `ucm/shared/infra/logger`

### Usage

Examples, benchmarks, and test flows that exercise the library.

- [Examples](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/examples): usage scripts · Offline inference, deployment, and metrics examples showing how UCM is launched. · `examples`
- [Benchmarks](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/benchmarks): perf harness · Replay and logging performance entrypoints used to evaluate cache behavior. · `benchmarks`
- [Tests](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/test): validation suite · End-to-end and component coverage for sparse, store, and connector flows. · `test`

### PD Support

Prefill/decode disaggregation support.

- [PD Disagg](https://github.com/modelengine-group/unified-cache-management/tree/HEAD/ucm/pd): disaggregation · Prefill/decode separation support, including proxy server and deployment patterns. · `ucm/pd`

## Connections

- Examples → vLLM: launches · Example entrypoints demonstrate vLLM deployment and offline inference.
- Examples → SGLang: launches · Example entrypoints demonstrate SGLang deployment.
- Examples → MindIE: launches · Example entrypoints demonstrate MindIE deployment.
- Benchmarks → Metrics: measures · Benchmarks inspect logging and trace performance.
- Tests → Store Factory: validates · End-to-end tests cover store selection and save/load behavior.
- Tests → GSA On-Device: validates · Sparse tests exercise device-side retrieval and cache reshape paths.
- vLLM → UCM Connectors: hooks · vLLM patch trees connect host runtime internals to UCM.
- SGLang → UCM Connectors: hooks · SGLang integration uses connector glue for cache control.
- MindIE → UCM Connectors: hooks · MindIE integration routes through UCM connector logic.
- UCM Connectors → Sparse Core: dispatches · Runtime hooks hand control to the active sparse algorithm.
- Sparse Core → Blend: selects · Sparse core can instantiate blend-style cache reuse.
- Sparse Core → ESA: selects · Sparse core can instantiate retrieval-based sparse attention.
- Sparse Core → GSA: selects · Sparse core can instantiate generalized sparse attention.
- Sparse Core → GSA On-Device: selects · Sparse core can route to device-side retrieval kernels.
- Sparse Core → KVStar: selects · Sparse core can instantiate multistep retrieval workflows.
- Sparse Core → ReRoPE: selects · Sparse core can apply RoPE/window extrapolation modifications.
- GSA → Store Factory: loads from · GSA offload/prefetch paths use the KV store abstraction.
- ESA → Store Factory: retrieves from · Retrieval-based sparse attention pulls cached KV through the store layer.
- KVStar → Store Factory: retrieves from · Retrieval tasks load blocks via store backends.
- Store Factory → Posix Store: chooses · Factory can instantiate local POSIX/AIO storage.
- Store Factory → NFS Store: chooses · Factory can instantiate shared filesystem storage.
- Store Factory → PC Store: chooses · Factory can instantiate prefix-cache storage.
- Store Factory → DS3FS: chooses · Factory can instantiate distributed filesystem storage.
- Store Factory → Pipeline Store: chooses · Factory can instantiate pipeline-oriented storage connectors.
- Store Factory → Fake/Empty: chooses · Factory can instantiate no-op or fake stores for fallback and testing.
- Posix Store → Transport: moves data via · POSIX storage relies on shared transport and buffer primitives.
- NFS Store → Transport: moves data via · NFS backend uses device-aware transport and buffers.
- PC Store → Transport: moves data via · PC store uses shared transport abstractions.
- DS3FS → Transport: moves data via · Distributed store shares transport infrastructure.
- Pipeline Store → Transport: moves data via · Pipeline store loads native transport components.
- GSA On-Device → Transport: executes on · Device-side sparse retrieval uses accelerator transport and kernels.
- vLLM → Metrics: emits · Runtime patches expose cache and scheduler metrics.
- Store Factory → Metrics: emits · Store layers contribute cache persistence metrics.
- vLLM → Logging: uses · Integration patches rely on shared logging.
- Store Factory → Logging: uses · Store implementations use shared logging infrastructure.
- PD Disagg → Store Factory: separates with · PD disaggregation relies on cache persistence and reload boundaries.
- PD Disagg → UCM Connectors: coordinates · Proxy/disaggregated flows coordinate through the UCM integration layer.
