# orbitinfra/orbitkv architecture

> Architecture diagram of the GitHub repository orbitinfra/orbitkv, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/orbitinfra/orbitkv
- Repository: https://github.com/orbitinfra/orbitkv

## Overview

OrbitKV is a reusable KV cache for vLLM and SGLang: engine-side connectors submit cache operations to a Rust Cache Manager, which coordinates cache identity, placement, queries and leases; OrbiTE handles byte movement across GPU, DRAM, SSD and peer hosts. The graph separates engine integration, manager control, cache/storage, and transfer infrastructure. Relationships are limited to README statements and SOURCE INDEX references; unsampled subsystems are represented where documented, without inferring their internal call paths.

Main entry points: [`crates/orbitkv-core/src/transfer_engine.rs`](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/transfer_engine.rs) (Transfer engine), [`python/src/client.rs`](https://github.com/orbitinfra/orbitkv/blob/main/python/src/client.rs) (Python client), [`crates/orbitkv-core/src/engine/query.rs`](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/engine/query.rs) (Query engine), [`crates/orbitkv-core/src/block.rs`](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/block.rs) (Cache blocks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_engines["Engine integrations"]
  node_vllm_connector["vLLM connector<br/>[worker.py]"]
  node_sglang_plugin["SGLang integration<br/>[plugin.py]"]
end

subgraph group_manager["Manager control"]
  node_python_client["Python client<br/>[client.rs]"]
  node_registration["Registration<br/>[registration.rs]"]
  node_recovery["Recovery<br/>[recovery.rs]"]
  node_local_restore["Local restore<br/>[local_restore.rs]"]
  node_session_endpoint["Session endpoint<br/>[session.rs]"]
  node_query_control["Query control<br/>[query_control.rs]"]
  node_pending["Pending operations<br/>[pending.rs]"]
  node_shards["Cache shards<br/>[shards.rs]"]
end

subgraph group_cache["Cache and storage"]
  node_engine_query["Query engine<br/>[query.rs]"]
  node_blocks["Cache blocks<br/>[block.rs]"]
  node_memory_pool["Memory pool<br/>[pool.rs]"]
  node_ssd_storage[("SSD storage")]
  node_state_bundle["State bundles<br/>[bundle.rs]"]
  node_catalog[("Catalog index<br/>[index.rs]")]
  node_dram[("DRAM tier<br/>[mod.rs]")]
end

subgraph group_transfer["Transfer and distribution"]
  node_channel_transport["Channel transport<br/>[transport.rs]"]
  node_transfer_engine["Transfer engine<br/>[transfer_engine.rs]"]
  node_local_transfer["Local transfer<br/>[local.rs]"]
  node_peer_segments["Peer segments<br/>[segment.rs]"]
  node_protocol["Engine protocol<br/>[engine.proto]"]
end

node_vllm(("vLLM engine"))
node_sglang(("SGLang engine"))
node_peer_hosts(("Peer hosts"))

node_vllm -->|"uses connector"| node_vllm_connector
node_sglang -->|"uses integration"| node_sglang_plugin
node_vllm_connector -->|"submits operations"| node_python_client
node_python_client -->|"delegates to"| node_registration
node_python_client -->|"delegates to"| node_recovery
node_python_client -->|"uses restore"| node_local_restore
node_session_endpoint -->|"routes to"| node_query_control
node_session_endpoint -->|"uses shards"| node_shards
node_query_control -->|"uses pending"| node_pending
node_query_control -.->|"queries engine"| node_engine_query
node_engine_query -->|"uses blocks"| node_blocks
node_blocks -->|"uses pool"| node_memory_pool
node_blocks -->|"uses SSD"| node_ssd_storage
node_blocks -->|"uses transfer"| node_local_transfer
node_memory_pool -->|"supports transfer"| node_local_transfer
node_transfer_engine -->|"uses pool"| node_memory_pool
node_transfer_engine -->|"uses segments"| node_peer_segments
node_peer_segments -->|"moves bytes"| node_peer_hosts
node_transfer_engine -.->|"moves cache data"| node_dram
node_transfer_engine -.->|"moves cache data"| node_ssd_storage
node_channel_transport -.->|"communicates with"| node_peer_hosts
node_engine_query -.->|"uses cache identity"| node_state_bundle
node_engine_query -.->|"uses catalog"| node_catalog
node_session_endpoint -.->|"serves protocol"| node_protocol

click node_vllm_connector "https://github.com/orbitinfra/orbitkv/blob/main/python/orbitkv/vllm/worker.py"
click node_sglang_plugin "https://github.com/orbitinfra/orbitkv/blob/main/python/orbitkv/sglang/plugin.py"
click node_python_client "https://github.com/orbitinfra/orbitkv/blob/main/python/src/client.rs"
click node_registration "https://github.com/orbitinfra/orbitkv/blob/main/python/src/registration.rs"
click node_recovery "https://github.com/orbitinfra/orbitkv/blob/main/python/src/recovery.rs"
click node_local_restore "https://github.com/orbitinfra/orbitkv/blob/main/python/src/local_restore.rs"
click node_session_endpoint "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/endpoint/session.rs"
click node_query_control "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/cache/query_control.rs"
click node_pending "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/cache/pending.rs"
click node_shards "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/cache/shards.rs"
click node_engine_query "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/engine/query.rs"
click node_blocks "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/block.rs"
click node_memory_pool "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/memory/pool.rs"
click node_ssd_storage "https://github.com/orbitinfra/orbitkv/tree/main/crates/orbitkv-core/src/storage/ssd"
click node_state_bundle "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-state/src/bundle.rs"
click node_catalog "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-catalog/src/index.rs"
click node_channel_transport "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-channel/src/transport.rs"
click node_transfer_engine "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/transfer_engine.rs"
click node_local_transfer "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/transfer/local.rs"
click node_peer_segments "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/peer/segment.rs"
click node_protocol "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-proto/proto/engine.proto"
click node_dram "https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/storage/dram/mod.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_vllm_connector,node_sglang_plugin toneBlue
class node_python_client,node_registration,node_recovery,node_local_restore,node_session_endpoint,node_query_control,node_pending,node_shards toneAmber
class node_engine_query,node_blocks,node_memory_pool,node_ssd_storage,node_state_bundle,node_catalog,node_dram toneMint
class node_channel_transport,node_transfer_engine,node_local_transfer,node_peer_segments,node_protocol toneRose
class node_vllm,node_sglang,node_peer_hosts toneIndigo
```

## Components

### Engine integrations

- [vLLM connector](https://github.com/orbitinfra/orbitkv/blob/main/python/orbitkv/vllm/worker.py): `python/orbitkv/vllm/worker.py`
- [SGLang integration](https://github.com/orbitinfra/orbitkv/blob/main/python/orbitkv/sglang/plugin.py): `python/orbitkv/sglang/plugin.py`

### Manager control

- [Python client](https://github.com/orbitinfra/orbitkv/blob/main/python/src/client.rs): `python/src/client.rs`
- [Registration](https://github.com/orbitinfra/orbitkv/blob/main/python/src/registration.rs): `python/src/registration.rs`
- [Recovery](https://github.com/orbitinfra/orbitkv/blob/main/python/src/recovery.rs): `python/src/recovery.rs`
- [Local restore](https://github.com/orbitinfra/orbitkv/blob/main/python/src/local_restore.rs): `python/src/local_restore.rs`
- [Session endpoint](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/endpoint/session.rs): `crates/orbitkv-server/src/endpoint/session.rs`
- [Query control](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/cache/query_control.rs): `crates/orbitkv-server/src/cache/query_control.rs`
- [Pending operations](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/cache/pending.rs): `crates/orbitkv-server/src/cache/pending.rs`
- [Cache shards](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-server/src/cache/shards.rs): `crates/orbitkv-server/src/cache/shards.rs`

### Cache and storage

- [Query engine](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/engine/query.rs): `crates/orbitkv-core/src/engine/query.rs`
- [Cache blocks](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/block.rs): `crates/orbitkv-core/src/block.rs`
- [Memory pool](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/memory/pool.rs): `crates/orbitkv-core/src/memory/pool.rs`
- [SSD storage](https://github.com/orbitinfra/orbitkv/tree/main/crates/orbitkv-core/src/storage/ssd): `crates/orbitkv-core/src/storage/ssd`
- [State bundles](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-state/src/bundle.rs): `crates/orbitkv-state/src/bundle.rs`
- [Catalog index](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-catalog/src/index.rs): `crates/orbitkv-catalog/src/index.rs`
- [DRAM tier](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/storage/dram/mod.rs): `crates/orbitkv-core/src/storage/dram/mod.rs`

### Transfer and distribution

- [Channel transport](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-channel/src/transport.rs): `crates/orbitkv-channel/src/transport.rs`
- [Transfer engine](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/transfer_engine.rs): `crates/orbitkv-core/src/transfer_engine.rs`
- [Local transfer](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/transfer/local.rs): `crates/orbitkv-core/src/transfer/local.rs`
- [Peer segments](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-core/src/peer/segment.rs): `crates/orbitkv-core/src/peer/segment.rs`
- [Engine protocol](https://github.com/orbitinfra/orbitkv/blob/main/crates/orbitkv-proto/proto/engine.proto): `crates/orbitkv-proto/proto/engine.proto`

### Other components

- **vLLM engine**
- **SGLang engine**
- **Peer hosts**

## Connections

- vLLM engine → vLLM connector: uses connector (evidence: `README.md`)
- SGLang engine → SGLang integration: uses integration (evidence: `README.md`)
- vLLM connector → Python client: submits operations (evidence: `python/orbitkv/vllm/worker.py`)
- Python client → Registration: delegates to (evidence: `python/src/client.rs`)
- Python client → Recovery: delegates to (evidence: `python/src/client.rs`)
- Python client → Local restore: uses restore (evidence: `python/src/client.rs`)
- Session endpoint → Query control: routes to (evidence: `crates/orbitkv-server/src/endpoint/session.rs`)
- Session endpoint → Cache shards: uses shards (evidence: `crates/orbitkv-server/src/endpoint/session.rs`)
- Query control → Pending operations: uses pending (evidence: `crates/orbitkv-server/src/cache/query_control.rs`)
- Query control → Query engine: queries engine
- Query engine → Cache blocks: uses blocks (evidence: `crates/orbitkv-core/src/engine/query.rs`)
- Cache blocks → Memory pool: uses pool (evidence: `crates/orbitkv-core/src/block.rs`)
- Cache blocks → SSD storage: uses SSD (evidence: `crates/orbitkv-core/src/block.rs`)
- Cache blocks → Local transfer: uses transfer (evidence: `crates/orbitkv-core/src/block.rs`)
- Memory pool → Local transfer: supports transfer (evidence: `crates/orbitkv-core/src/memory/pool.rs`)
- Transfer engine → Memory pool: uses pool (evidence: `crates/orbitkv-core/src/transfer_engine.rs`)
- Transfer engine → Peer segments: uses segments (evidence: `crates/orbitkv-core/src/transfer_engine.rs`)
- Peer segments → Peer hosts: moves bytes (evidence: `README.md`)
- Transfer engine → DRAM tier: moves cache data
- Transfer engine → SSD storage: moves cache data
- Channel transport → Peer hosts: communicates with
- Query engine → State bundles: uses cache identity
- Query engine → Catalog index: uses catalog
- Session endpoint → Engine protocol: serves protocol
