How orbitinfra/orbitkv works
OrbitKV is a reusable KV cache for vLLM and SGLang: engine-side connectors submit cache operations to a Rust Cache Manager, which coordinates cache identity, placement, queries and leases; OrbiTE handles byte movement across GPU, DRAM, SSD and peer hosts. The graph separates engine integration, manager control, cache/storage, and transfer infrastructure. Relationships are limited to README statements and SOURCE INDEX references; unsampled subsystems are represented where documented, without inferring their internal call paths.
Main entry points: transfer_engine.rs Transfer engine · client.rs Python client · query.rs Query engine · block.rs Cache blocks
Components 25
Engine integrations
- vLLM connector
python/orbitkv/vllm/worker.py - SGLang integration
python/orbitkv/sglang/plugin.py
Manager control
- Python client
python/src/client.rs - Registration
python/src/registration.rs - Recovery
python/src/recovery.rs - Local restore
python/src/local_restore.rs - Session endpoint
crates/orbitkv-server/src/endpoint/session.rs - Query control
crates/orbitkv-server/src/cache/query_control.rs - Pending operations
crates/orbitkv-server/src/cache/pending.rs - Cache shards
crates/orbitkv-server/src/cache/shards.rs
Cache and storage
- Query engine
crates/orbitkv-core/src/engine/query.rs - Cache blocks
crates/orbitkv-core/src/block.rs - Memory pool
crates/orbitkv-core/src/memory/pool.rs - SSD storage
crates/orbitkv-core/src/storage/ssd - State bundles
crates/orbitkv-state/src/bundle.rs - Catalog index
crates/orbitkv-catalog/src/index.rs - DRAM tier
crates/orbitkv-core/src/storage/dram/mod.rs
Transfer and distribution
- Channel transport
crates/orbitkv-channel/src/transport.rs - Transfer engine
crates/orbitkv-core/src/transfer_engine.rs - Local transfer
crates/orbitkv-core/src/transfer/local.rs - Peer segments
crates/orbitkv-core/src/peer/segment.rs - Engine protocol
crates/orbitkv-proto/proto/engine.proto
Other components
- vLLM engine
- SGLang engine
- Peer hosts
Connections 24
- vLLM engine → vLLM connector: uses connector
README.md - SGLang engine → SGLang integration: uses integration
README.md - vLLM connector → Python client: submits operations
python/orbitkv/vllm/worker.py - Python client → Registration: delegates to
python/src/client.rs - Python client → Recovery: delegates to
python/src/client.rs - Python client → Local restore: uses restore
python/src/client.rs - Session endpoint → Query control: routes to
crates/orbitkv-server/src/endpoint/session.rs - Session endpoint → Cache shards: uses shards
crates/orbitkv-server/src/endpoint/session.rs - Query control → Pending operations: uses pending
crates/orbitkv-server/src/cache/query_control.rs - Query control → Query engine: queries engine
- Query engine → Cache blocks: uses blocks
crates/orbitkv-core/src/engine/query.rs - Cache blocks → Memory pool: uses pool
crates/orbitkv-core/src/block.rs - Cache blocks → SSD storage: uses SSD
crates/orbitkv-core/src/block.rs - Cache blocks → Local transfer: uses transfer
crates/orbitkv-core/src/block.rs - Memory pool → Local transfer: supports transfer
crates/orbitkv-core/src/memory/pool.rs - Transfer engine → Memory pool: uses pool
crates/orbitkv-core/src/transfer_engine.rs - Transfer engine → Peer segments: uses segments
crates/orbitkv-core/src/transfer_engine.rs - Peer segments → Peer hosts: moves bytes
README.md - Transfer engine → DRAM tier: moves cache data
- Transfer engine → SSD storage: moves cache data
- Channel transport → Peer hosts: communicates with
- Query engine → State bundles: uses cache identity
- Query engine → Catalog index: uses catalog
- Session endpoint → Engine protocol: serves protocol