zhengjiahong-ai/llm-kv-cache-optimization

KVopt is a research scaffold for cost-aware KV-cache eviction in LLM serving, with Continuum-style retention and scheduling as supporting system mechanisms. The principal workflow is workload/request handling, cache-pressure victim selection, and block allocation or eviction; the vLLM path adapts policy decisions to native cache structures. Continuum coordinates lifecycle observations, histories, TTL-based retention, and scheduling candidates. The README documents broader capabilities than the sampled source confirms; unsampled wiring is shown only where documented, and the proposed cost-aware policy itself has no identifiable implementation in this tree.