How prasanthrayudu313/cxl-kv-simulation works
CXL KV-Cache Simulator models LLM request handling and KV placement across GPU HBM and shared CXL memory. Its principal workflow generates requests, admits them under a placement policy, simulates prefill/decode (or split prefill/decode), and records performance and memory metrics. Workload variants, compression, cost modeling, and experiment runners support research comparisons; the dashboard is a separate in-browser visualization. Connections for unsampled modules are limited to documented capabilities and SOURCE INDEX references.
Main entry points: engine.py Coupled Engine · run_all_new.py Experiment Suites · disaggregated_engine.py Disaggregated Engine · config.py Hardware Config
Components 17
Simulation Runtime
- Simulation Runner
run_simulation.py - Coupled Engine
core/engine.py - Disaggregated Engine
core/disaggregated_engine.py - Request & Metrics
core/models.py - Hardware Config
config.py
KV Placement
- Placement Policies
policies/placement.py
Workload Inputs
- Synthetic Workload
workload/generator.py - ShareGPT Workload
workload/sharegpt.py - Multi-turn Workload
workload/multiturn.py
Research Evaluation
- Experiment Suites
experiments/run_all_new.py - Extended Experiments
experiments/run_extended.py - Result Plotting
experiments/plot_results.py - KV Compression
core/compression.py - Compression Accuracy
core/compression_accuracy.py - Cost Model
core/cost_model.py
Visualization
- Browser Dashboard
dashboard/index.html
Other components
- Researcher
Connections 27
- Researcher → Simulation Runner: runs
README.md - Simulation Runner → Coupled Engine: creates
run_simulation.py - Coupled Engine → Synthetic Workload: generates requests
core/engine.py - Coupled Engine → Placement Policies: admits and evicts
core/engine.py - Coupled Engine → Request & Metrics: tracks request state
core/engine.py - Coupled Engine → Hardware Config: uses parameters
core/engine.py - Disaggregated Engine → Synthetic Workload: generates requests
core/disaggregated_engine.py - Disaggregated Engine → Placement Policies: admits requests
core/disaggregated_engine.py - Disaggregated Engine → Request & Metrics: tracks request state
core/disaggregated_engine.py - Disaggregated Engine → Hardware Config: uses parameters
core/disaggregated_engine.py - Synthetic Workload → Request & Metrics: creates requests
workload/generator.py - Synthetic Workload → Hardware Config: reads distributions
workload/generator.py - ShareGPT Workload → Request & Metrics: creates requests
workload/sharegpt.py - Multi-turn Workload → Hardware Config: reads workload settings
workload/multiturn.py - Experiment Suites → Coupled Engine: runs comparisons
experiments/run_all_new.py - Experiment Suites → Disaggregated Engine: compares architectures
experiments/run_all_new.py - Experiment Suites → KV Compression: sweeps compression
experiments/run_all_new.py - Experiment Suites → Cost Model: uses cost model
experiments/run_all_new.py - Experiment Suites → Multi-turn Workload: runs sessions
experiments/run_all_new.py - Experiment Suites → Placement Policies: compares policies
experiments/run_all_new.py - Extended Experiments → ShareGPT Workload: runs calibrated traces
experiments/run_extended.py - Extended Experiments → Multi-turn Workload: runs sessions
experiments/run_extended.py - Extended Experiments → Coupled Engine: runs simulations
experiments/run_extended.py - Compression Accuracy → KV Compression: evaluates accuracy
- Researcher → Browser Dashboard: opens
README.md - Browser Dashboard → Hardware Config: aligns constants
README.md - Researcher → Result Plotting: plots results
README.md