How badbat4560/nvfp4-qsa works
NVFP4 QSA is an experimental packed KV-cache component for Qwen serving: BF16 K/V is quantized into a compact paged cache, then selected tokens are read by sparse-attention kernels. The portable package exposes guarded cache operations and an independent codec; separate vLLM integration work adapts serving and planner accounting. Historical validation code compares kernels and formats against references and reports metrics. The README documents paired server evaluation and a distinct, not fully validated public port. Some integration internals and benchmark wiring were not excerpted, so their call edges are omitted.
Main entry points: w_validate.py Write validation · run_validation.py Validation runner · __init__.py Package API · qsa.py Serving snapshot
Components 21
Portable cache
- Package API
python/nvfp4_qsa/__init__.py - Cache operations
python/nvfp4_qsa/cache.py - Triton kernels
python/nvfp4_qsa/kernels.py - Independent codec
python/nvfp4_qsa/oracle.py
Serving integration
- Public port candidate
integration/public/qsa.py - Serving snapshot
integration/snapshot/qsa.py - Snapshot NVFP4
integration/snapshot/nvfp4_qsa.py - Snapshot interface
integration/snapshot/interface.py
Kernel validation
- Validation codec
src/nvfp4_oracle.py - Sparse read kernel
src/nvfp4_kernel.py - FP8 harness
src/qsa_fp8_harness.py - QSA reference
src/qsa_ref.py - Validation runner
src/run_validation.py - Comparison metrics
src/metrics.py - Write kernel
slice5/src/write_kernel.py - Write validation
slice5/src/w_validate.py
Deployment support
- B12x sanitizer
deployment/vllm/native_nvfp4/b12x_sanitize.py
Other components
- Python caller
- CUDA runtime
- vLLM serving
- Evaluation observations
results/2026-09-25
Connections 23
- Python caller → Package API: uses
README.md - Package API → Cache operations: exports
python/nvfp4_qsa/__init__.py - Package API → Independent codec: exports
python/nvfp4_qsa/__init__.py - Cache operations → Triton kernels: calls
python/nvfp4_qsa/cache.py - Triton kernels → CUDA runtime: executes on
- vLLM serving → Public port candidate: port candidate
README.md - vLLM serving → Serving snapshot: serving snapshot
README.md - Serving snapshot → Snapshot NVFP4: configures
- Serving snapshot → Snapshot interface: uses
- Validation runner → Validation codec: imports
src/run_validation.py - Validation runner → Sparse read kernel: imports
src/run_validation.py - Validation runner → QSA reference: imports
src/run_validation.py - Validation runner → Comparison metrics: imports
src/run_validation.py - Validation runner → FP8 harness: imports
src/run_validation.py - Sparse read kernel → FP8 harness: imports
src/nvfp4_kernel.py - Write validation → Write kernel: calls
slice5/src/w_validate.py - Write validation → Validation codec: imports
slice5/src/w_validate.py - Write validation → Sparse read kernel: imports
slice5/src/w_validate.py - Write validation → FP8 harness: imports
slice5/src/w_validate.py - Write validation → QSA reference: imports
slice5/src/w_validate.py - Write validation → Comparison metrics: imports
slice5/src/w_validate.py - B12x sanitizer → vLLM serving: patches wrapper
deployment/vllm/native_nvfp4/b12x_sanitize.py - Evaluation observations → Python caller: reports
README.md