# badbat4560/nvfp4-qsa architecture

> Architecture diagram of the GitHub repository badbat4560/nvfp4-qsa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/badbat4560/nvfp4-qsa
- Repository: https://github.com/badbat4560/nvfp4-qsa

## Overview

NVFP4 QSA is an experimental packed KV-cache component for Qwen serving: BF16 K/V is quantized into a compact paged cache, then selected tokens are read by sparse-attention kernels. The portable package exposes guarded cache operations and an independent codec; separate vLLM integration work adapts serving and planner accounting. Historical validation code compares kernels and formats against references and reports metrics. The README documents paired server evaluation and a distinct, not fully validated public port. Some integration internals and benchmark wiring were not excerpted, so their call edges are omitted.

Main entry points: [`slice5/src/w_validate.py`](https://github.com/badbat4560/nvfp4-qsa/blob/main/slice5/src/w_validate.py) (Write validation), [`src/run_validation.py`](https://github.com/badbat4560/nvfp4-qsa/blob/main/src/run_validation.py) (Validation runner), [`python/nvfp4_qsa/__init__.py`](https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/__init__.py) (Package API), [`integration/snapshot/qsa.py`](https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/snapshot/qsa.py) (Serving snapshot)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_portable["Portable cache"]
  node_api["Package API<br/>[__init__.py]"]
  node_cache["Cache operations<br/>[cache.py]"]
  node_kernels["Triton kernels<br/>[kernels.py]"]
  node_codec["Independent codec<br/>[oracle.py]"]
end

subgraph group_integration["Serving integration"]
  node_publicport["Public port candidate<br/>[qsa.py]"]
  node_snapshot["Serving snapshot<br/>[qsa.py]"]
  node_snapshotkernel["Snapshot NVFP4<br/>[nvfp4_qsa.py]"]
  node_interface["Snapshot interface<br/>[interface.py]"]
end

subgraph group_validation["Kernel validation"]
  node_legacyoracle["Validation codec<br/>[nvfp4_oracle.py]"]
  node_legacykernel["Sparse read kernel<br/>[nvfp4_kernel.py]"]
  node_qsa_harness["FP8 harness<br/>[qsa_fp8_harness.py]"]
  node_reference["QSA reference<br/>[qsa_ref.py]"]
  node_validationrun["Validation runner<br/>[run_validation.py]"]
  node_metrics["Comparison metrics<br/>[metrics.py]"]
  node_writekernel["Write kernel<br/>[write_kernel.py]"]
  node_writevalidation["Write validation<br/>[w_validate.py]"]
end

subgraph group_deployment["Deployment support"]
  node_sanitizer["B12x sanitizer<br/>[b12x_sanitize.py]"]
end

node_user(("Python caller"))
node_cuda{{"CUDA runtime"}}
node_vllm{{"vLLM serving"}}
node_results["Evaluation observations"]

node_user -->|"uses"| node_api
node_api -->|"exports"| node_cache
node_api -->|"exports"| node_codec
node_cache -->|"calls"| node_kernels
node_kernels -.->|"executes on"| node_cuda
node_vllm -.->|"port candidate"| node_publicport
node_vllm -.->|"serving snapshot"| node_snapshot
node_snapshot -.->|"configures"| node_snapshotkernel
node_snapshot -.->|"uses"| node_interface
node_validationrun -->|"imports"| node_legacyoracle
node_validationrun -->|"imports"| node_legacykernel
node_validationrun -->|"imports"| node_reference
node_validationrun -->|"imports"| node_metrics
node_validationrun -->|"imports"| node_qsa_harness
node_legacykernel -->|"imports"| node_qsa_harness
node_writevalidation -->|"calls"| node_writekernel
node_writevalidation -->|"imports"| node_legacyoracle
node_writevalidation -->|"imports"| node_legacykernel
node_writevalidation -->|"imports"| node_qsa_harness
node_writevalidation -->|"imports"| node_reference
node_writevalidation -->|"imports"| node_metrics
node_sanitizer -->|"patches wrapper"| node_vllm
node_results -->|"reports"| node_user

click node_api "https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/__init__.py"
click node_cache "https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/cache.py"
click node_kernels "https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/kernels.py"
click node_codec "https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/oracle.py"
click node_publicport "https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/public/qsa.py"
click node_snapshot "https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/snapshot/qsa.py"
click node_snapshotkernel "https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/snapshot/nvfp4_qsa.py"
click node_interface "https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/snapshot/interface.py"
click node_legacyoracle "https://github.com/badbat4560/nvfp4-qsa/blob/main/src/nvfp4_oracle.py"
click node_legacykernel "https://github.com/badbat4560/nvfp4-qsa/blob/main/src/nvfp4_kernel.py"
click node_qsa_harness "https://github.com/badbat4560/nvfp4-qsa/blob/main/src/qsa_fp8_harness.py"
click node_reference "https://github.com/badbat4560/nvfp4-qsa/blob/main/src/qsa_ref.py"
click node_validationrun "https://github.com/badbat4560/nvfp4-qsa/blob/main/src/run_validation.py"
click node_metrics "https://github.com/badbat4560/nvfp4-qsa/blob/main/src/metrics.py"
click node_writekernel "https://github.com/badbat4560/nvfp4-qsa/blob/main/slice5/src/write_kernel.py"
click node_writevalidation "https://github.com/badbat4560/nvfp4-qsa/blob/main/slice5/src/w_validate.py"
click node_sanitizer "https://github.com/badbat4560/nvfp4-qsa/blob/main/deployment/vllm/native_nvfp4/b12x_sanitize.py"
click node_results "https://github.com/badbat4560/nvfp4-qsa/tree/main/results/2026-09-25"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_api,node_cache,node_kernels,node_codec toneBlue
class node_publicport,node_snapshot,node_snapshotkernel,node_interface toneAmber
class node_legacyoracle,node_legacykernel,node_qsa_harness,node_reference,node_validationrun,node_metrics,node_writekernel,node_writevalidation toneMint
class node_sanitizer toneRose
class node_user,node_cuda,node_vllm toneIndigo
class node_results toneTeal
```

## Components

### Portable cache

- [Package API](https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/__init__.py): `python/nvfp4_qsa/__init__.py`
- [Cache operations](https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/cache.py): `python/nvfp4_qsa/cache.py`
- [Triton kernels](https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/kernels.py): `python/nvfp4_qsa/kernels.py`
- [Independent codec](https://github.com/badbat4560/nvfp4-qsa/blob/main/python/nvfp4_qsa/oracle.py): `python/nvfp4_qsa/oracle.py`

### Serving integration

- [Public port candidate](https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/public/qsa.py): `integration/public/qsa.py`
- [Serving snapshot](https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/snapshot/qsa.py): `integration/snapshot/qsa.py`
- [Snapshot NVFP4](https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/snapshot/nvfp4_qsa.py): `integration/snapshot/nvfp4_qsa.py`
- [Snapshot interface](https://github.com/badbat4560/nvfp4-qsa/blob/main/integration/snapshot/interface.py): `integration/snapshot/interface.py`

### Kernel validation

- [Validation codec](https://github.com/badbat4560/nvfp4-qsa/blob/main/src/nvfp4_oracle.py): `src/nvfp4_oracle.py`
- [Sparse read kernel](https://github.com/badbat4560/nvfp4-qsa/blob/main/src/nvfp4_kernel.py): `src/nvfp4_kernel.py`
- [FP8 harness](https://github.com/badbat4560/nvfp4-qsa/blob/main/src/qsa_fp8_harness.py): `src/qsa_fp8_harness.py`
- [QSA reference](https://github.com/badbat4560/nvfp4-qsa/blob/main/src/qsa_ref.py): `src/qsa_ref.py`
- [Validation runner](https://github.com/badbat4560/nvfp4-qsa/blob/main/src/run_validation.py): `src/run_validation.py`
- [Comparison metrics](https://github.com/badbat4560/nvfp4-qsa/blob/main/src/metrics.py): `src/metrics.py`
- [Write kernel](https://github.com/badbat4560/nvfp4-qsa/blob/main/slice5/src/write_kernel.py): `slice5/src/write_kernel.py`
- [Write validation](https://github.com/badbat4560/nvfp4-qsa/blob/main/slice5/src/w_validate.py): `slice5/src/w_validate.py`

### Deployment support

- [B12x sanitizer](https://github.com/badbat4560/nvfp4-qsa/blob/main/deployment/vllm/native_nvfp4/b12x_sanitize.py): `deployment/vllm/native_nvfp4/b12x_sanitize.py`

### Other components

- **Python caller**
- **CUDA runtime**
- **vLLM serving**
- [Evaluation observations](https://github.com/badbat4560/nvfp4-qsa/tree/main/results/2026-09-25): `results/2026-09-25`

## Connections

- Python caller → Package API: uses (evidence: `README.md`)
- Package API → Cache operations: exports (evidence: `python/nvfp4_qsa/__init__.py`)
- Package API → Independent codec: exports (evidence: `python/nvfp4_qsa/__init__.py`)
- Cache operations → Triton kernels: calls (evidence: `python/nvfp4_qsa/cache.py`)
- Triton kernels → CUDA runtime: executes on
- vLLM serving → Public port candidate: port candidate (evidence: `README.md`)
- vLLM serving → Serving snapshot: serving snapshot (evidence: `README.md`)
- Serving snapshot → Snapshot NVFP4: configures
- Serving snapshot → Snapshot interface: uses
- Validation runner → Validation codec: imports (evidence: `src/run_validation.py`)
- Validation runner → Sparse read kernel: imports (evidence: `src/run_validation.py`)
- Validation runner → QSA reference: imports (evidence: `src/run_validation.py`)
- Validation runner → Comparison metrics: imports (evidence: `src/run_validation.py`)
- Validation runner → FP8 harness: imports (evidence: `src/run_validation.py`)
- Sparse read kernel → FP8 harness: imports (evidence: `src/nvfp4_kernel.py`)
- Write validation → Write kernel: calls (evidence: `slice5/src/w_validate.py`)
- Write validation → Validation codec: imports (evidence: `slice5/src/w_validate.py`)
- Write validation → Sparse read kernel: imports (evidence: `slice5/src/w_validate.py`)
- Write validation → FP8 harness: imports (evidence: `slice5/src/w_validate.py`)
- Write validation → QSA reference: imports (evidence: `slice5/src/w_validate.py`)
- Write validation → Comparison metrics: imports (evidence: `slice5/src/w_validate.py`)
- B12x sanitizer → vLLM serving: patches wrapper (evidence: `deployment/vllm/native_nvfp4/b12x_sanitize.py`)
- Evaluation observations → Python caller: reports (evidence: `README.md`)
