How hidedabush/neural-network-quantization-study works
This repository is a neural-network quantization research study: experiments load models and data, compare floating-point baselines with quantized inference, and examine accuracy and performance tradeoffs. Its runtime includes Python PTQ and sensitivity experiments alongside custom CUDA quantization and INT8 GEMM kernels; results can be plotted for a paper-style report. The README also describes QAT, mixed precision, and TensorRT comparison, but those implementations are not present in the supplied tree, so their links are not inferred. Some CUDA and reporting paths are documented but their source was not sampled.
Main entry points: run_ptq.py PTQ Experiment · layer_sweep.py Layer Sensitivity Sweep · model_loader.py Model Loader · dataset_loader.py CIFAR-10 Loader
Components 18
Model Experiments
- PTQ Experiment
experiments/ptq/run_ptq.py - Weight Calibration
experiments/ptq/run_ptq.py - Linear Quantization
experiments/ptq/run_ptq.py - Accuracy And Latency
experiments/ptq/run_ptq.py - Layer Sensitivity Sweep
experiments/sensitivity/layer_sweep.py - QAT Capability
experiments - Mixed Precision Capability
experiments
CUDA Kernels
- Quantize Kernels
cuda_kernels/src/quantize.cu - INT8 GEMM
cuda_kernels/src/gemm_int8.cu - Kernel Benchmark
cuda_kernels/src/benchmark.cu
Models And Data
- Model Loader
models/model_loader.py - CIFAR-10 Loader
datasets/dataset_loader.py
Results And Analysis
- Result Plotting
tools/plot_results.py - Technical Report
docs/paper/report_draft.md
Other components
- Researcher
- Model Providers
- CIFAR-10 Dataset
- TensorRT Baseline
Connections 18
- Researcher → PTQ Experiment: runs
- Researcher → Layer Sensitivity Sweep: runs
- PTQ Experiment → CIFAR-10 Loader: loads data
- PTQ Experiment → Model Loader: loads model
- Layer Sensitivity Sweep → CIFAR-10 Loader: loads data
- Layer Sensitivity Sweep → Model Loader: loads model
- CIFAR-10 Loader → CIFAR-10 Dataset: downloads
- Model Loader → Model Providers: loads weights
- PTQ Experiment → Weight Calibration: calibrates
- Weight Calibration → Linear Quantization: provides scales
- PTQ Experiment → Linear Quantization: replaces layers
- PTQ Experiment → Accuracy And Latency: evaluates models
- Layer Sensitivity Sweep → Accuracy And Latency: measures accuracy
- PTQ Experiment → Quantize Kernels: planned integration
- Quantize Kernels → INT8 GEMM: kernel component
- Researcher → Kernel Benchmark: runs
- Researcher → Result Plotting: generates figures
- Result Plotting → Technical Report: supports findings