How locuslab/wanda works
Wanda is a PyTorch research toolkit for pruning large language models by combining weight magnitudes with input-activation norms. The primary workflow loads a pretrained model, gathers calibration activations, prunes its layers, then measures perplexity and optionally runs zero-shot tasks or saves the result. The repository also documents OBS-style ablations, OPT support, image-classifier pruning, and LoRA fine-tuning. The graph emphasizes evidenced runtime links; some branches are represented from the README and source index, since their implementation was not excerpted.
Main entry points: main.py LLM workflow · prune.py LLaMA pruning · prune_opt.py OPT pruning · image_classifiers/main.py Classifier workflow
Components 22
LLM pruning
- LLM workflow
main.py - OPT workflow
main_opt.py - Pretrained model
- Calibration data
lib/data.py - Pruned model
Pruning methods
- LLaMA pruning
lib/prune.py - OPT pruning
lib/prune_opt.py - Activation statistics
lib/layerwrapper.py - SparseGPT
lib/sparsegpt.py - OBS ablation
lib/ablate.py
Evaluation
- Perplexity
lib/eval.py - Zero-shot tasks
lib/eval.py - LM evaluation harness
Image classifiers
- Classifier workflow
image_classifiers/main.py - Image datasets
image_classifiers/datasets.py - Vision architectures
image_classifiers/models - Train and evaluate
image_classifiers/engine.py - Classifier pruning
image_classifiers/prune_utils.py
Fine-tuning
- Dense fine-tuning
dense_ft/trainer.py - Sparse fine-tuning
dense_ft/sparse_trainer.py - LoRA fine-tuning
lora_ft/finetune_lm.py
Other components
- Researcher
Connections 22
- Researcher → LLM workflow: runs
README.md - LLM workflow → Pretrained model: loads
main.py - LLM workflow → LLaMA pruning: dispatches
main.py - LLM workflow → Perplexity: evaluates
main.py - LLM workflow → Zero-shot tasks: evaluates
main.py - LLM workflow → Pruned model: saves
main.py - OPT workflow → OPT pruning: dispatches
main_opt.py - OPT workflow → Perplexity: evaluates
main_opt.py - LLaMA pruning → Calibration data: loads
lib/prune.py - OPT pruning → Calibration data: loads
lib/prune_opt.py - LLaMA pruning → Activation statistics: collects activations
lib/prune.py - LLaMA pruning → SparseGPT: uses
lib/prune.py - LLaMA pruning → OBS ablation: uses
lib/prune.py - OPT pruning → SparseGPT: uses
lib/prune_opt.py - OPT pruning → OBS ablation: uses
lib/prune_opt.py - Perplexity → Calibration data: loads data
lib/eval.py - Zero-shot tasks → LM evaluation harness: calls
lib/eval.py - Researcher → Classifier workflow: runs
README.md - Classifier workflow → Image datasets: builds datasets
image_classifiers/main.py - Classifier workflow → Vision architectures: selects model
image_classifiers/main.py - Train and evaluate → Classifier pruning: uses
image_classifiers/engine.py - Researcher → LoRA fine-tuning: runs
README.md