# locuslab/wanda architecture

> Architecture diagram of the GitHub repository locuslab/wanda, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/locuslab/wanda
- Repository: https://github.com/locuslab/wanda

## Overview

Wanda is a PyTorch research toolkit for pruning large language models by combining weight magnitudes with input-activation norms. The primary workflow loads a pretrained model, gathers calibration activations, prunes its layers, then measures perplexity and optionally runs zero-shot tasks or saves the result. The repository also documents OBS-style ablations, OPT support, image-classifier pruning, and LoRA fine-tuning. The graph emphasizes evidenced runtime links; some branches are represented from the README and source index, since their implementation was not excerpted.

Main entry points: [`main.py`](https://github.com/locuslab/wanda/blob/main/main.py) (LLM workflow), [`lib/prune.py`](https://github.com/locuslab/wanda/blob/main/lib/prune.py) (LLaMA pruning), [`lib/prune_opt.py`](https://github.com/locuslab/wanda/blob/main/lib/prune_opt.py) (OPT pruning), [`image_classifiers/main.py`](https://github.com/locuslab/wanda/blob/main/image_classifiers/main.py) (Classifier workflow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_llm["LLM pruning"]
  node_entry["LLM workflow<br/>[main.py]"]
  node_opt_entry["OPT workflow<br/>[main_opt.py]"]
  node_pretrained["Pretrained model"]
  node_calibration["Calibration data<br/>[data.py]"]
  node_sparse_model["Pruned model"]
end

subgraph group_methods["Pruning methods"]
  node_pruner["LLaMA pruning<br/>[prune.py]"]
  node_opt_pruner["OPT pruning<br/>[prune_opt.py]"]
  node_wanda_stats["Activation statistics<br/>[layerwrapper.py]"]
  node_sparsegpt["SparseGPT<br/>[sparsegpt.py]"]
  node_ablation["OBS ablation<br/>[ablate.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_ppl["Perplexity<br/>[eval.py]"]
  node_zero_shot["Zero-shot tasks<br/>[eval.py]"]
  node_harness["LM evaluation harness"]
end

subgraph group_vision["Image classifiers"]
  node_classifier_entry["Classifier workflow<br/>[main.py]"]
  node_image_data["Image datasets<br/>[datasets.py]"]
  node_image_models["Vision architectures"]
  node_classifier_engine["Train and evaluate<br/>[engine.py]"]
  node_image_pruning["Classifier pruning<br/>[prune_utils.py]"]
end

subgraph group_finetuning["Fine-tuning"]
  node_dense_tune["Dense fine-tuning<br/>[trainer.py]"]
  node_sparse_tune["Sparse fine-tuning<br/>[sparse_trainer.py]"]
  node_lora_tune["LoRA fine-tuning<br/>[finetune_lm.py]"]
end

node_user(("Researcher"))

node_user -->|"runs"| node_entry
node_entry -->|"loads"| node_pretrained
node_entry -->|"dispatches"| node_pruner
node_entry -->|"evaluates"| node_ppl
node_entry -->|"evaluates"| node_zero_shot
node_entry -->|"saves"| node_sparse_model
node_opt_entry -->|"dispatches"| node_opt_pruner
node_opt_entry -->|"evaluates"| node_ppl
node_pruner -->|"loads"| node_calibration
node_opt_pruner -->|"loads"| node_calibration
node_pruner -->|"collects activations"| node_wanda_stats
node_pruner -->|"uses"| node_sparsegpt
node_pruner -->|"uses"| node_ablation
node_opt_pruner -->|"uses"| node_sparsegpt
node_opt_pruner -->|"uses"| node_ablation
node_ppl -->|"loads data"| node_calibration
node_zero_shot -->|"calls"| node_harness
node_user -->|"runs"| node_classifier_entry
node_classifier_entry -->|"builds datasets"| node_image_data
node_classifier_entry -->|"selects model"| node_image_models
node_classifier_engine -->|"uses"| node_image_pruning
node_user -->|"runs"| node_lora_tune

click node_entry "https://github.com/locuslab/wanda/blob/main/main.py"
click node_opt_entry "https://github.com/locuslab/wanda/blob/main/main_opt.py"
click node_calibration "https://github.com/locuslab/wanda/blob/main/lib/data.py"
click node_pruner "https://github.com/locuslab/wanda/blob/main/lib/prune.py"
click node_opt_pruner "https://github.com/locuslab/wanda/blob/main/lib/prune_opt.py"
click node_wanda_stats "https://github.com/locuslab/wanda/blob/main/lib/layerwrapper.py"
click node_sparsegpt "https://github.com/locuslab/wanda/blob/main/lib/sparsegpt.py"
click node_ablation "https://github.com/locuslab/wanda/blob/main/lib/ablate.py"
click node_ppl "https://github.com/locuslab/wanda/blob/main/lib/eval.py"
click node_zero_shot "https://github.com/locuslab/wanda/blob/main/lib/eval.py"
click node_classifier_entry "https://github.com/locuslab/wanda/blob/main/image_classifiers/main.py"
click node_image_data "https://github.com/locuslab/wanda/blob/main/image_classifiers/datasets.py"
click node_image_models "https://github.com/locuslab/wanda/tree/main/image_classifiers/models"
click node_classifier_engine "https://github.com/locuslab/wanda/blob/main/image_classifiers/engine.py"
click node_image_pruning "https://github.com/locuslab/wanda/blob/main/image_classifiers/prune_utils.py"
click node_dense_tune "https://github.com/locuslab/wanda/blob/main/dense_ft/trainer.py"
click node_sparse_tune "https://github.com/locuslab/wanda/blob/main/dense_ft/sparse_trainer.py"
click node_lora_tune "https://github.com/locuslab/wanda/blob/main/lora_ft/finetune_lm.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_entry,node_opt_entry,node_pretrained,node_calibration,node_sparse_model toneBlue
class node_pruner,node_opt_pruner,node_wanda_stats,node_sparsegpt,node_ablation toneAmber
class node_ppl,node_zero_shot,node_harness toneMint
class node_classifier_entry,node_image_data,node_image_models,node_classifier_engine,node_image_pruning toneRose
class node_dense_tune,node_sparse_tune,node_lora_tune,node_user toneIndigo
```

## Components

### LLM pruning

- [LLM workflow](https://github.com/locuslab/wanda/blob/main/main.py): `main.py`
- [OPT workflow](https://github.com/locuslab/wanda/blob/main/main_opt.py): `main_opt.py`
- **Pretrained model**
- [Calibration data](https://github.com/locuslab/wanda/blob/main/lib/data.py): `lib/data.py`
- **Pruned model**

### Pruning methods

- [LLaMA pruning](https://github.com/locuslab/wanda/blob/main/lib/prune.py): `lib/prune.py`
- [OPT pruning](https://github.com/locuslab/wanda/blob/main/lib/prune_opt.py): `lib/prune_opt.py`
- [Activation statistics](https://github.com/locuslab/wanda/blob/main/lib/layerwrapper.py): `lib/layerwrapper.py`
- [SparseGPT](https://github.com/locuslab/wanda/blob/main/lib/sparsegpt.py): `lib/sparsegpt.py`
- [OBS ablation](https://github.com/locuslab/wanda/blob/main/lib/ablate.py): `lib/ablate.py`

### Evaluation

- [Perplexity](https://github.com/locuslab/wanda/blob/main/lib/eval.py): `lib/eval.py`
- [Zero-shot tasks](https://github.com/locuslab/wanda/blob/main/lib/eval.py): `lib/eval.py`
- **LM evaluation harness**

### Image classifiers

- [Classifier workflow](https://github.com/locuslab/wanda/blob/main/image_classifiers/main.py): `image_classifiers/main.py`
- [Image datasets](https://github.com/locuslab/wanda/blob/main/image_classifiers/datasets.py): `image_classifiers/datasets.py`
- [Vision architectures](https://github.com/locuslab/wanda/tree/main/image_classifiers/models): `image_classifiers/models`
- [Train and evaluate](https://github.com/locuslab/wanda/blob/main/image_classifiers/engine.py): `image_classifiers/engine.py`
- [Classifier pruning](https://github.com/locuslab/wanda/blob/main/image_classifiers/prune_utils.py): `image_classifiers/prune_utils.py`

### Fine-tuning

- [Dense fine-tuning](https://github.com/locuslab/wanda/blob/main/dense_ft/trainer.py): `dense_ft/trainer.py`
- [Sparse fine-tuning](https://github.com/locuslab/wanda/blob/main/dense_ft/sparse_trainer.py): `dense_ft/sparse_trainer.py`
- [LoRA fine-tuning](https://github.com/locuslab/wanda/blob/main/lora_ft/finetune_lm.py): `lora_ft/finetune_lm.py`

### Other components

- **Researcher**

## Connections

- Researcher → LLM workflow: runs (evidence: `README.md`)
- LLM workflow → Pretrained model: loads (evidence: `main.py`)
- LLM workflow → LLaMA pruning: dispatches (evidence: `main.py`)
- LLM workflow → Perplexity: evaluates (evidence: `main.py`)
- LLM workflow → Zero-shot tasks: evaluates (evidence: `main.py`)
- LLM workflow → Pruned model: saves (evidence: `main.py`)
- OPT workflow → OPT pruning: dispatches (evidence: `main_opt.py`)
- OPT workflow → Perplexity: evaluates (evidence: `main_opt.py`)
- LLaMA pruning → Calibration data: loads (evidence: `lib/prune.py`)
- OPT pruning → Calibration data: loads (evidence: `lib/prune_opt.py`)
- LLaMA pruning → Activation statistics: collects activations (evidence: `lib/prune.py`)
- LLaMA pruning → SparseGPT: uses (evidence: `lib/prune.py`)
- LLaMA pruning → OBS ablation: uses (evidence: `lib/prune.py`)
- OPT pruning → SparseGPT: uses (evidence: `lib/prune_opt.py`)
- OPT pruning → OBS ablation: uses (evidence: `lib/prune_opt.py`)
- Perplexity → Calibration data: loads data (evidence: `lib/eval.py`)
- Zero-shot tasks → LM evaluation harness: calls (evidence: `lib/eval.py`)
- Researcher → Classifier workflow: runs (evidence: `README.md`)
- Classifier workflow → Image datasets: builds datasets (evidence: `image_classifiers/main.py`)
- Classifier workflow → Vision architectures: selects model (evidence: `image_classifiers/main.py`)
- Train and evaluate → Classifier pruning: uses (evidence: `image_classifiers/engine.py`)
- Researcher → LoRA fine-tuning: runs (evidence: `README.md`)
