# pilancilab/caldera architecture

> Architecture diagram of the GitHub repository pilancilab/caldera, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pilancilab/caldera
- Repository: https://github.com/pilancilab/caldera

## Overview

##### Project Type and Purpose
- CALDERA is a research‐oriented Python library and command‐line toolkit for post‐training compression of large language model (LLM) weights via a low‐rank, low‐precision decomposition.
- It is not a web app or microservice; rather, it is a modular Python package (`src/caldera`) with companion scripts, shell runners, and Jupyter notebooks to facilitate experiments: compute Hessians, quantize layers, evaluate, and fine‐tune.

##### File‐Structure Analysis
Top‐Level Components
- `src/caldera/` – Core Python package implementing decomposition algorithm, quantized layer classes, and utilities.
– `decomposition/alg.py` – Core alternating‐minimization algorithm.
– `decomposition/weight_compression.py`, `layer_quantization.py`, `quantized_layer.py` – Classes orchestrating compression and runtime inference of quantized layers.
– `utils/` – Enumerations and basic quantizers.
- `scripts/` – Standalone Python entry points (CLI scripts) to run zero‐shot, perplexity, GLUE, Winogrande, Wikitext evaluations and fine‐tuning.
- `shell_scripts/` – Bash wrappers that supply reasonable defaults to the `scripts/` CLIs; coordinate runs, set output directories, point to configuration YAML/JSON.
- `quip-sharp/` – Git submodule providing the QuIP# quantizer C++/CUDA library and Python bindings.
- `notebooks/` – Jupyter notebooks for interactive testing and throughput evaluation.
- Config files: `pyproject.toml`, `requirements.txt`, `accelerate_config.yaml`, `deepspeed_config.json`.

a. Core Library (Python Package “caldera”)
- Modules:
- `caldera.decomposition.alg`
- `caldera.decomposition.weight_compression`
- `caldera.decomposition.layer_quantization`
- `caldera.quantized_layer`
- `caldera.utils`
- Datastores: Hessian files, quantization parameter JSON outputs, saved quantized model checkpoints.
b. CLI Layer
- Python scripts in `scripts/`
- Orchestrated by Bash wrappers in `shell_scripts/`
c. External Dependencies
- PyTorch (model loading, tensor ops)
- CUDA/C++ kernels via QuIP# (`quiptools`)
- `fast-hadamard-transform` (optional dependency for Hessian estimation)
- LLM model weights (e.g., LLaMA)
d. Data Flow & Artifacts
1. **Compute Hessians**
- `run_save_hessians.sh` → `scripts/save_llama_hessians.py` → outputs per-layer Hessian files.
2. **Decompose & Quantize**
- `run_quantize_save_caldera.sh` → `scripts/quantize_save_llama.py`
- Calls into `caldera.decomposition.*` and QuIP# APIs → emits quantized layer artifacts.
3. **Evaluation**
- `run_eval_ppl.sh` / `run_eval_zeroshot.sh` → `scripts/eval_ppl.py` / `scripts/eval_zero_shot.py` → reads quantized model → writes metrics.
4. **Fine‐tuning**
- `run_finetune_*.sh` → corresponding `scripts/finetune_*.py` → uses LoRA adaptation on low‐rank factors.

##### Architectural Patterns & Design Principles
- Modular Python package separating algorithm implementation from CLI interfaces and shell orchestration.
- Data‐driven pipeline: each stage reads/writes files, enabling reproducibility and partial reruns.
- Submodule pattern for external C++/CUDA library (QuIP#) co‐built and version‐controlled.
- Use of dataclasses (`dataclasses.py`) for clear parameter passing and JSON serialization.

##### Final Advice
- Every subcomponent in `src/caldera` and show its interaction with scripts.
- Pipeline nature: decomposer → quantizer → evaluator → finetuner.
- QuIP# build step (`setup_quip_sharp.sh`) to emphasize the C++/CUDA integration.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Dependencies
    subgraph "External Dependencies"
        direction TB
        PyTorch["PyTorch"]:::external
        QuIPSharp["QuIP# Submodule"]:::external
        FHT["fast-hadamard-transform"]:::external
    end

    %% Data Store of LLM Weights
    LLMWeights["LLM Model Weights"]:::data

    %% Pipeline Phases
    subgraph "Hessian Computation"
        direction TB
        SH["Save Hessians Script"]:::cli
        HF["Hessian Files"]:::data
    end

    subgraph "Quantization"
        direction TB
        QS["Quantize & Save Script"]:::cli
        subgraph "CALDERA Package"
            direction TB
            ALG["AlternatingMinimization\n(src/caldera/decomposition/alg.py)"]:::core
            WC["WeightCompression\n(src/caldera/decomposition/weight_compression.py)"]:::core
            LQ["LayerQuantization\n(src/caldera/decomposition/layer_quantization.py)"]:::core
            QL["CalderaQuantizedLinear\n(src/caldera/decomposition/quantized_layer.py)"]:::core
            DC["Dataclasses\n(src/caldera/decomposition/dataclasses.py)"]:::core
            UIEnum["Enums\n(src/caldera/utils/enums.py)"]:::core
            UIQuant["Quantizers\n(src/caldera/utils/quantization.py)"]:::core
            class ALG,WC,LQ,QL,DC,UIEnum,UIQuant core
        end
        QA["Quantized Artifacts"]:::data
    end

    subgraph "Evaluation"
        direction TB
        EPL["Eval Perplexity\n(scripts/eval_ppl.py)"]:::cli
        EZS["Eval Zero-shot\n(scripts/eval_zero_shot.py)"]:::cli
        EM["Evaluation Metrics"]:::data
    end

    subgraph "Fine-tuning"
        direction TB
        FRHT["Fine-tune RHT\n(scripts/finetune_RHT.py)"]:::cli
        FGL["Fine-tune GLUE\n(scripts/finetune_glue.py)"]:::cli
        FWO["Fine-tune Winogrande\n(scripts/finetune_winogrande.py)"]:::cli
        FWI["Fine-tune Wikitext\n(scripts/finetune_wikitext.py)"]:::cli
        FTOut["Fine-tuned Models"]:::data
    end

    %% Shell Orchestration
    subgraph "Shell Wrappers" 
        direction TB
        RS{"run_save_hessians.sh"}:::cli
        RQ{"run_quantize_save_caldera.sh"}:::cli
        RP{"run_eval_ppl.sh"}:::cli
        RZ{"run_eval_zeroshot.sh"}:::cli
        RF{"run_finetune_*.sh"}:::cli
        CFG{"accelerate_config.yaml\n&\ndeepspeed_config.json"}:::config
    end

    %% Interactive and Assets
    Notebooks["Jupyter Notebooks\n(notebooks/)"]:::external
    Assets["Static Assets\n(assets/)"]:::external

    %% Config Files
    Configs{"pyproject.toml\nrequirements.txt"}:::config

    %% Connections
    LLMWeights -->|load| SH
    SH -->|"save_llama_hessians.py"| HF

    HF --> RS
    RS --> SH

    HF -->|input| QS
    QS -->ALG
    QS -->WC
    QS -->LQ
    QS -->QL
    QS -->DC
    QS -->UIEnum
    QS -->UIQuant
    QS -->QuIPSharp
    QuIPSharp -->|CUDA Kernels| LQ
    QS --> QA

    QA --> RP
    RP --> EPL
    RP --> EM

    QA --> RZ
    RZ --> EZS
    RZ --> EM

    QA --> RF
    RF --> FRHT
    RF --> FGL
    RF --> FWO
    RF --> FWI
    FRHT --> FTOut
    FGL --> FTOut
    FWO --> FTOut
    FWI --> FTOut

    Notebooks -->|use| ALG
    Notebooks -->|use| QS

    Assets -->|reference| ALG

    Configs -.-> RS
    Configs -.-> RQ
    Configs -.-> RP
    Configs -.-> RZ
    Configs -.-> RF

    %% Click Events for Scripts and Modules
    click ALG "https://github.com/pilancilab/caldera/blob/main/src/caldera/decomposition/alg.py"
    click WC "https://github.com/pilancilab/caldera/blob/main/src/caldera/decomposition/weight_compression.py"
    click LQ "https://github.com/pilancilab/caldera/blob/main/src/caldera/decomposition/layer_quantization.py"
    click QL "https://github.com/pilancilab/caldera/blob/main/src/caldera/decomposition/quantized_layer.py"
    click DC "https://github.com/pilancilab/caldera/blob/main/src/caldera/decomposition/dataclasses.py"
    click UIEnum "https://github.com/pilancilab/caldera/blob/main/src/caldera/utils/enums.py"
    click UIQuant "https://github.com/pilancilab/caldera/blob/main/src/caldera/utils/quantization.py"
    click SH "https://github.com/pilancilab/caldera/blob/main/scripts/save_llama_hessians.py"
    click QS "https://github.com/pilancilab/caldera/blob/main/scripts/quantize_save_llama.py"
    click EPL "https://github.com/pilancilab/caldera/blob/main/scripts/eval_ppl.py"
    click EZS "https://github.com/pilancilab/caldera/blob/main/scripts/eval_zero_shot.py"
    click FRHT "https://github.com/pilancilab/caldera/blob/main/scripts/finetune_RHT.py"
    click FGL "https://github.com/pilancilab/caldera/blob/main/scripts/finetune_glue.py"
    click FWO "https://github.com/pilancilab/caldera/blob/main/scripts/finetune_winogrande.py"
    click FWI "https://github.com/pilancilab/caldera/blob/main/scripts/finetune_wikitext.py"
    click RS "https://github.com/pilancilab/caldera/blob/main/shell_scripts/run_save_hessians.sh"
    click RQ "https://github.com/pilancilab/caldera/blob/main/shell_scripts/run_quantize_save_caldera.sh"
    click RP "https://github.com/pilancilab/caldera/blob/main/shell_scripts/run_eval_ppl.sh"
    click RZ "https://github.com/pilancilab/caldera/blob/main/shell_scripts/run_eval_zeroshot.sh"
    click RF "https://github.com/pilancilab/caldera/blob/main/shell_scripts/run_finetune_RHT.sh"
    click CFG "https://github.com/pilancilab/caldera/blob/main/shell_scripts/accelerate_config.yaml"
    click CFG "https://github.com/pilancilab/caldera/blob/main/shell_scripts/deepspeed_config.json"
    click QuIPSharp "https://github.com/pilancilab/caldera/tree/main/quip-sharp/"
    click Notebooks "https://github.com/pilancilab/caldera/tree/main/notebooks/"
    click Assets "https://github.com/pilancilab/caldera/tree/main/assets/"

    %% Styles
    classDef core fill:#e0f8d8,stroke:#2d662d,color:#2d662d;
    classDef cli fill:#d0e7ff,stroke:#1a4a81,color:#1a4a81;
    classDef external fill:#ffe9b3,stroke:#a86500,color:#a86500;
    classDef data fill:#f2f2f2,stroke:#4d4d4d,color:#4d4d4d;
    classDef config fill:#fff0b3,stroke:#b38f00,color:#b38f00;
```
