How calculatedcontent/weightwatcher works
Project Type and Purpose
• This is an open-source Python library and CLI tool (WeightWatcher) for diagnosing deep neural networks using Random Matrix Theory.
• It provides a programmatic API (weightwatcher package), a command-line entrypoint, Jupyter Notebook examples, environment checks, and unit tests.
High-Level Architectural Components
a. Entry & Packaging
• setup.py / setup.cfg / setup.mac – Defines packaging and installation on PyPI.
• __main__.py – CLI entrypoint (“python -m weightwatcher”).
b. Core Analysis Engine (weightwatcher package)
• weightwatcher.py – Core WeightWatcher class with methods analyze(), describe(), get_summary(), distances(), etc.
• RMT_Util.py – Utilities for Random Matrix Theory computations (e.g., build correlation matrices, compute eigenvalues).
• WW_powerlaw.py – Power‐law fitting routines (PL, TPL, E_TPL).
• constants.py – Configuration constants (layer types, thresholds).
c. Visualization & Plotting
• Built-in plotting via matplotlib (ESD histograms, power-law fits, MP fits).
• savefig options write PNGs for each layer.
d. Examples & Tutorials
• examples/ – Jupyter notebooks demonstrating usage (LSA, model plots, WW-VGG, WW-GPT, etc.).
• examples/README.md – Guide to examples.
e. Environment Checks
• checks/ – Python scripts verifying environment dependencies and performance (e.g., SciPy SVD speed, TensorFlow presence).
f. Testing
• tests/ – Unit tests for core functionality (test_svd, test.py).
g. External Dependencies
• PyTorch, TensorFlow, NumPy, SciPy, pandas, matplotlib, powerlaw library.
Component Interactions & Data Flow
• CLI or user script imports weightwatcher → instantiates WeightWatcher(model) → calls analyze()
‣ WeightWatcher traverses model layers → extracts weight matrices → uses RMT_Util to build X = WᵀW → computes eigenvalues.
‣ WW_powerlaw fits power‐laws to the empirical spectral density (ESD) tails.
‣ Optionally, performs MP fits, randomization tests, distance calculations.
‣ Optionally, invokes matplotlib to plot ESDs and write figures.
‣ Aggregates per-layer metrics into a pandas DataFrame (“details”) and summary dict.
• tests invoke functions in weightwatcher package to validate correctness.
• checks scripts run preflight diagnostics on the environment or performance.
Architectural Patterns & Principles
• Modular decomposition: separate core logic (analysis, fitting), utilities (RMT, PL fits), visualization, and entrypoints.
• Separation of concerns: analysis vs plotting vs I/O vs packaging.
• Extensibility: new fitting methods, new layer types, new metrics can be added via RMT_Util and WW_powerlaw modules.
[CLI Entrypoint]
[Core Engine]
[RMT Utilities]
[Power-Law Fit Module]
[Visualization Module]
[Examples / Tutorials]
[Environment Checks]
[Unit Tests]
[Packaging / PyPI]
(PyTorch), (TensorFlow), (NumPy), (SciPy), (pandas), (matplotlib), (powerlaw)
• CLI Entrypoint → Core Engine
• Core Engine → RMT Utilities → Core Engine
• Core Engine → Power-Law Fit Module → Core Engine
• Core Engine → Visualization Module → File System (plots)
• Core Engine → returns DataFrame & Summary (to user)
• Environment Checks → validate External Libraries
• Unit Tests → exercise Core Engine & Utils
• Packaging → publishes Core Engine to PyPI
• Blue rectangles for internal modules (core, utils).
• Green ovals for external dependencies (libraries).
• Yellow parallelograms for I/O (notebooks, plots, CSV/DataFrame).