# ryanwangzf/pac-bayes-ib architecture

> Architecture diagram of the GitHub repository ryanwangzf/pac-bayes-ib, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ryanwangzf/pac-bayes-ib
- Repository: https://github.com/ryanwangzf/pac-bayes-ib

## Overview

This repository is a research demonstration of PAC-Bayes Information Bottleneck (PAC-Bayes IB) methods for neural-network training and information-in-weights analysis. The principal workflow loads and preprocesses a dataset, initializes an MLP or VGG model and prior weights, trains while tracking information and accuracy, then evaluates or plots the results. Separate paths demonstrate IIW-regularized training and conventional VGG baselines. The README documents notebook experiments for phase transitions and batch-size/generalization analysis. Some notebook and implementation details were not sampled, so only documented or excerpt-supported connections are shown.

Main entry points: [`run_vgg_PIB.py`](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_PIB.py) (PIB VGG experiment), [`src/utils.py`](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py) (Standard training), [`run_vgg_vanilla.py`](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_vanilla.py) (VGG baselines), [`src/dataset.py`](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/dataset.py) (Dataset loaders)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_loaders["Dataset loaders<br/>[dataset.py]"]
  node_preprocessing["Preprocessing and metrics<br/>[utils.py]"]
end

subgraph group_models["Neural models"]
  node_mlp["MLP model<br/>[models.py]"]
  node_vgg["VGG model<br/>[models.py]"]
  node_information_measure["Information measure<br/>[models.py]"]
end

subgraph group_training["Training and analysis"]
  node_standard_training["Standard training<br/>[utils.py]"]
  node_prior["Prior training<br/>[utils.py]"]
  node_information_tracking["Information tracking<br/>[utils.py]"]
  node_pib_training["PAC-Bayes IB training<br/>[pib_utils.py]"]
  node_iiw_training["IIW regularization<br/>[pib_utils.py]"]
  node_plots["Analysis plots<br/>[utils.py]"]
end

subgraph group_experiments["Experiment entry points"]
  node_pib_run["PIB VGG experiment<br/>[run_vgg_PIB.py]"]
  node_baseline_runs["VGG baselines<br/>[run_vgg_vanilla.py]"]
  node_dropout_run["Dropout baseline<br/>[run_vgg_dropout.py]"]
  node_l2_run["L2 baseline<br/>[run_vgg_l2.py]"]
  node_notebooks["Notebook experiments"]
  node_batch_notebook["Batch-size analysis"]
  node_ib_notebook["PAC-Bayes IB demo"]
end

node_researcher(("Researcher"))
node_datasets[("Datasets")]

node_researcher -->|"runs"| node_pib_run
node_researcher -->|"explores"| node_notebooks
node_researcher -->|"explores"| node_batch_notebook
node_researcher -->|"explores"| node_ib_notebook
node_datasets -->|"provides data"| node_loaders
node_loaders -->|"imports preprocessing"| node_preprocessing
node_pib_run -->|"loads data"| node_loaders
node_pib_run -->|"preprocesses data"| node_preprocessing
node_pib_run -->|"initializes"| node_vgg
node_pib_run -->|"trains prior"| node_standard_training
node_pib_run -->|"starts PIB training"| node_pib_training
node_pib_run -->|"evaluates results"| node_preprocessing
node_pib_training -->|"computes information"| node_information_measure
node_pib_training -->|"evaluates accuracy"| node_preprocessing
node_standard_training -->|"evaluates accuracy"| node_preprocessing
node_prior -->|"trains model"| node_standard_training
node_information_tracking -->|"tracks information"| node_information_measure
node_information_tracking -->|"supplies metrics"| node_plots
node_ib_notebook -->|"demonstrates"| node_iiw_training
node_batch_notebook -->|"analyzes"| node_information_tracking
node_notebooks -->|"demonstrates"| node_information_tracking
node_baseline_runs -->|"loads data"| node_loaders
node_baseline_runs -->|"preprocesses data"| node_preprocessing
node_baseline_runs -->|"initializes"| node_vgg
node_baseline_runs -->|"trains model"| node_standard_training
node_dropout_run -->|"initializes dropout"| node_vgg
node_dropout_run -->|"trains model"| node_standard_training
node_l2_run -->|"initializes"| node_vgg
node_l2_run -->|"trains with L2"| node_standard_training

click node_loaders "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/dataset.py"
click node_preprocessing "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py"
click node_mlp "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/models.py"
click node_vgg "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/models.py"
click node_standard_training "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py"
click node_prior "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py"
click node_information_tracking "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py"
click node_pib_training "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/pib_utils.py"
click node_iiw_training "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/pib_utils.py"
click node_information_measure "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/models.py"
click node_plots "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py"
click node_pib_run "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_PIB.py"
click node_baseline_runs "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_vanilla.py"
click node_dropout_run "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_dropout.py"
click node_l2_run "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_l2.py"
click node_notebooks "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/demo_track_information.ipynb"
click node_batch_notebook "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/demo_batch_size_info.ipynb"
click node_ib_notebook "https://github.com/ryanwangzf/pac-bayes-ib/blob/main/demo_train_pac_bayes_ib.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loaders,node_preprocessing toneBlue
class node_mlp,node_vgg,node_information_measure,node_datasets toneAmber
class node_standard_training,node_prior,node_information_tracking,node_pib_training,node_iiw_training,node_plots toneMint
class node_pib_run,node_baseline_runs,node_dropout_run,node_l2_run,node_notebooks,node_batch_notebook,node_ib_notebook toneRose
class node_researcher toneIndigo
```

## Components

### Data preparation

- [Dataset loaders](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/dataset.py): `src/dataset.py`
- [Preprocessing and metrics](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py): `src/utils.py`

### Neural models

- [MLP model](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/models.py): `src/models.py`
- [VGG model](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/models.py): `src/models.py`
- [Information measure](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/models.py): `src/models.py`

### Training and analysis

- [Standard training](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py): `src/utils.py`
- [Prior training](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py): `src/utils.py`
- [Information tracking](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py): `src/utils.py`
- [PAC-Bayes IB training](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/pib_utils.py): `src/pib_utils.py`
- [IIW regularization](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/pib_utils.py): `src/pib_utils.py`
- [Analysis plots](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/src/utils.py): `src/utils.py`

### Experiment entry points

- [PIB VGG experiment](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_PIB.py): `run_vgg_PIB.py`
- [VGG baselines](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_vanilla.py): `run_vgg_vanilla.py`
- [Dropout baseline](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_dropout.py): `run_vgg_dropout.py`
- [L2 baseline](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/run_vgg_l2.py): `run_vgg_l2.py`
- [Notebook experiments](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/demo_track_information.ipynb): `demo_track_information.ipynb`
- [Batch-size analysis](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/demo_batch_size_info.ipynb): `demo_batch_size_info.ipynb`
- [PAC-Bayes IB demo](https://github.com/ryanwangzf/pac-bayes-ib/blob/main/demo_train_pac_bayes_ib.ipynb): `demo_train_pac_bayes_ib.ipynb`

### Other components

- **Researcher**
- **Datasets**

## Connections

- Researcher → PIB VGG experiment: runs (evidence: `README.md`)
- Researcher → Notebook experiments: explores (evidence: `README.md`)
- Researcher → Batch-size analysis: explores (evidence: `README.md`)
- Researcher → PAC-Bayes IB demo: explores (evidence: `README.md`)
- Datasets → Dataset loaders: provides data (evidence: `src/dataset.py`)
- Dataset loaders → Preprocessing and metrics: imports preprocessing (evidence: `src/dataset.py`)
- PIB VGG experiment → Dataset loaders: loads data (evidence: `run_vgg_PIB.py`)
- PIB VGG experiment → Preprocessing and metrics: preprocesses data (evidence: `run_vgg_PIB.py`)
- PIB VGG experiment → VGG model: initializes (evidence: `run_vgg_PIB.py`)
- PIB VGG experiment → Standard training: trains prior (evidence: `run_vgg_PIB.py`)
- PIB VGG experiment → PAC-Bayes IB training: starts PIB training (evidence: `run_vgg_PIB.py`)
- PIB VGG experiment → Preprocessing and metrics: evaluates results (evidence: `run_vgg_PIB.py`)
- PAC-Bayes IB training → Information measure: computes information (evidence: `src/pib_utils.py`)
- PAC-Bayes IB training → Preprocessing and metrics: evaluates accuracy (evidence: `src/pib_utils.py`)
- Standard training → Preprocessing and metrics: evaluates accuracy (evidence: `src/utils.py`)
- Prior training → Standard training: trains model (evidence: `src/utils.py`)
- Information tracking → Information measure: tracks information (evidence: `README.md`)
- Information tracking → Analysis plots: supplies metrics (evidence: `README.md`)
- PAC-Bayes IB demo → IIW regularization: demonstrates (evidence: `README.md`)
- Batch-size analysis → Information tracking: analyzes (evidence: `README.md`)
- Notebook experiments → Information tracking: demonstrates (evidence: `README.md`)
- VGG baselines → Dataset loaders: loads data (evidence: `run_vgg_vanilla.py`)
- VGG baselines → Preprocessing and metrics: preprocesses data (evidence: `run_vgg_vanilla.py`)
- VGG baselines → VGG model: initializes (evidence: `run_vgg_vanilla.py`)
- VGG baselines → Standard training: trains model (evidence: `run_vgg_vanilla.py`)
- Dropout baseline → VGG model: initializes dropout (evidence: `run_vgg_dropout.py`)
- Dropout baseline → Standard training: trains model (evidence: `run_vgg_dropout.py`)
- L2 baseline → VGG model: initializes (evidence: `run_vgg_l2.py`)
- L2 baseline → Standard training: trains with L2 (evidence: `run_vgg_l2.py`)
