# jupiterliu/sae architecture

> Architecture diagram of the GitHub repository jupiterliu/sae, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jupiterliu/sae
- Repository: https://github.com/jupiterliu/sae

## Overview

1. This repository is split into two major pipelines: **class-incremental learning (CIL) training** and **adversarial attack benchmarking**. The training side produces target incremental models; the attack side loads those checkpoints and evaluates multiple attack methods against them.

2. The two main entry points are:
   - `trainCIL.py` for training CIL models from an experiment config.
   - `attack.py` for running an attack against a chosen trained model and target class.
   Shell scripts in `scripts/` batch these flows across CIFAR-100 and ImageNet-100.

3. The **CIL training subsystem** is organized around:
   - `models/`: implementations of nine incremental-learning algorithms: BiC, DER, Finetune, Foster, iCaRL, MEMO, PodNet, Replay, and WA.
   - `convs/`: backbone/network variants used by those algorithms, including ResNet-style and cosine/modified variants.
   - `utils/`: dataset handling, model factory logic, incremental-learning utilities, augmentation, plotting, and RL helpers.
   - `exps/`: JSON experiment definitions that bind together dataset, method, and hyperparameters.

4. The **data layer** supports two main datasets:
   - CIFAR-100, which is downloaded and prepared automatically.
   - ImageNet-100, which is derived from ImageNet-1K using the provided class mapping / split files in `datasets/`.
   The dataset utilities and managers in `utils/data_manager.py`, `datasets/`, and `utils/data.py` feed both CIL training and attack evaluation.

5. The **attack benchmarking subsystem** is centered in `attacks/` and is driven by a shared base interface in `attacks/BASEAttack.py` plus `attacks/attack_config.py`. `attack.py` selects the concrete implementation based on `--attack_method`, wires in the trained target model, and executes the attack on the requested target class.

6. The attack implementations are mostly self-contained subprojects:
   - `attacks/SAE/` is the most fully structured package, with its own package source tree under `src/gat/`, runtime API, surrogate-model utilities, loss functions, optimizers, generators, datasets, and tests.
   - `attacks/AIM/` has a similar packaged layout with `src/gat/`, runtime helpers, model components, and examples.
   - `attacks/CGNC/`, `attacks/CleanSheet/`, `attacks/Gaker/`, and `attacks/UnivIntruder/` each bundle their own generators, losses, dataset adapters, and evaluation code.

7. Several attacks rely on a **surrogate/model-guidance layer** rather than only perturbing pixels directly. In the structured packages this shows up as:
   - surrogate builders/hooks (`attacks/AIM/src/gat/models/surrogate/`)
   - generator modules (`.../generator/`)
   - custom losses and optimizers (`.../loss/`, `.../optim/`)
   - runtime glue for execution, logging, and metrics (`.../runtime/`)
   This separation suggests a common architecture: build or load a surrogate, generate adversarial examples, then evaluate transfer/sustainability against the frozen target CIL model.

8. **CLIP-backed text/image guidance** is an important external dependency for some attacks, especially `SAE` and `UnivIntruder`. The repo notes that a pretrained CLIP model is fetched from HuggingFace, and several attack packages include text templates, class-name mappings, and CLIP-related loss functions. That makes the attack side partly dependent on multimodal representation alignment, not just classifier gradients.

9. The **evaluation flow** is checkpoint-driven:
   - Train a CIL model with `trainCIL.py` using an `exps/*.json` config.
   - Store outputs under `logs/`.
   - Run `attack.py` with the same config plus attack-specific arguments.
   - Use the attack scripts to sweep methods/datasets and collect benchmark metrics.
   This means the main architectural boundary is between “offline training artifacts” and “attack-time loading/evaluation.”

10. The **testing and validation surface** is strongest in the packaged attack modules and CleanSheet:
    - `attacks/AIM/tests/`
    - `attacks/CleanSheet/tests/`
    - `attacks/AIM/src/gat/...`
    These tests cover datasets, transforms, surrogate components, meter/util functions, configs, and trainer logic, which indicates those attack packages are intended to be reusable libraries rather than just scripts.

11. `attacks/AIM/examples/` and `attacks/Gaker/Generator/train.py` show that some attack methods can be run in standalone training/generation modes, independent of the top-level benchmark runner. So the repo is not just a thin wrapper; it contains both a unified benchmark harness and method-specific subpipelines.

12. From an architecture-diagram perspective, the simplest accurate view is:
    - **Configs/scripts** select a CIL method and attack method.
    - **Dataset utilities** feed both training and attack phases.
    - **CIL model implementations** produce the target checkpoint.
    - **Attack modules** load the checkpoint, optionally use CLIP/surrogates/generators, craft adversarial samples, and report metrics.
    - **Logs/results** are the sink for both training artifacts and attack evaluations.

Main entry points: [`attacks/BASEAttack.py`](https://github.com/jupiterliu/sae/blob/master/attacks/BASEAttack.py) (Attack base), [`attack.py`](https://github.com/jupiterliu/sae/blob/master/attack.py) (attack.py), [`trainCIL.py`](https://github.com/jupiterliu/sae/blob/master/trainCIL.py) (trainCIL.py), [`utils/data_manager.py`](https://github.com/jupiterliu/sae/blob/master/utils/data_manager.py) (Data manager)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_training["CIL training"]
  node_node_traincil["trainCIL.py<br/>entrypoint<br/>[trainCIL.py]"]
  node_node_models["CIL models<br/>algorithms"]
  node_node_convs["Backbones<br/>networks"]
  node_node_utils["Training utils<br/>helpers"]
  node_node_exps["Experiment configs<br/>json configs"]
end

subgraph group_group_attacks["Attack benchmarking"]
  node_node_attackpy["attack.py<br/>entrypoint<br/>[attack.py]"]
  node_node_baseattack["Attack base<br/>interface<br/>[BASEAttack.py]"]
  node_node_attackcfg["Attack config<br/>[attack_config.py]"]
  node_node_aim["AIM<br/>structured attack"]
  node_node_sae["SAE<br/>CLIP-guided attack"]
  node_node_univ["UnivIntruder<br/>multimodal attack"]
  node_node_gaker["Gaker<br/>generator attack"]
  node_node_cgnc["CGNC<br/>generator attack"]
  node_node_cleansheet["CleanSheet<br/>packaged attack"]
end

subgraph group_group_data["Data layer"]
  node_node_datasets["Dataset prep<br/>dataset sources"]
  node_node_datamanager["Data manager<br/>loader orchestration<br/>[data_manager.py]"]
  node_node_datautil["Data utils<br/>dataset utilities<br/>[data.py]"]
end

subgraph group_group_runtime["Execution flow"]
  node_node_scripts["Shell scripts<br/>batch runners"]
  node_node_artifacts[("Artifacts<br/>checkpoints/results")]
end

node_node_exps -->|"selects run"| node_node_traincil
node_node_traincil -->|"trains"| node_node_models
node_node_traincil -->|"uses backbone"| node_node_convs
node_node_traincil -->|"loads data"| node_node_datamanager
node_node_datamanager -->|"prepares"| node_node_datasets
node_node_datautil -->|"supports"| node_node_datamanager
node_node_utils -->|"supports"| node_node_models
node_node_scripts -->|"batch train"| node_node_traincil
node_node_attackpy -->|"dispatches"| node_node_baseattack
node_node_attackpy -->|"reads config"| node_node_attackcfg
node_node_attackpy -->|"loads checkpoint"| node_node_artifacts
node_node_attackpy -->|"loads data"| node_node_datamanager
node_node_baseattack -->|"implements"| node_node_aim
node_node_baseattack -->|"implements"| node_node_sae
node_node_baseattack -->|"implements"| node_node_univ
node_node_baseattack -->|"implements"| node_node_gaker
node_node_baseattack -->|"implements"| node_node_cgnc
node_node_baseattack -->|"implements"| node_node_cleansheet
node_node_aim -->|"evaluates"| node_node_artifacts
node_node_sae -->|"evaluates"| node_node_artifacts
node_node_univ -->|"evaluates"| node_node_artifacts
node_node_gaker -->|"evaluates"| node_node_artifacts
node_node_cgnc -->|"evaluates"| node_node_artifacts
node_node_cleansheet -->|"evaluates"| node_node_artifacts
node_node_scripts -->|"batch attack"| node_node_attackpy

click node_node_traincil "https://github.com/jupiterliu/sae/blob/master/trainCIL.py"
click node_node_models "https://github.com/jupiterliu/sae/tree/master/models"
click node_node_convs "https://github.com/jupiterliu/sae/tree/master/convs"
click node_node_utils "https://github.com/jupiterliu/sae/tree/master/utils"
click node_node_exps "https://github.com/jupiterliu/sae/tree/master/exps"
click node_node_datasets "https://github.com/jupiterliu/sae/tree/master/datasets"
click node_node_datamanager "https://github.com/jupiterliu/sae/blob/master/utils/data_manager.py"
click node_node_datautil "https://github.com/jupiterliu/sae/blob/master/utils/data.py"
click node_node_attackpy "https://github.com/jupiterliu/sae/blob/master/attack.py"
click node_node_baseattack "https://github.com/jupiterliu/sae/blob/master/attacks/BASEAttack.py"
click node_node_attackcfg "https://github.com/jupiterliu/sae/blob/master/attacks/attack_config.py"
click node_node_aim "https://github.com/jupiterliu/sae/tree/master/attacks/AIM"
click node_node_sae "https://github.com/jupiterliu/sae/tree/master/attacks/SAE"
click node_node_univ "https://github.com/jupiterliu/sae/tree/master/attacks/UnivIntruder"
click node_node_gaker "https://github.com/jupiterliu/sae/tree/master/attacks/Gaker"
click node_node_cgnc "https://github.com/jupiterliu/sae/tree/master/attacks/CGNC"
click node_node_cleansheet "https://github.com/jupiterliu/sae/tree/master/attacks/CleanSheet"
click node_node_scripts "https://github.com/jupiterliu/sae/tree/master/scripts"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_node_traincil,node_node_models,node_node_convs,node_node_utils,node_node_exps toneBlue
class node_node_attackpy,node_node_baseattack,node_node_attackcfg,node_node_aim,node_node_sae,node_node_univ,node_node_gaker,node_node_cgnc,node_node_cleansheet toneAmber
class node_node_datasets,node_node_datamanager,node_node_datautil toneMint
class node_node_scripts,node_node_artifacts toneRose
```

## Components

### CIL training

Class-incremental learning pipeline that trains target checkpoints.

- [trainCIL.py](https://github.com/jupiterliu/sae/blob/master/trainCIL.py): entrypoint · Main training entry point for class-incremental experiments. · `trainCIL.py`
- [CIL models](https://github.com/jupiterliu/sae/tree/master/models): algorithms · Incremental-learning methods such as BiC, DER, iCaRL, MEMO, PodNet, Replay, WA, Foster, and Finetune. · `models`
- [Backbones](https://github.com/jupiterliu/sae/tree/master/convs): networks · ResNet-style and modified convolutional backbones used by CIL methods. · `convs`
- [Training utils](https://github.com/jupiterliu/sae/tree/master/utils): helpers · Model factory, incremental-learning helpers, augmentation, plotting, and RL utilities. · `utils`
- [Experiment configs](https://github.com/jupiterliu/sae/tree/master/exps): json configs · Method- and dataset-specific experiment definitions for CIL runs. · `exps`

### Attack benchmarking

Attack runners and method-specific attack implementations.

- [attack.py](https://github.com/jupiterliu/sae/blob/master/attack.py): entrypoint · Main attack runner that selects an attack method and evaluates a target checkpoint. · `attack.py`
- [Attack base](https://github.com/jupiterliu/sae/blob/master/attacks/BASEAttack.py): interface · Shared attack interface and configuration layer. · `attacks/BASEAttack.py`
- [Attack config](https://github.com/jupiterliu/sae/blob/master/attacks/attack_config.py): config · Common attack configuration and method selection settings. · `attacks/attack_config.py`
- [AIM](https://github.com/jupiterliu/sae/tree/master/attacks/AIM): structured attack · Packaged attack implementation with surrogate, generator, loss, optim, runtime, and tests. · `attacks/AIM`
- [SAE](https://github.com/jupiterliu/sae/tree/master/attacks/SAE): CLIP-guided attack · Attack package with CLIP-related utilities, text templates, and model helpers. · `attacks/SAE`
- [UnivIntruder](https://github.com/jupiterliu/sae/tree/master/attacks/UnivIntruder): multimodal attack · Attack package with dataset adapters and CLIP/SigLIP-style losses. · `attacks/UnivIntruder`
- [Gaker](https://github.com/jupiterliu/sae/tree/master/attacks/Gaker): generator attack · Attack package with generator training and benchmark utilities. · `attacks/Gaker`
- [CGNC](https://github.com/jupiterliu/sae/tree/master/attacks/CGNC): generator attack · Attack package with text-feature extraction, generator, and inference code. · `attacks/CGNC`
- [CleanSheet](https://github.com/jupiterliu/sae/tree/master/attacks/CleanSheet): packaged attack · Attack package with training, poison data, models, and broad tests. · `attacks/CleanSheet`

### Data layer

Shared dataset preparation and loaders used by both pipelines.

- [Dataset prep](https://github.com/jupiterliu/sae/tree/master/datasets): dataset sources · CIFAR-100 and ImageNet-100 preparation, splits, and class mapping files. · `datasets`
- [Data manager](https://github.com/jupiterliu/sae/blob/master/utils/data_manager.py): loader orchestration · Shared dataset and batch management used by training and attack evaluation. · `utils/data_manager.py`
- [Data utils](https://github.com/jupiterliu/sae/blob/master/utils/data.py): dataset utilities · Shared dataset helpers and data access code for both pipelines. · `utils/data.py`

### Execution flow

Entry points, batch scripts, and artifact flow.

- [Shell scripts](https://github.com/jupiterliu/sae/tree/master/scripts): batch runners · Batch scripts for CIL training and attack sweeps on CIFAR-100 and ImageNet-100. · `scripts`
- **Artifacts**: checkpoints/results · Training checkpoints and attack outputs produced during evaluation runs.

## Connections

- Experiment configs → trainCIL.py: selects run · Experiment JSON defines dataset, method, and hyperparameters for training.
- trainCIL.py → CIL models: trains · Training entry point instantiates the chosen incremental-learning algorithm.
- trainCIL.py → Backbones: uses backbone · CIL methods build on shared convolutional backbones and variants.
- trainCIL.py → Data manager: loads data · Training consumes dataset managers and loaders.
- Data manager → Dataset prep: prepares · Dataset manager uses dataset-specific splits and mapping files.
- Data utils → Data manager: supports · Shared data helpers feed loader and dataset orchestration.
- Training utils → CIL models: supports · Factories and utility functions support incremental method execution.
- Shell scripts → trainCIL.py: batch train · Shell scripts launch training sweeps across datasets and methods.
- attack.py → Attack base: dispatches · Top-level attack runner selects a concrete attack implementation through the base interface.
- attack.py → Attack config: reads config · Attack runner consumes shared attack configuration.
- attack.py → Artifacts: loads checkpoint · Attack evaluation operates on trained target artifacts.
- attack.py → Data manager: loads data · Attack evaluation reuses the shared dataset layer.
- Attack base → AIM: implements · AIM is one concrete attack family behind the shared interface.
- Attack base → SAE: implements · SAE is one concrete attack family behind the shared interface.
- Attack base → UnivIntruder: implements · UnivIntruder is one concrete attack family behind the shared interface.
- Attack base → Gaker: implements · Gaker is one concrete attack family behind the shared interface.
- Attack base → CGNC: implements · CGNC is one concrete attack family behind the shared interface.
- Attack base → CleanSheet: implements · CleanSheet is one concrete attack family behind the shared interface.
- AIM → Artifacts: evaluates · AIM attacks run against frozen target models and report metrics.
- SAE → Artifacts: evaluates · SAE loads checkpoints and benchmarks transfer-style attacks.
- UnivIntruder → Artifacts: evaluates · UnivIntruder benchmarks against trained target checkpoints.
- Gaker → Artifacts: evaluates · Gaker generator outputs are evaluated against target models.
- CGNC → Artifacts: evaluates · CGNC inference benchmarks adversarial generation against target models.
- CleanSheet → Artifacts: evaluates · CleanSheet attacks and related training flows produce benchmark outputs.
- Shell scripts → attack.py: batch attack · Shell scripts sweep attack methods and dataset/model settings.