# ubc-systopia/dp-adambc architecture

> Architecture diagram of the GitHub repository ubc-systopia/dp-adambc, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ubc-systopia/dp-adambc
- Repository: https://github.com/ubc-systopia/dp-adambc

## Overview

##### Project type and purpose
- This repository implements Differentially Private optimizers (DP-AdamBC, DP-Adam, DP-SGD) and compares them on vision (CIFAR10) and NLP (GLUE) tasks.
- It consists of a core Python pipeline (in the “GNN” directory) plus two parallel reference implementations: one in JAX and one in PyTorch.

##### File structure
Top-level modules:
– GNN/        ← core data/model/training framework, privacy accountants & hyperparameter tuning
– jax_implementation/  ← JAX-based pipeline: data_utils, models, trainers
– pytorch_implementation/ ← PyTorch-based pipeline: configlib, Opacus integration for text classification
– README.md  ← usage examples, DH-AdamBC paper reference

Inside GNN/:
– main.py, train.py      ← orchestration of training runs
– dataset_readers.py + download_datasets.py + input_pipeline.py  ← data ingestion & batching
– models.py + normalizations.py                             ← model definitions & preprocessing
– optim/, optimizers.py + adam_optimizers.py                ← DP optimizers, optimizer interface
– privacy_accountants.py                                     ← ε accounting
– sampler.py                                                  ← sampling logic for privacy
– autotune/
- best_params/ + SQL scripts + tune_adambc_example.json  ← hyperparameter-tuning subsystem
– configs/                                                    ← named experiment configurations
– *_test.py                                                 ← unit tests
– run_tuning.sh, requirements.txt                            ← tooling & dependencies

##### README for additional insights
- Shows how to invoke:
– Core (GNN) or Top-level main.py for vision tasks (CNN5 on CIFAR10)
– jax_implementation/main.py for DP-AdamBC via JAX
– pytorch_implementation/text_classification_glue.py or opacus_text_classification.py for NLP

a. Main components
- Data Layer
– Raw dataset download & storage
– Dataset readers & input pipeline (batching, augmentation)
- Core DP Framework (“GNN”)
– Models & normalizations
– Sampler (per-step sampling for DP)
– Optimizer interface & DP algorithms (DPAdam, DPIterative/DP-SGD)
– Privacy accountant (tracks cumulative ε)
– Hyperparameter tuner (Autotune subsystem)
– Experiment configs
– Orchestrator (main.py / train.py)
- Implementation Tracks
– JAX pipeline: data_utils → model modules → trainer modules → utils
– PyTorch pipeline: configlib → text classifiers → Opacus wrappers
- External Services
– SQL (for storing/retrieving best hyperparams)
– JSON config files

b. Relationships & interactions
– Data Layer feeds into both Core framework and two implementation tracks
– Orchestrator reads a config → instantiates dataset pipeline & model → selects trainer & optimizer → runs training loop
– During training, optimizer interacts with sampler and privacy accountant on each step
– Hyperparameter tuner invokes multiple orchestrator runs, logs results to SQL/JSON
– Implementation tracks mirror this flow but use framework-specific primitives

c. Architectural patterns & principles
– Modular “plugin” architecture: optimizers, trainers, models each follow a common interface
– Separation of concerns between data ingestion, model definition, optimization, and privacy accounting
– Extensible tuning subsystem decoupled from training logic via scripts and JSON specs

d. Key technologies & libraries
– Python (core)
– JAX & Flax (jax_implementation)
– PyTorch & Opacus (pytorch_implementation)
– SQL database or SQLite (autotune)
– JSON for hyperparameter specs
– Command-line args (via argparse / configlib)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        DL1["Download Datasets"]:::data
        DL2["Dataset Readers"]:::data
        DL3["Input Pipeline"]:::data
    end

    %% Core DP Framework
    subgraph "Core DP Framework"
        Orch["Orchestrator\n(main.py / train.py)"]:::core
        Models["Models & Preprocessing"]:::core
        Optim["Optimizer Interface & DP Algorithms"]:::core
        Samp["Sampler"]:::core
        PA["Privacy Accountant"]:::core
        Configs["Experiment Configs"]:::core
    end

    %% Hyperparameter Tuner
    subgraph "Hyperparameter Tuner"
        AT["Autotune (autotune.py)"]:::tune
        TuneJSON["Tuning JSON\n(tune_adambc_example.json)"]:::tune
        BestParams["Best Params\n(directory)"]:::tune
        RunTune["Run Tuning Shell\n(run_tuning.sh)"]:::tune
    end

    %% Implementation Tracks
    subgraph "Implementation Tracks"
        subgraph "JAX Pipeline"
            Jcfg["ConfigLib"]:::impl
            JData["Data Utils"]:::impl
            JModels["Models"]:::impl
            JTrain["Trainers"]:::impl
            JMain["main.py"]:::impl
        end
        subgraph "PyTorch Pipeline"
            PTcfg["ConfigLib"]:::impl
            PTcorr["Adam Correction"]:::impl
            PTopacus["Opacus Text Classification"]:::impl
            PTglue["Text Classification GLUE"]:::impl
        end
    end

    %% External Services
    SQL["SQL Database"]:::ext
    JSONcfg["JSON Config Files"]:::ext

    %% Connections
    DL1 --> DL2 --> DL3
    DL3 --> Orch
    Configs --> Orch
    Orch --> Models
    Orch --> Samp
    Orch --> Optim
    Samp --> Optim
    Optim --> PA
    Orch -->|"metrics & logs"| AT
    AT --> BestParams
    AT --> TuneJSON
    BestParams --> SQL
    TuneJSON --> JSONcfg
    AT --> RunTune
    RunTune --> Orch

    %% JAX Flow
    DL3 --> JData
    JMain --> JData
    JMain --> JModels
    JMain --> JTrain
    JMain --> Optim
    JMain --> PA

    %% PyTorch Flow
    DL3 --> PTcfg
    PTcfg --> PTcorr
    PTcorr --> PTopacus
    PTopacus --> PTglue
    PTglue --> Optim
    PTglue --> PA

    %% Click Events
    click DL1 "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/download_datasets.py"
    click DL2 "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/dataset_readers.py"
    click DL3 "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/input_pipeline.py"
    click Models "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/models.py"
    click Models "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/normalizations.py"
    click Optim "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/optimizers.py"
    click Optim "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/optim/adam_optimizers.py"
    click PA "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/privacy_accountants.py"
    click Samp "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/sampler.py"
    click AT "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/autotune.py"
    click TuneJSON "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/autotune/tune_adambc_example.json"
    click BestParams "https://github.com/ubc-systopia/dp-adambc/tree/main/GNN/autotune/best_params/"
    click RunTune "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/run_tuning.sh"
    click Configs "https://github.com/ubc-systopia/dp-adambc/tree/main/GNN/configs/"
    click Orch "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/main.py"
    click Orch "https://github.com/ubc-systopia/dp-adambc/blob/main/GNN/train.py"
    click Jcfg "https://github.com/ubc-systopia/dp-adambc/blob/main/jax_implementation/configlib.py"
    click JData "https://github.com/ubc-systopia/dp-adambc/tree/main/jax_implementation/data_utils/"
    click JModels "https://github.com/ubc-systopia/dp-adambc/tree/main/jax_implementation/models/"
    click JTrain "https://github.com/ubc-systopia/dp-adambc/tree/main/jax_implementation/trainer/"
    click JMain "https://github.com/ubc-systopia/dp-adambc/blob/main/jax_implementation/main.py"
    click PTcfg "https://github.com/ubc-systopia/dp-adambc/blob/main/pytorch_implementation/configlib.py"
    click PTcorr "https://github.com/ubc-systopia/dp-adambc/blob/main/pytorch_implementation/adam_corr.py"
    click PTopacus "https://github.com/ubc-systopia/dp-adambc/blob/main/pytorch_implementation/opacus_text_classification.py"
    click PTglue "https://github.com/ubc-systopia/dp-adambc/blob/main/pytorch_implementation/text_classification_glue.py"

    %% Styles
    classDef data fill:#add8e6,stroke:#333,stroke-width:1px;
    classDef core fill:#90ee90,stroke:#333,stroke-width:1px;
    classDef impl fill:#ffa500,stroke:#333,stroke-width:1px;
    classDef tune fill:#dda0dd,stroke:#333,stroke-width:1px;
    classDef ext fill:#dda0dd,stroke-dasharray: 5 5,stroke:#333,stroke-width:1px;
```
