# juntang-zhuang/adabelief-optimizer architecture

> Architecture diagram of the GitHub repository juntang-zhuang/adabelief-optimizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/juntang-zhuang/adabelief-optimizer
- Repository: https://github.com/juntang-zhuang/adabelief-optimizer

## Overview

##### Project Type and Purpose
- This is an open-source research code repository centered on the AdaBelief optimizer.
- It provides:
- The core AdaBelief implementations for PyTorch and TensorFlow (packaged under pypi_packages).
- A suite of experiments (under PyTorch_Experiments and TensorFlow_Experiments) demonstrating AdaBelief vs. other optimizers on tasks like LSTM language modeling, CIFAR10/ImageNet classification, GAN training, and more.
- Jupyter notebooks and scripts for visualization and reproduction of results.

##### File Structure Analysis
- Top-level:
LICENSE, README.md, imgs/ (figures), pypi_packages/ (versioned optimizer source), PyTorch_Experiments/, TensorFlow_Experiments/
- pypi_packages/
- adabelief_pytorchX.Y.Z/ – core AdaBelief code (AdaBelief.py, setup.py, etc.)
- adabelief_tfX.Y.Z/ – TensorFlow counterpart
- ranger_adabelief – combined optimizer variant
- PyTorch_Experiments/
- LSTM/ – data loaders, model.py, main.py, optimizer variants, curve output
- classification_cifar10/ – models/, optimizers/, run.sh, visualization.ipynb, curve data
- imagenet/, wgan/, wgan-gp/ – each with main scripts, optimizers folder, run.sh, readme.md
- TensorFlow_Experiments/
- convolutional_network.ipynb
- imgs/ – static assets used in README and notebooks

##### README Insights
- Describes the AdaBelief algorithm, hyper-parameters, update rules, and comparisons.
- Directs users to examples in PyTorch_Experiments (via run.sh) and the pypi_packages for the “official” source.

a. Main Components (group into logical tiers or modules):
1. Core Library
- AdaBelief-PyTorch (pypi_packages/adabelief_pytorchX.Y.Z)
- AdaBelief-TF (pypi_packages/adabelief_tfX.Y.Z)
- RangerAdaBelief (pypi_packages/ranger_adabelief)
2. Experiment Framework
- Data Sources (PTB, CIFAR10, ImageNet, GAN toy datasets)
- Data Loaders / Preprocessors (data.py, dataloader.py, getdata.sh)
- Model Definitions (LSTM, ResNet, DenseNet, VGG, SENet, ShuffleNet, Xception, GAN generator/discriminator)
- Optimizer Modules (wrap AdaBelief and competitor optimizers: AdamW, RAdam, Yogi, MSVAG, Fromage, SGD)
- Training Orchestrators
- main.py / run.sh scripts
- experiment-specific entry points (run_all_layerX.py, run.sh)
- Output Stores
- curve/ directories (CSV or serialized metrics)
- checkpoint exports
3. Visualization & Reporting
- Jupyter notebooks (LSTM_test.ipynb, visualization.ipynb, convolutional_network.ipynb)
- imgs/ folder for embedding charts in README
4. Packaging & Distribution
- setup.py in pypi_packages modules
- Versioned folders for release management
- README.md at root describing installation, usage, and citations

b. Relationships & Data Flow
1. Data Source → Data Loader → Model Definition
2. Model + AdaBelief Optimizer (or other optimizer) → Training Loop
3. Training Loop → Metric Collection → ‘curve’ Output Files
4. curve Files → Visualization Notebooks → Plots in imgs/ and notebooks
5. Core Library Modules → Imported by Experiment Framework
6. Packaging Modules → pip install → External Users

c. Architectural Patterns & Principles
- Modular separation: Core library is decoupled from experiment code.
- Versioned packaging: each release lives in its own pypi_packages subfolder.
- Reproducible experiments: standardized run.sh / scripts for each task.
- Extensibility: support for adding new optimizers, models, datasets.

d. Key Technologies
- PyTorch (training, model definition)
- TensorFlow (in TensorFlow_Experiments and adabelief_tf)
- Python scripting (main.py, run.sh)
- Jupyter notebooks for reporting
- pip packaging with setup.py

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Core Library
    subgraph "Core Library"
        direction TB
        ABP["AdaBelief-PyTorch"]:::core
        ABTF["AdaBelief-TF"]:::core
        RAB["RangerAdaBelief"]:::core
    end

    %% Experiment Framework
    subgraph "Experiment Framework"
        direction TB

        subgraph "LSTM Experiments"
            direction TB
            LSTM_DL["Data Loader / Preprocessor"]:::exp
            LSTM_MD["Model Definition"]:::exp
            subgraph "Optimizer Modules"
                direction TB
                LSTM_AdamW["AdamW"]:::exp
                LSTM_RAdam["RAdam"]:::exp
                LSTM_MSVAG["MSVAG"]:::exp
                LSTM_Fromage["Fromage"]:::exp
                LSTM_Yogi["Yogi"]:::exp
            end
            LSTM_ORCH["Training Orchestrator\n(main.py)"]:::exp
            LSTM_EP1["run_all_layer1.py"]:::exp
            LSTM_EP2["run_all_layer2.py"]:::exp
            LSTM_EP3["run_all_layer3.py"]:::exp
            LSTM_OUT["Output Store\n(curve/)"]:::exp

            LSTM_DL -->|"loads data"| LSTM_MD
            LSTM_MD -->|"defines model"| LSTM_AdamW
            LSTM_MD -->|"defines model"| LSTM_RAdam
            LSTM_MD -->|"defines model"| LSTM_MSVAG
            LSTM_MD -->|"defines model"| LSTM_Fromage
            LSTM_MD -->|"defines model"| LSTM_Yogi
            LSTM_AdamW -->|"uses core"| ABP
            LSTM_RAdam -->|"uses core"| ABP
            LSTM_MSVAG -->|"uses core"| ABP
            LSTM_Fromage -->|"uses core"| ABP
            LSTM_Yogi -->|"uses core"| ABP
            LSTM_AdamW -->|"optimizes"| LSTM_ORCH
            LSTM_RAdam -->|"optimizes"| LSTM_ORCH
            LSTM_MSVAG -->|"optimizes"| LSTM_ORCH
            LSTM_Fromage -->|"optimizes"| LSTM_ORCH
            LSTM_Yogi -->|"optimizes"| LSTM_ORCH
            LSTM_EP1 -->|"entry point"| LSTM_ORCH
            LSTM_EP2 -->|"entry point"| LSTM_ORCH
            LSTM_EP3 -->|"entry point"| LSTM_ORCH
            LSTM_ORCH -->|"outputs metrics"| LSTM_OUT
        end

        subgraph "CIFAR10 Classification"
            direction TB
            CIFAR_Models["Model Definitions\n(densenet, resnet, vgg)"]:::exp
            CIFAR_Opts["Optimizer Modules\n(AdamW, Fromage, MSVAG, RAdam, Yogi)"]:::exp
            CIFAR_ORCH["Training Orchestrator\n(main.py, run.sh)"]:::exp
            CIFAR_OUT["Output Store\n(curve/)"]:::exp
            CIFAR_NOTE["Visualization Notebook"]:::vis

            CIFAR_Models -->|"model defined"| CIFAR_Opts
            CIFAR_Opts -->|"uses core"| ABP
            CIFAR_Opts -->|"optimizes"| CIFAR_ORCH
            CIFAR_ORCH -->|"outputs metrics"| CIFAR_OUT
            CIFAR_OUT -->|"visualized in"| CIFAR_NOTE
        end

        subgraph "ImageNet Classification"
            direction TB
            IMAGENET_DL["Data Loader"]:::exp
            IMAGENET_Models["Model Definitions\n(senet, shufflenet, xception)"]:::exp
            IMAGENET_Opts["Optimizers\n(AdamW, RAdam, SGD_GC)"]:::exp
            IMAGENET_ORCH["Training Orchestrator\n(main_when.py, run.sh)"]:::exp
            IMAGENET_EXP["Export Directory"]:::exp

            IMAGENET_DL -->|"loads data"| IMAGENET_Models
            IMAGENET_Models -->|"defines model"| IMAGENET_Opts
            IMAGENET_Opts -->|"uses core"| ABP
            IMAGENET_Opts -->|"optimizes"| IMAGENET_ORCH
            IMAGENET_ORCH -->|"exports checkpoints"| IMAGENET_EXP
        end

        subgraph "GAN Training"
            direction TB
            subgraph "WGAN"
                direction TB
                WGAN_ORCH["main.py"]:::exp
                WGAN_Opts["Optimizers\n(AdamW, Fromage, MSVAG, RAdam, Yogi)"]:::exp
                WGAN_SH["run.sh"]:::exp
                WGAN_RM["readme.md"]:::exp

                WGAN_Opts -->|"uses core"| ABP
                WGAN_Opts -->|"optimizes"| WGAN_ORCH
                WGAN_SH -->|"invokes"| WGAN_ORCH
            end
            subgraph "WGAN-GP"
                direction TB
                WGANGP_ORCH["main.py"]:::exp
                WGANGP_Opts["Optimizers\n(AdamW, Fromage, MSVAG, RAdam, Yogi)"]:::exp
                WGANGP_SH["run.sh"]:::exp
                WGANGP_RM["readme.md"]:::exp

                WGANGP_Opts -->|"uses core"| ABP
                WGANGP_Opts -->|"optimizes"| WGANGP_ORCH
                WGANGP_SH -->|"invokes"| WGANGP_ORCH
            end
        end
    end

    %% Visualization & Reporting
    subgraph "Visualization & Reporting"
        direction TB
        NB_LSTM["LSTM_test.ipynb"]:::vis
        NB_CIFAR["visualization.ipynb"]:::vis
        NB_TF["convolutional_network.ipynb"]:::vis
        IMGS["imgs/"]:::vis

        LSTM_OUT -->|"visualized in"| NB_LSTM
        CIFAR_OUT -->|"visualized in"| NB_CIFAR
        NB_TF
        IMGS
    end

    %% Packaging & Distribution
    subgraph "Packaging & Distribution"
        direction TB
        PKG_PYTORCH["setup.py\n(adabelief_pytorch)"]:::pkg
        PKG_TF["setup.py\n(adabelief_tf)"]:::pkg
        PKG_RANGER["setup.py\n(ranger_adabelief)"]:::pkg
        README["README.md"]:::pkg
        LICENSE["LICENSE"]:::pkg
    end

    %% Click Events
    click ABP "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/pypi_packages/adabelief_pytorch0.2.1/adabelief_pytorch/AdaBelief.py"
    click ABTF "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/pypi_packages/adabelief_tf0.2.1/adabelief_tf/AdaBelief_tf.py"
    click RAB "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/pypi_packages/ranger_adabelief_0.1.0/ranger_adabelief/ranger_adabelief.py"

    click LSTM_DL "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/data.py"
    click LSTM_MD "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/model.py"
    click LSTM_AdamW "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/AdamW.py"
    click LSTM_RAdam "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/RAdam.py"
    click LSTM_MSVAG "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/MSVAG.py"
    click LSTM_Fromage "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/fromage.py"
    click LSTM_Yogi "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/yogi.py"
    click LSTM_ORCH "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/main.py"
    click LSTM_EP1 "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/run_all_layer1.py"
    click LSTM_EP2 "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/run_all_layer2.py"
    click LSTM_EP3 "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/PyTorch_Experiments/LSTM/run_all_layer3.py"

    click PKG_PYTORCH "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/pypi_packages/adabelief_pytorch0.2.1/setup.py"
    click PKG_TF "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/pypi_packages/adabelief_tf0.2.1/setup.py"
    click PKG_RANGER "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/pypi_packages/ranger_adabelief_0.1.0/setup.py"
    click README "https://github.com/juntang-zhuang/adabelief-optimizer/blob/update_0.2.0/README.md"

    %% Styles
    classDef core fill:#AEDFF7,stroke:#000,stroke-width:1px;
    classDef exp fill:#C6F5C6,stroke:#000,stroke-width:1px;
    classDef vis fill:#FFD59E,stroke:#000,stroke-width:1px;
    classDef pkg fill:#E0E0E0,stroke:#000,stroke-width:1px;
```
