# andrewk1/pytorch-deep-bayesian-bandits architecture

> Architecture diagram of the GitHub repository andrewk1/pytorch-deep-bayesian-bandits, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andrewk1/pytorch-deep-bayesian-bandits
- Repository: https://github.com/andrewk1/pytorch-deep-bayesian-bandits

## Overview

##### Project Type and Purpose
- This is an open-source research tool—a Python/ PyTorch library and experimental framework—for “Deep Bayesian Contextual Bandits.”
- It integrates an unsupervised representation learner (a VAE) with contextual bandit algorithms (deep MLP + Thompson sampling) to run sequential decision-making experiments on datasets (MNIST, Mushroom, Yelp, etc.).

##### High-Level Architectural Overview
- The system is a monolithic ML pipeline composed of:
a. Data sources and dataset management
b. Data loading & preprocessing
c. Unsupervised representation learning (VAE)
d. Contextual bandit engine (MLP + sampling algorithms)
e. Experiment orchestration & result storage
f. Interactive analysis via Jupyter notebooks

##### File-Structure Analysis
- Top-level scripts:
- run_experiment*.py – command-line entry points (single-threaded, multithreaded, dataset-specific)
- bandits/ package:
- core/ – definitions of ContextualBandit, ContextualDataset, Hyperparameter containers
- data/ – data extraction, sampling, caching VAE features, dataset readers (MNIST, Mushroom, raw stock contexts)
- algorithms/ – neural_bandit_model (MLP encoder), neural_linear_sampling (Thompson sampling logic)
- vae/ package:
- model/ – VAE architecture code
- results/ – serialized VAE outputs and embeddings (vae_features.npy)
- vae.ipynb – interactive notebook to train/explore the VAE
- Notebooks at root: prototyping data loading, baseline models, paragraph encodings (BERT).

##### Key Architectural Patterns & Principles
- Modular separation by concern:
- Data ingestion & sampling
- Unsupervised feature learning
- Contextual bandit decision logic
- Experiment control & hyperparameter management
- Clear layer of abstraction: VAE embeddings → Bandit MLP input
- Configurability via hyperparams objects and command-line scripts
- Optional multithreading for parallel experiment runs

##### Core Components
A. Data Layer
- Raw datasets (bandits/data/selected_datasets, raw mushroom, MNIST)
- Data extractor & sampler modules
B. Preprocessing & Feature Cache
- cache_vae_features.py → generates/loads vae_features.npy
C. Unsupervised Learning (VAE)
- vae.model → Encoder/Decoder networks
- vae training pipeline (vae.ipynb & scripts)
D. Contextual Bandit Engine
- MLP contextual encoder (neural_bandit_model.py)
- Thompson sampling logic (neural_linear_sampling.py)
- Core abstractions (ContextualBandit, ContextualDataset)
E. Experiment Orchestrator
- run_experiment*.py – wiring data, VAE embeddings, bandit engine, logging results
- Hyperparameter loader (hyperparams.py)
F. Output & Analysis
- Results directory (vae/results, experiment logs)
- Jupyter notebooks for post-hoc analysis

##### Relationships & Data Flow
- Embeddings feed into ContextualDataset → MLP Bandit Engine
- Bandit Engine outputs decisions & rewards → logged by Experiment Orchestrator
- Results archived → notebooks for visualization

##### Technologies & Frameworks
- Python 3.x, PyTorch for all neural modules
- NumPy for data handling
- Jupyter Notebooks for prototyping/analysis
- Optional Python multiprocessing for multithreaded runs

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Layer
    subgraph "Data Layer"
        RawData["Raw Datasets"]:::data
    end

    %% Unsupervised Phase
    subgraph "Unsupervised Phase"
        DataExtractor["Data Extractor"]:::compute
        DataSampler["Data Sampler"]:::compute
        CacheScript["Cache VAE Features"]:::compute
        EmbeddingStore["vae_features.npy"]:::storage
        VAEModel["VAE Model (vae/model)"]:::compute
        VAEResults["VAE Results"]:::storage
        VAENotebook["VAE Training Notebook"]:::visual
    end

    %% Bandit Phase
    subgraph "Bandit Phase"
        ContextualDataset["ContextualDataset"]:::compute
        ContextualBandit["ContextualBandit"]:::compute
        MLPModel["MLP Contextual Encoder"]:::compute
        ThompsonSampling["Thompson Sampling Logic"]:::compute
        subgraph "Experiment Orchestrator"
            RunnerSingle["run_experiment.py"]:::orchestrator
            RunnerMulti["run_experiment_multithreaded.py"]:::orchestrator
            RunnerMushroom["run_experiment_mushroom.py"]:::orchestrator
            Hyperparams["Hyperparameter Loader"]:::orchestrator
        end
        Logs["Experiment Logs"]:::storage
    end

    %% Analysis Phase
    subgraph "Analysis Phase"
        DataLoader["DataLoader.ipynb"]:::visual
        DataLoaderBaseLine["DataLoaderBaseLine.ipynb"]:::visual
        BERTNotebook["TrainDataEncodetextsBertParagraphEncoder.ipynb"]:::visual
        VAENotebook
    end

    %% Relationships
    RawData --> DataExtractor --> DataSampler --> CacheScript --> EmbeddingStore
    EmbeddingStore --> VAEModel --> VAEResults
    VAENotebook --> VAEModel

    EmbeddingStore --> ContextualDataset --> ContextualBandit
    ContextualBandit --> MLPModel
    ContextualBandit --> ThompsonSampling

    ContextualDataset --> RunnerSingle
    MLPModel --> RunnerSingle
    ThompsonSampling --> RunnerSingle
    Hyperparams --> RunnerSingle

    RunnerSingle --> Logs

    Logs --> DataLoader
    Logs --> DataLoaderBaseLine
    Logs --> BERTNotebook
    Logs --> VAENotebook

    %% Click Events
    click RawData "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/tree/master/bandits/data/selected_datasets/"
    click DataExtractor "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/data/data_extractor.py"
    click DataSampler "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/data/data_sampler.py"
    click CacheScript "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/data/cache_vae_features.py"
    click EmbeddingStore "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/data/vae_features.npy"
    click VAEModel "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/tree/master/vae/model/"
    click VAEResults "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/tree/master/vae/results/"
    click VAENotebook "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/vae/vae.ipynb"
    click ContextualDataset "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/core/contextual_dataset.py"
    click ContextualBandit "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/core/contextual_bandit.py"
    click Hyperparams "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/core/hyperparams.py"
    click MLPModel "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/algorithms/neural_bandit_model.py"
    click ThompsonSampling "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/bandits/algorithms/neural_linear_sampling.py"
    click RunnerSingle "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/run_experiment.py"
    click RunnerMulti "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/run_experiment_multithreaded.py"
    click RunnerMushroom "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/run_experiment_mushroom.py"
    click DataLoader "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/DataLoader.ipynb"
    click DataLoaderBaseLine "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/DataLoaderBaseLine.ipynb"
    click BERTNotebook "https://github.com/andrewk1/pytorch-deep-bayesian-bandits/blob/master/TrainDataEncodetextsBertParagraphEncoder.ipynb"

    %% Styles
    classDef data fill:#1f77b4,stroke:#333, color:#fff;
    classDef compute fill:#17becf,stroke:#333, color:#000;
    classDef storage fill:#ff7f0e,stroke:#333, color:#000;
    classDef orchestrator fill:#9467bd,stroke:#333, color:#fff;
    classDef visual fill:#7f7f7f,stroke:#333, color:#fff;
```
