# nav3054/balle2018-scale-hyperprior architecture

> Architecture diagram of the GitHub repository nav3054/balle2018-scale-hyperprior, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nav3054/balle2018-scale-hyperprior
- Repository: https://github.com/nav3054/balle2018-scale-hyperprior

## Overview

##### Project type and purpose
- This is a machine-learning training pipeline (not a web app).
- Purpose: train and evaluate a learned image compression model (Ballé et al. 2018) using TensorFlow 2.x.
- Key use cases: ingest large image datasets, train the scale-hyperprior compression model, log and checkpoint results, evaluate on held-out data.

##### File structure
- Top-level files
- README.md, LICENSE
- data/
- imagenet_patch_loader.py – dataset extraction, splitting, patch creation
- models/
- balle_hyperprior_2018_model.py – the Keras Model implementation (encoder/decoder, entropy models)
- training/
- train.py – main entrypoint, argument parsing, orchestration
- callbacks.py – checkpoint management, early stopping, image logging
- losses.py – rate-distortion loss implementation
- No frontend or microservices; purely a single-process training pipeline.

##### Extract additional insights from the README
- CLI interface for configuring data paths, hyperparameters
- Modular utilities for data loading, model definition, loss calculation, callbacks
- Outputs: model checkpoints, image logs, final metrics
- Underlying pattern: sequential ML pipeline with clearly separated concerns.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Command-Line Entrypoint
    CLI["Command-Line Entrypoint"]:::training
    click CLI "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/training/train.py"

    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        DataLoader["Imagenet Patch Loader"]:::data
        click DataLoader "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/data/imagenet_patch_loader.py"
        TFDS["TensorFlow Dataset Objects\n(train, val, test)"]:::data
    end

    %% Model Definition
    subgraph "Model Definition"
        Model["Balle2018Hyperprior Model"]:::model
        click Model "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/models/balle_hyperprior_2018_model.py"
        subgraph "Encoders & Decoders"
            MainEnc["Main Encoder (g_a)"]:::model
            HyperEnc["Hyper Encoder (h_a)"]:::model
            HyperDec["Hyper Decoder (h_s)"]:::model
            MainDec["Main Decoder (g_s)"]:::model
        end
        Entropy["Entropy Models"]:::model
    end

    %% Loss Computation
    Loss["Rate-Distortion Loss"]:::training
    click Loss "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/training/losses.py"

    %% Training Orchestrator
    subgraph "Training Orchestrator"
        Optimizer["Adam Optimizer"]:::training
        click Optimizer "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/training/train.py"
        TrainLoop["train_step / test_step"]:::training
        click TrainLoop "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/training/train.py"
        Callbacks["Callbacks\n(checkpoint, early stopping,\nimage logging, NaN termination)"]:::training
        click Callbacks "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/training/callbacks.py"
    end

    %% Outputs / Storage
    subgraph "Outputs / Storage"
        Checkpoints["Model Checkpoints"]:::storage
        click Checkpoints "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/training/callbacks.py"
        Logs["Image Logs & Final Metrics"]:::storage
        click Logs "https://github.com/nav3054/balle2018-scale-hyperprior/blob/main/training/callbacks.py"
    end

    %% Data Flow & Control Flow
    CLI -->|"args: data paths, hyperparams"| DataLoader
    DataLoader -->|"image patches"| TFDS
    TFDS -->|"batches"| TrainLoop
    TrainLoop -->|"forward pass"| Model
    Model -->|"latent representations"| Entropy
    Entropy -->|"entropy codes"| Model
    Model -->|"reconstructed images"| Loss
    Loss -->|"loss value"| TrainLoop
    TrainLoop -->|"gradients"| Optimizer
    Optimizer -->|"update weights"| Model
    TrainLoop -->|"invoke callbacks"| Callbacks
    Callbacks -->|"checkpoint write"| Checkpoints
    Callbacks -->|"log images & metrics"| Logs

    %% Styles
    classDef data fill:#add8e6,stroke:#333,stroke-width:1px
    classDef model fill:#90ee90,stroke:#333,stroke-width:1px
    classDef training fill:#ffa500,stroke:#333,stroke-width:1px
    classDef storage fill:#d3d3d3,stroke:#333,stroke-width:1px
```
