# hammale2003/nanogpt-gradio-experiment-workbench architecture

> Architecture diagram of the GitHub repository hammale2003/nanogpt-gradio-experiment-workbench, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hammale2003/nanogpt-gradio-experiment-workbench
- Repository: https://github.com/hammale2003/nanogpt-gradio-experiment-workbench

## Overview

##### Project type and purpose
- This is a full-stack Python application combining a web UI (Gradio) with a machine‐learning training core (nanoGPT).
- Purpose: let users interactively configure and launch nanoGPT training runs—toggling optimizations (gradient accumulation, activation recompute, data parallelism), preparing datasets, and visualizing training metrics in real time.

##### File structure
- Top‐level files:
- README.md – project overview
- gradio_app.py – defines Gradio interface
- train_core.py – encapsulates the core training loop and toggles
- requirements.txt – dependencies
- nanoGPT/ subdirectory: a cloned copy of Karpathy’s nanoGPT repo, containing:
- model.py, train.py, sample.py, bench.py – core GPT code
- config/ – predefined experiment configurations
- data/ – dataset folders (openwebtxt, shakespeare, char‐based) with prepare.py scripts
- notebooks/ – analysis artifacts
- Patterns:
- UI/backend separation (gradio_app.py ↔ train_core.py)
- Modular “config → data prep → model init → training loop → metrics” pipeline
- Optional toggle flags for advanced parallelism

##### README
- Confirms architecture: Gradio UI collects hyperparameters and optimization flags → calls train_core → uses nanoGPT library and data preparation scripts → yields live metrics to UI
- Highlights modularity: core logic isolated in train_core.py; data prep handled by dataset‐specific scripts; reliance on PyTorch and torch.utils.checkpoint for recompute.

##### Elements
- Blue for UI-related components
- Green for compute/training components
- Orange for data/datasets
- Gray for external libraries and hardware

##### Detailed guidance
- On the right, group “Metrics & Visualization” and “Run Summary & Comparative Analysis”.
- Finally, annotate external dependencies at the bottom: “PyTorch”, “Gradio”, “Matplotlib”, “Pandas”, “Datasets on disk”.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Interaction
    User["User"]:::external
    User -->|"interacts with"| GradioUI

    %% UI Layer
    subgraph "UI Layer"
        GradioUI["Gradio Web UI"]:::ui
        ConfigMgr["Configuration Manager"]:::ui
    end
    User --> GradioUI
    GradioUI -->|"collects hyperparameters"| ConfigMgr

    %% Backend Layer
    subgraph "Backend Training Layer"
        TrainCore["Training Core"]:::compute

        subgraph "Data Preparation" 
            OWTP["OpenWebText Prepare"]:::data
            ShkW["Shakespeare (word) Prepare"]:::data
            ShkC["Shakespeare (char) Prepare"]:::data
        end

        DataLoader["DataLoader"]:::compute
        ModelInit["Model Init\n(nanoGPT/model.py)"]:::compute
        TrainLoop["Training Loop\n(nanoGPT/train.py)"]:::compute
        Optimizer["Optimizer & Checkpoint"]:::compute
    end

    ConfigMgr -->|"config payload"| TrainCore
    TrainCore --> OWTP
    TrainCore --> ShkW
    TrainCore --> ShkC
    OWTP --> DataLoader
    ShkW --> DataLoader
    ShkC --> DataLoader
    DataLoader --> ModelInit
    ModelInit --> TrainLoop
    TrainLoop --> Optimizer

    %% GPU and Metrics
    TrainLoop -->|"forward/backward"| GPU["GPU(s) via PyTorch"]:::external
    GPU -->|"loss, iter, mem"| Metrics["Live Metrics"]:::ui
    Metrics --> GradioUI
    Optimizer -->|"checkpoint files"| Storage

    %% Visualization & Summary
    subgraph "Results & Analysis"
        MetricsVis["Metrics & Visualization"]:::ui
        Summary["Run Summary\n& Comparative Analysis"]:::ui
    end
    Metrics --> MetricsVis
    MetricsVis --> Summary

    %% Data Storage
    subgraph "Data Storage" 
        Storage["Checkpoints & Binaries"]:::data
        DatasetFiles["Dataset Storage"]:::data
    end
    Optimizer --> Storage
    DataLoader --> DatasetFiles

    %% External Dependencies
    subgraph "External Dependencies"
        PyTorch["PyTorch (CUDA)"]:::external
        GradioDep["Gradio"]:::external
        Matplotlib["Matplotlib"]:::external
        Pandas["Pandas"]:::external
        DatasetsDisk["Datasets on Disk"]:::external
    end
    TrainCore --> PyTorch
    GradioUI --> GradioDep
    MetricsVis --> Matplotlib
    MetricsVis --> Pandas
    DataLoader --> DatasetsDisk

    %% Click Events
    click GradioUI "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/gradio_app.py"
    click ConfigMgr "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/configurator.py"
    click TrainCore "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/train_core.py"
    click ModelInit "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/model.py"
    click TrainLoop "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/train.py"
    click OWTP "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/data/openwebtxt/prepare.py"
    click ShkW "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/data/shakespeare/prepare.py"
    click ShkC "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/data/shakespeare_char/prepare.py"
    click Storage "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/data/shakespeare_char/train.bin"
    click DatasetFiles "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/data/shakespeare_char/val.bin"
    click DatasetFiles "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/data/shakespeare_char/meta.pkl"
    click bench "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/bench.py"
    click transformer "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/notebooks/transformer_sizing.ipynb"
    click scaling "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/nanoGPT/notebooks/scaling_laws.ipynb"
    click Configs "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/tree/main/nanoGPT/config/"
    click Requirements "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/requirements.txt"
    click README "https://github.com/hammale2003/nanogpt-gradio-experiment-workbench/blob/main/README.md"

    %% Styles
    classDef ui fill:#D6EAF8,stroke:#1B4F72,color:#1B2631;
    classDef compute fill:#D5F5E3,stroke:#186A3B,color:#145A32;
    classDef data fill:#FCF3CF,stroke:#B9770E,color:#784212;
    classDef external fill:#EAECEE,stroke:#616A6B,color:#1C2833;
```
