# cognizaunce/gpt architecture

> Architecture diagram of the GitHub repository cognizaunce/gpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cognizaunce/gpt
- Repository: https://github.com/cognizaunce/gpt

## Overview

##### Project type and purpose
- This is a machine-learning training and inference pipeline for generating Shakespeare-style poems using a GPT model.  It is not a traditional web or mobile app, but a data-driven research/experimentation codebase with scripts and notebooks for preprocessing, training, checkpoint management, and inference.

##### File structure
- Top-level scripts and notebooks
- CharacterDataset.py – builds a character-level dataset from raw text
- PoemGeneration.py & PoemGeneration.ipynb – inference drivers that load a trained model and generate poems
- ShakespeareGPT.py – likely an alternate inference or orchestration script
- data/ directory
- input.txt, linalg.txt – raw text sources for training
- checkpoints/ directory
- transformers.pt – serialized model weights
- gpt/ package
- gpt.py, gpt1.py – core model definitions (GPT architecture and variants)
- sublayers.py – attention, feed-forward, layer-norm implementations
- trainer.py – training loop, optimizer, batching, logging
- utils/gpt_config.py – hyperparameters and model configuration
- utils/utils.py – auxiliary functions (tokenization, sampling)

##### README for additional insights
- README only states: “ShakespeareGPT. Makes poems that sound like shakespeare.”

##### Detail without over-complicating
- Every top-level script/notebook as a consumer or producer of data/model
- How gpt/utils and gpt/sublayers feed into gpt/gpt and gpt/trainer
- Do not omit the checkpoints folder—model persistence is a critical component

## Diagram (Mermaid)

```mermaid
graph TB
    %% Data Ingestion & Preprocessing
    subgraph "Data Ingestion & Preprocessing"
        RawInput["input.txt"]:::data
        RawLinalg["linalg.txt"]:::data
        Dataset["CharacterDataset.py"]:::data
    end

    %% Model Definition
    subgraph "Model Definition"
        GPTcore["gpt.py"]:::model
        GPT1["gpt1.py"]:::model
        Sublayers["sublayers.py"]:::model
        Config["gpt_config.py"]:::model
    end

    %% Training Pipeline
    subgraph "Training Pipeline"
        Trainer["trainer.py"]:::train
    end

    %% Storage
    subgraph "Storage"
        CheckpointDir[(checkpoints/)]:::storage
        CheckpointFile["transformers.pt"]:::storage
    end

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        PoemGenScript["PoemGeneration.py"]:::inference
        PoemNotebook["PoemGeneration.ipynb"]:::inference
        Shakespeare["ShakespeareGPT.py"]:::inference
    end

    %% Data Flow & Relationships
    RawInput -->|reads| Dataset
    RawLinalg -->|reads| Dataset
    Dataset -->|provides batches| Trainer
    GPTcore -->|architecture| Trainer
    GPT1 -->|architecture variant| Trainer
    Sublayers -->|building blocks| Trainer
    Config -->|hyperparameters| Trainer
    Trainer -->|writes| CheckpointFile
    CheckpointFile -->|stored in| CheckpointDir
    CheckpointFile -->|loaded by| PoemGenScript
    CheckpointFile -->|loaded by| PoemNotebook
    CheckpointFile -->|loaded by| Shakespeare
    GPTcore -->|model definition| PoemGenScript
    GPT1 -->|model definition| PoemGenScript
    Sublayers -->|model blocks| PoemGenScript
    Config -->|config| PoemGenScript

    %% Click Events
    click RawInput "https://github.com/cognizaunce/gpt/blob/main/data/input.txt"
    click RawLinalg "https://github.com/cognizaunce/gpt/blob/main/data/linalg.txt"
    click Dataset "https://github.com/cognizaunce/gpt/blob/main/CharacterDataset.py"
    click GPTcore "https://github.com/cognizaunce/gpt/blob/main/gpt/gpt.py"
    click GPT1 "https://github.com/cognizaunce/gpt/blob/main/gpt/gpt1.py"
    click Sublayers "https://github.com/cognizaunce/gpt/blob/main/gpt/sublayers.py"
    click Config "https://github.com/cognizaunce/gpt/blob/main/gpt/utils/gpt_config.py"
    click Trainer "https://github.com/cognizaunce/gpt/blob/main/gpt/trainer.py"
    click CheckpointDir "https://github.com/cognizaunce/gpt/tree/main/checkpoints/"
    click CheckpointFile "https://github.com/cognizaunce/gpt/blob/main/checkpoints/transformers.pt"
    click PoemGenScript "https://github.com/cognizaunce/gpt/blob/main/PoemGeneration.py"
    click PoemNotebook "https://github.com/cognizaunce/gpt/blob/main/PoemGeneration.ipynb"
    click Shakespeare "https://github.com/cognizaunce/gpt/blob/main/ShakespeareGPT.py"

    %% Styles
    classDef data fill:#d5f5e3,stroke:#2e7d32,stroke-width:1px
    classDef model fill:#bbdefb,stroke:#1976d2,stroke-width:1px
    classDef train fill:#ffe0b2,stroke:#ef6c00,stroke-width:1px
    classDef inference fill:#e1bee7,stroke:#7b1fa2,stroke-width:1px
    classDef storage fill:#cfd8dc,stroke:#455a64,stroke-width:1px
```
