# dbsxodud-11/pag architecture

> Architecture diagram of the GitHub repository dbsxodud-11/pag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dbsxodud-11/pag
- Repository: https://github.com/dbsxodud-11/pag

## Overview

##### Project type and purpose
- This is an ML research/training pipeline (“PAG”) for learning to sample effective and diverse text‐to‐image prompts.
- It orchestrates data preparation, model training (on GPT-2 with specialized objectives), aesthetic scoring (via Stable Diffusion + CLIP models), and final evaluation.

##### File structure
- Top level:
- main.py – CLI entrypoint for training
- eval.py – CLI entrypoint for evaluation
- dataset.py – data loading & preprocessing
- gfn_trainer.py – core training loop & objective functions
- utils.py – shared helper functions
- save/ – holds pretrained and newly trained GPT-2 SFT checkpoints
- prompts/ – JSONL prompt datasets + convert_format.ipynb
- aesthetic/ – submodule containing pretrained aesthetic predictors, inference scripts, and training code
- requirements.txt – Python dependencies
- Patterns:
- Clear separation of entrypoints (main vs. eval), data-layer (dataset.py + prompts), model-layer (save/, aesthetic/), and training logic (gfn_trainer.py).

##### Key insights from README
- Uses GPT-2 (“lm_name”) and Stable Diffusion (“sd_name”) with aesthetic scoring (“aes”).
- Workflow: install deps → download pretrained SFT and prompts → run convert_format → train with main.py → evaluate with eval.py.
- External resources: HuggingFace CLIP, Stable-Diffusion checkpoints, Microsoft LMOps Promptist data.

##### Main components & interactions (
a. Data Ingestion Layer
- prompts/* (raw JSONL) → convert_format.ipynb → standardized dataset → dataset.py
b. Training Orchestrator
- main.py
- invokes dataset.py to load prompts
- constructs model:
- GPT-2 SFT (pretrained in save/gpt2-sft-position-final)
- Stable Diffusion (CompVis/stable-diffusion-v1-4 via API)
- Aesthetic predictor (aesthetic/inference.py loads .pth models)
- gfn_trainer.py implements: flow reactivation, reward decomposition, reweighted training
- utils.py for logging, checkpointing
- output: new SFT checkpoint → save/pag
c. Evaluation Orchestrator
- eval.py loads save/pag checkpoint + prompts for evaluation
- runs generation + aesthetic scoring to produce metrics
d. External Services & Dependencies
- HuggingFace Transformers & Diffusers
- OpenAI CLIP (installed via git+https://github.com/openai/CLIP.git)
- Pretrained weights (placed under save/ and aesthetic/)

##### Architectural patterns & design principles
- Modular: distinct layers for data, model, training logic, evaluation
- CLI-driven orchestration (main.py / eval.py) for reproducibility
- Pluggable scoring modules (aesthetic/)
- Separation of concerns: dataset vs. trainer vs. inference vs. evaluation

## Diagram (Mermaid)

```mermaid
graph TB
    %% Data Preparation Layer
    subgraph "Data Preparation"
        DP1["prompts/"]:::storage
        DP2["convert_format.ipynb"]:::code
        DP3["dataset.py"]:::code
    end

    DP1 --> DP2
    DP2 --> DP3

    %% Model Components
    subgraph "Model Components"
        MC1["GPT-2 SFT Checkpoint"]:::storage
        MC2["Stable Diffusion API"]:::external
        MC3["aesthetic/inference.py"]:::code
        MC4[".pth Models"]:::storage
    end

    MC4 --> MC3

    %% Training Engine
    subgraph "Training Engine"
        TE1["main.py"]:::code
        TE2["gfn_trainer.py"]:::code
        TE3["utils.py"]:::code
        TE4["save/pag"]:::storage
    end

    DP3 --> TE1
    TE1 --> TE2
    TE2 --> MC2
    TE2 --> MC3
    TE2 -->|updates weights| TE4
    TE3 -.-> TE1
    TE3 -.-> TE2

    %% Evaluation Engine
    subgraph "Evaluation Engine"
        EE1["eval.py"]:::code
        EE2["prompts/*.jsonl"]:::storage
        EE3["Evaluation Metrics"]:::data
    end

    TE4 --> EE1
    EE2 --> EE1
    EE1 --> MC3
    EE1 --> EE3

    %% External Resources
    subgraph "External Resources"
        ER1["HuggingFace Transformers & Diffusers"]:::external
        ER2["OpenAI CLIP Repo"]:::external
        ER3["requirements.txt"]:::code
        ER4["aesthetic/setup-req.sh"]:::code
    end

    ER3 --> TE1
    ER3 --> EE1
    ER1 --> MC2
    ER1 --> TE2
    ER2 --> MC3

    %% Click Events
    click DP1 "https://github.com/dbsxodud-11/pag/tree/main/prompts/"
    click DP2 "https://github.com/dbsxodud-11/pag/blob/main/prompts/convert_format.ipynb"
    click DP3 "https://github.com/dbsxodud-11/pag/blob/main/dataset.py"
    click MC1 "https://github.com/dbsxodud-11/pag/tree/main/save/gpt2-sft-position-final/latest/"
    click MC3 "https://github.com/dbsxodud-11/pag/blob/main/aesthetic/inference.py"
    click TE1 "https://github.com/dbsxodud-11/pag/blob/main/main.py"
    click TE2 "https://github.com/dbsxodud-11/pag/blob/main/gfn_trainer.py"
    click TE3 "https://github.com/dbsxodud-11/pag/blob/main/utils.py"
    click TE4 "https://github.com/dbsxodud-11/pag/tree/main/save/"
    click EE1 "https://github.com/dbsxodud-11/pag/blob/main/eval.py"
    click EE2 "https://github.com/dbsxodud-11/pag/tree/main/prompts/"
    click ER3 "https://github.com/dbsxodud-11/pag/blob/main/requirements.txt"
    click ER4 "https://github.com/dbsxodud-11/pag/blob/main/aesthetic/setup-req.sh"

    %% Styles
    classDef storage fill:#cce5ff,stroke:#339;
    classDef code fill:#d4edda,stroke:#393;
    classDef data fill:#fff3cd,stroke:#993;
    classDef external fill:#f8d7da,stroke:#933;
```
