# greatzby/llm8 architecture

> Architecture diagram of the GitHub repository greatzby/llm8, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/greatzby/llm8
- Repository: https://github.com/greatzby/llm8

## Overview

This repository is research code for training language models to plan paths through generated graphs, including multi-stage GraphA composition tasks. The documented simple-graph workflow generates path examples, prepares model-ready data, then trains a NanoGPT-style model. Separate scripts provide instruction-data preparation and Q-learning/GRPO training variants. The excerpts are partial, and the README’s named test_simple.py is absent from the tree, so evaluation wiring is not represented.

Main entry points: [`train.py`](https://github.com/greatzby/llm8/blob/main/train.py) (NanoGPT Training), [`ql_grapha_kstage_llama_v6.py`](https://github.com/greatzby/llm8/blob/main/ql_grapha_kstage_llama_v6.py) (Llama Q-learning), [`grpo_grapha_kstage_qwen_v1.py`](https://github.com/greatzby/llm8/blob/main/grpo_grapha_kstage_qwen_v1.py) (GRPO Fine-tuning), [`ql_grapha_kstage_qwen_v6.py`](https://github.com/greatzby/llm8/blob/main/ql_grapha_kstage_qwen_v6.py) (Q-learning)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Graph Data"]
  node_simple_gen["Random Graph Examples<br/>[create_graph.py]"]
  node_staged_gen["GraphA DAG"]
  node_multihop_gen["Multi-stage DAG"]
  node_graph_files["Graph Artifacts"]
end

subgraph group_prep["Dataset Preparation"]
  node_instruction_text["Instruction Text"]
  node_bin_prep["Token Binaries"]
  node_dataset_files["Training Data"]
end

subgraph group_train["Model Training"]
  node_nanogpt_train["NanoGPT Training<br/>[train.py]"]
  node_qlearn["Q-learning"]
  node_llama_qlearn["Llama Q-learning"]
  node_grpo["GRPO Fine-tuning"]
  node_token_grpo["Token GRPO"]
  node_checkpoints["Model Checkpoints"]
end

subgraph group_runtime["Model Core"]
  node_model["GPT Model<br/>[model.py]"]
end

node_user(("Researcher"))
node_hf_models{{"Hugging Face Models"}}

node_user -->|"runs"| node_simple_gen
node_user -->|"runs"| node_staged_gen
node_user -->|"runs"| node_multihop_gen
node_simple_gen -->|"writes graph data"| node_graph_files
node_staged_gen -->|"writes graph data"| node_graph_files
node_multihop_gen -->|"writes graph data"| node_graph_files
node_user -->|"runs"| node_instruction_text
node_instruction_text -->|"writes text data"| node_dataset_files
node_user -->|"runs"| node_bin_prep
node_bin_prep -->|"writes token data"| node_dataset_files
node_user -->|"runs"| node_nanogpt_train
node_dataset_files -->|"provides training data"| node_nanogpt_train
node_nanogpt_train -->|"uses"| node_model
node_nanogpt_train -->|"saves"| node_checkpoints
node_user -->|"runs"| node_qlearn
node_user -->|"runs"| node_llama_qlearn
node_user -->|"runs"| node_grpo
node_user -->|"runs"| node_token_grpo
node_dataset_files -->|"provides graph data"| node_qlearn
node_dataset_files -->|"provides graph data"| node_llama_qlearn
node_dataset_files -->|"provides graph data"| node_grpo
node_dataset_files -->|"provides graph data"| node_token_grpo
node_qlearn -->|"saves"| node_checkpoints
node_llama_qlearn -->|"saves"| node_checkpoints
node_grpo -->|"saves"| node_checkpoints
node_token_grpo -->|"saves"| node_checkpoints
node_hf_models -.->|"loads base model"| node_llama_qlearn
node_hf_models -.->|"loads base model"| node_grpo

click node_simple_gen "https://github.com/greatzby/llm8/blob/main/data/simple_graph/create_graph.py"
click node_staged_gen "https://github.com/greatzby/llm8/blob/main/data/simple_graph/create_graph_graphA.py"
click node_multihop_gen "https://github.com/greatzby/llm8/blob/main/data/simple_graph/create_graph_multihop.py"
click node_instruction_text "https://github.com/greatzby/llm8/blob/main/data/simple_graph/textualize_instruction_dataset.py"
click node_bin_prep "https://github.com/greatzby/llm8/blob/main/data/simple_graph/prepare_composition_promptaware.py"
click node_nanogpt_train "https://github.com/greatzby/llm8/blob/main/train.py"
click node_qlearn "https://github.com/greatzby/llm8/blob/main/ql_grapha_kstage_qwen_v6.py"
click node_llama_qlearn "https://github.com/greatzby/llm8/blob/main/ql_grapha_kstage_llama_v6.py"
click node_grpo "https://github.com/greatzby/llm8/blob/main/grpo_grapha_kstage_qwen_v1.py"
click node_token_grpo "https://github.com/greatzby/llm8/blob/main/train_token_grpo.py"
click node_model "https://github.com/greatzby/llm8/blob/main/model.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_simple_gen,node_staged_gen,node_multihop_gen,node_graph_files toneBlue
class node_instruction_text,node_bin_prep,node_dataset_files toneAmber
class node_nanogpt_train,node_qlearn,node_llama_qlearn,node_grpo,node_token_grpo,node_checkpoints toneMint
class node_model toneRose
class node_user,node_hf_models toneIndigo
```

## Components

### Graph Data

- [Random Graph Examples](https://github.com/greatzby/llm8/blob/main/data/simple_graph/create_graph.py): `data/simple_graph/create_graph.py`
- [GraphA DAG](https://github.com/greatzby/llm8/blob/main/data/simple_graph/create_graph_graphA.py): `data/simple_graph/create_graph_graphA.py`
- [Multi-stage DAG](https://github.com/greatzby/llm8/blob/main/data/simple_graph/create_graph_multihop.py): `data/simple_graph/create_graph_multihop.py`
- **Graph Artifacts**

### Dataset Preparation

- [Instruction Text](https://github.com/greatzby/llm8/blob/main/data/simple_graph/textualize_instruction_dataset.py): `data/simple_graph/textualize_instruction_dataset.py`
- [Token Binaries](https://github.com/greatzby/llm8/blob/main/data/simple_graph/prepare_composition_promptaware.py): `data/simple_graph/prepare_composition_promptaware.py`
- **Training Data**

### Model Training

- [NanoGPT Training](https://github.com/greatzby/llm8/blob/main/train.py): `train.py`
- [Q-learning](https://github.com/greatzby/llm8/blob/main/ql_grapha_kstage_qwen_v6.py): `ql_grapha_kstage_qwen_v6.py`
- [Llama Q-learning](https://github.com/greatzby/llm8/blob/main/ql_grapha_kstage_llama_v6.py): `ql_grapha_kstage_llama_v6.py`
- [GRPO Fine-tuning](https://github.com/greatzby/llm8/blob/main/grpo_grapha_kstage_qwen_v1.py): `grpo_grapha_kstage_qwen_v1.py`
- [Token GRPO](https://github.com/greatzby/llm8/blob/main/train_token_grpo.py): `train_token_grpo.py`
- **Model Checkpoints**

### Model Core

- [GPT Model](https://github.com/greatzby/llm8/blob/main/model.py): `model.py`

### Other components

- **Researcher**
- **Hugging Face Models**

## Connections

- Researcher → Random Graph Examples: runs
- Researcher → GraphA DAG: runs
- Researcher → Multi-stage DAG: runs
- Random Graph Examples → Graph Artifacts: writes graph data
- GraphA DAG → Graph Artifacts: writes graph data
- Multi-stage DAG → Graph Artifacts: writes graph data
- Researcher → Instruction Text: runs
- Instruction Text → Training Data: writes text data
- Researcher → Token Binaries: runs
- Token Binaries → Training Data: writes token data
- Researcher → NanoGPT Training: runs
- Training Data → NanoGPT Training: provides training data
- NanoGPT Training → GPT Model: uses
- NanoGPT Training → Model Checkpoints: saves
- Researcher → Q-learning: runs
- Researcher → Llama Q-learning: runs
- Researcher → GRPO Fine-tuning: runs
- Researcher → Token GRPO: runs
- Training Data → Q-learning: provides graph data
- Training Data → Llama Q-learning: provides graph data
- Training Data → GRPO Fine-tuning: provides graph data
- Training Data → Token GRPO: provides graph data
- Q-learning → Model Checkpoints: saves
- Llama Q-learning → Model Checkpoints: saves
- GRPO Fine-tuning → Model Checkpoints: saves
- Token GRPO → Model Checkpoints: saves
- Hugging Face Models → Llama Q-learning: loads base model
- Hugging Face Models → GRPO Fine-tuning: loads base model