How greatzby/llm8 works
This repository is research code for training language models to plan paths through generated graphs, including multi-stage GraphA composition tasks. The documented simple-graph workflow generates path examples, prepares model-ready data, then trains a NanoGPT-style model. Separate scripts provide instruction-data preparation and Q-learning/GRPO training variants. The excerpts are partial, and the README’s named test_simple.py is absent from the tree, so evaluation wiring is not represented.
Main entry points: train.py NanoGPT Training · ql_grapha_kstage_llama_v6.py Llama Q-learning · grpo_grapha_kstage_qwen_v1.py GRPO Fine-tuning · ql_grapha_kstage_qwen_v6.py Q-learning
Components 16
Graph Data
- Random Graph Examples
data/simple_graph/create_graph.py - GraphA DAG
data/simple_graph/create_graph_graphA.py - Multi-stage DAG
data/simple_graph/create_graph_multihop.py - Graph Artifacts
Dataset Preparation
- Instruction Text
data/simple_graph/textualize_instruction_dataset.py - Token Binaries
data/simple_graph/prepare_composition_promptaware.py - Training Data
Model Training
- NanoGPT Training
train.py - Q-learning
ql_grapha_kstage_qwen_v6.py - Llama Q-learning
ql_grapha_kstage_llama_v6.py - GRPO Fine-tuning
grpo_grapha_kstage_qwen_v1.py - Token GRPO
train_token_grpo.py - Model Checkpoints
Model Core
- GPT Model
model.py
Other components
- Researcher
- Hugging Face Models
Connections 28
- Researcher → Random Graph Examples: runs
- Researcher → GraphA DAG: runs
- Researcher → Multi-stage DAG: runs
- Random Graph Examples → Graph Artifacts: writes graph data
- GraphA DAG → Graph Artifacts: writes graph data
- Multi-stage DAG → Graph Artifacts: writes graph data
- Researcher → Instruction Text: runs
- Instruction Text → Training Data: writes text data
- Researcher → Token Binaries: runs
- Token Binaries → Training Data: writes token data
- Researcher → NanoGPT Training: runs
- Training Data → NanoGPT Training: provides training data
- NanoGPT Training → GPT Model: uses
- NanoGPT Training → Model Checkpoints: saves
- Researcher → Q-learning: runs
- Researcher → Llama Q-learning: runs
- Researcher → GRPO Fine-tuning: runs
- Researcher → Token GRPO: runs
- Training Data → Q-learning: provides graph data
- Training Data → Llama Q-learning: provides graph data
- Training Data → GRPO Fine-tuning: provides graph data
- Training Data → Token GRPO: provides graph data
- Q-learning → Model Checkpoints: saves
- Llama Q-learning → Model Checkpoints: saves
- GRPO Fine-tuning → Model Checkpoints: saves
- Token GRPO → Model Checkpoints: saves
- Hugging Face Models → Llama Q-learning: loads base model
- Hugging Face Models → GRPO Fine-tuning: loads base model