# bobdavis84/autoresearch architecture

> Architecture diagram of the GitHub repository bobdavis84/autoresearch, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bobdavis84/autoresearch
- Repository: https://github.com/bobdavis84/autoresearch

## Overview

1. **Repository purpose**
   - This repo is a small, single-GPU LLM research sandbox for autonomous experimentation.
   - The core workflow is: change model code, run a short training job, compare validation quality, and keep or discard the change.

2. **Primary architectural boundary**
   - The codebase is intentionally split into a **fixed infrastructure layer** and an **editable research layer**.
   - `prepare.py` is treated as stable support code.
   - `train.py` is the main mutable experiment surface.
   - `program.md` is the human-authored instruction layer that steers the agent.

3. **Data and preprocessing subsystem**
   - `prepare.py` owns one-time setup: dataset download, tokenizer training, and fixed constants.
   - It also provides runtime utilities used by training, especially dataloading and evaluation support.
   - This file is the bridge from raw text data to model-ready token sequences.

4. **Training subsystem**
   - `train.py` contains the full model and optimization stack: GPT model definition, Muon + AdamW optimizer logic, and the training loop.
   - This is where architecture changes happen: model depth, attention pattern, batch size, hyperparameters, or optimizer details.
   - The file is designed to be edited repeatedly by the autonomous agent.

5. **Agent orchestration layer**
   - `program.md` acts as the “control program” for the research agent.
   - It describes how the agent should behave, what to edit, and how to iterate.
   - The human updates this file to change the research strategy without touching training code directly.

6. **Experiment loop and feedback flow**
   - The loop is: agent reads `program.md`, edits `train.py`, training runs for a fixed wall-clock budget, then validation results determine whether the change is promising.
   - The feedback signal is `val_bpb` (validation bits per byte), where lower is better.
   - This makes every run comparable within the same time budget.

7. **Compute/runtime assumptions**
   - The repo is optimized for a **single NVIDIA GPU** and a **5-minute training budget** per run.
   - That fixed runtime constraint is part of the architecture: it shapes how model size, batch size, and evaluation cost are chosen.
   - The runtime is intentionally simple rather than distributed or multi-node.

8. **Technology stack**
   - The project is Python-based and managed with `uv`.
   - PyTorch is the core ML runtime.
   - The dependency and environment state live in `pyproject.toml` and `uv.lock`.

9. **Supporting files and tooling**
   - `setup.sh` likely automates environment/bootstrap steps.
   - `README.linux.md` suggests platform-specific setup guidance.
   - `analysis.ipynb` is for offline exploration or inspection, not part of the main execution path.

10. **Repository intent from an architecture perspective**
   - The design minimizes moving parts: one stable data/prep layer, one editable training file, one agent instruction file.
   - That separation makes it easy to diagram a closed-loop research system: data preparation feeds training, training produces metrics, and the agent uses those metrics to decide the next code change.

Main entry points: [`train.py`](https://github.com/bobdavis84/autoresearch/blob/master/train.py) (Training), [`analysis.ipynb`](https://github.com/bobdavis84/autoresearch/blob/master/analysis.ipynb) (Analysis), [`prepare.py`](https://github.com/bobdavis84/autoresearch/blob/master/prepare.py) (Prepare), [`program.md`](https://github.com/bobdavis84/autoresearch/blob/master/program.md) (Program)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_fixed_infra["Fixed infra"]
  node_prepare_py["Prepare<br/>stable support<br/>[prepare.py]"]
  node_data_setup[("Data setup<br/>bootstrap<br/>[prepare.py]")]
  node_dataloader["Dataloader<br/>runtime utility<br/>[prepare.py]"]
  node_eval_support(("Eval support<br/>runtime utility<br/>[prepare.py]"))
end

subgraph group_research_loop["Research loop"]
  node_program_md["Program<br/>agent prompt<br/>[program.md]"]
  node_train_py["Training<br/>editable runtime<br/>[train.py]"]
  node_model["GPT model<br/>model def<br/>[train.py]"]
  node_optimizer["Optimizers<br/>optim stack<br/>[train.py]"]
  node_loop(("Train loop<br/>execution loop<br/>[train.py]"))
  node_validation["val_bpb<br/>metric"]
end

subgraph group_tooling["Tooling"]
  node_env["Env<br/>python tooling<br/>[pyproject.toml]"]
  node_setup["Bootstrap<br/>shell setup<br/>[setup.sh]"]
  node_analysis["Analysis<br/>notebook<br/>[analysis.ipynb]"]
end

node_program_md -->|"directs edits"| node_train_py
node_prepare_py -->|"supplies utils"| node_train_py
node_data_setup -->|"feeds batches"| node_dataloader
node_dataloader -->|"inputs"| node_train_py
node_eval_support -->|"computes"| node_validation
node_train_py -->|"contains"| node_model
node_train_py -->|"contains"| node_optimizer
node_train_py -->|"runs"| node_loop
node_loop -->|"reports"| node_validation
node_validation -->|"guides next step"| node_program_md
node_env -->|"supports"| node_train_py
node_setup -->|"bootstraps"| node_env
node_analysis -.->|"inspects"| node_train_py
node_analysis -.->|"inspects"| node_prepare_py

click node_program_md "https://github.com/bobdavis84/autoresearch/blob/master/program.md"
click node_train_py "https://github.com/bobdavis84/autoresearch/blob/master/train.py"
click node_model "https://github.com/bobdavis84/autoresearch/blob/master/train.py"
click node_optimizer "https://github.com/bobdavis84/autoresearch/blob/master/train.py"
click node_loop "https://github.com/bobdavis84/autoresearch/blob/master/train.py"
click node_prepare_py "https://github.com/bobdavis84/autoresearch/blob/master/prepare.py"
click node_data_setup "https://github.com/bobdavis84/autoresearch/blob/master/prepare.py"
click node_dataloader "https://github.com/bobdavis84/autoresearch/blob/master/prepare.py"
click node_eval_support "https://github.com/bobdavis84/autoresearch/blob/master/prepare.py"
click node_env "https://github.com/bobdavis84/autoresearch/blob/master/pyproject.toml"
click node_setup "https://github.com/bobdavis84/autoresearch/blob/master/setup.sh"
click node_analysis "https://github.com/bobdavis84/autoresearch/blob/master/analysis.ipynb"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_prepare_py,node_data_setup,node_dataloader,node_eval_support toneBlue
class node_program_md,node_train_py,node_model,node_optimizer,node_loop,node_validation toneAmber
class node_env,node_setup,node_analysis toneMint
```

## Components

### Fixed infra

Stable setup, data preparation, and runtime support code.

- [Prepare](https://github.com/bobdavis84/autoresearch/blob/master/prepare.py): stable support · Fixed infrastructure for dataset setup, tokenizer training, constants, and shared runtime utilities. · `prepare.py`
- [Data setup](https://github.com/bobdavis84/autoresearch/blob/master/prepare.py): bootstrap · One-time dataset download and tokenizer training. · `prepare.py`
- [Dataloader](https://github.com/bobdavis84/autoresearch/blob/master/prepare.py): runtime utility · Builds model-ready token batches from prepared text data. · `prepare.py`
- [Eval support](https://github.com/bobdavis84/autoresearch/blob/master/prepare.py): runtime utility · Shared evaluation helpers used to measure validation quality consistently. · `prepare.py`

### Research loop

Editable experiment surface and agent control path.

- [Program](https://github.com/bobdavis84/autoresearch/blob/master/program.md): agent prompt · Human-authored instructions that steer the research agent's edits and iteration strategy. · `program.md`
- [Training](https://github.com/bobdavis84/autoresearch/blob/master/train.py): editable runtime · Main mutable experiment surface containing the model, optimizer, and training loop. · `train.py`
- [GPT model](https://github.com/bobdavis84/autoresearch/blob/master/train.py): model def · Transformer/GPT architecture definition and related forward logic. · `train.py`
- [Optimizers](https://github.com/bobdavis84/autoresearch/blob/master/train.py): optim stack · Muon and AdamW optimization logic and parameter handling. · `train.py`
- [Train loop](https://github.com/bobdavis84/autoresearch/blob/master/train.py): execution loop · Runs short training jobs under a fixed budget and evaluates changes. · `train.py`
- **val_bpb**: metric · Validation bits-per-byte feedback signal used to compare runs; lower is better.

### Tooling

Environment, docs, and offline inspection aids.

- [Env](https://github.com/bobdavis84/autoresearch/blob/master/pyproject.toml): python tooling · Dependency and environment definition managed with uv and PyTorch. · `pyproject.toml`
- [Bootstrap](https://github.com/bobdavis84/autoresearch/blob/master/setup.sh): shell setup · Repository bootstrap and environment setup script. · `setup.sh`
- [Analysis](https://github.com/bobdavis84/autoresearch/blob/master/analysis.ipynb): notebook · Offline exploration and inspection notebook. · `analysis.ipynb`

## Connections

- Program → Training: directs edits · Instruction layer tells the agent how to mutate the training code.
- Prepare → Training: supplies utils · Training imports fixed preprocessing, dataloading, and evaluation helpers.
- Data setup → Dataloader: feeds batches · Prepared data and tokenizer outputs are turned into trainable sequences.
- Dataloader → Training: inputs · Batches flow into the mutable training code.
- Eval support → val_bpb: computes · Evaluation helpers produce the validation metric used for comparison.
- Training → GPT model: contains · The training file defines the core model architecture.
- Training → Optimizers: contains · The training file defines optimizer configuration and update logic.
- Training → Train loop: runs · The main execution path is the short-budget training loop.
- Train loop → val_bpb: reports · Each run ends with a validation score for the agent to judge.
- val_bpb → Program: guides next step · The metric closes the feedback loop that informs the next edit.
- Env → Training: supports · Python environment and dependencies enable training execution.
- Bootstrap → Env: bootstraps · Setup script prepares the managed environment.
- Analysis → Training: inspects · Notebook is used for offline analysis of the experiment code and results.
- Analysis → Prepare: inspects · Notebook may also examine preprocessing and evaluation behavior.
