# larocs/hrm architecture

> Architecture diagram of the GitHub repository larocs/hrm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/larocs/hrm
- Repository: https://github.com/larocs/hrm

## Overview

##### Project type and purpose
- This is a research-oriented machine-learning codebase implementing the “Hierarchical Reasoning Model” (HRM). It provides end-to-end pipelines for (a) data preparation (ARC, Sudoku, maze), (b) model definition and training (two-level recurrent HRM), and (c) evaluation/visualization (W&B, puzzle_visualizer.html, arc_eval.ipynb).

##### File structure
- Top-level scripts:
- pretrain.py – orchestrates training (single- or multi-GPU via torchrun)
- evaluate.py – evaluation entry point
- arc_eval.ipynb – notebook for ARC evaluation analysis
- config/arch/*.yaml – YAML configurations for different HRM architectures
- dataset/ – data-building scripts (ARC, maze, Sudoku), plus raw-data subfolders
- models/ – HRM implementation (high-level and low-level modules), layers, losses, embeddings
- utils/ – shared helper functions
- requirements.txt & CUDA/FlashAttention setup instructions in README

##### README insights
- Two-module recurrent architecture: slow “high level” planner and fast “low level” executor
- No CoT data or pretraining; single forward pass reasoning
- Uses PyTorch, CUDA extensions (FlashAttention), Weights & Biases for logging

##### Detailed callouts to capture essentials
- Dataset builders as independent processes (can be parallelized or scheduled)
- Multi-GPU training orchestration via torchrun
- Checkpoint lifecycle: create, store, evaluate
- Evaluation split: automated script vs. interactive notebook vs. HTML visualization

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Configuration Manager"
        ArchConfig["hrm_v1.yaml"]:::code
        PreConfig["cfg_pretrain.yaml"]:::code
    end

    subgraph "Data Preparation Pipeline"
        ARCBuild["build_arc_dataset.py"]:::code
        MazeBuild["build_maze_dataset.py"]:::code
        SudokuBuild["build_sudoku_dataset.py"]:::code
        RawData["raw-data/"]:::artifact
    end

    subgraph "Data Loader & Preprocessing"
        Loader["common.py"]:::code
    end

    subgraph "HRM Model"
        HRMAct["hrm_act_v1.py"]:::code
        Layers["layers.py"]:::code
        Embedding["sparse_embedding.py"]:::code
        Losses["losses.py"]:::code
        ModelCommon["common.py"]:::code
    end

    subgraph "Training Orchestrator"
        Pretrain["pretrain.py"]:::code
        Checkpoint["Checkpoints"]:::artifact
    end

    subgraph "Evaluation & Logging"
        Evaluate["evaluate.py"]:::code
        Notebook["arc_eval.ipynb"]:::code
        Visualizer["puzzle_visualizer.html"]:::artifact
        Npyjs["npyjs.js"]:::code
    end

    subgraph "External Services & Tools"
        CUDA["CUDA / FlashAttention"]:::external
        WandB["Weights & Biases"]:::external
        HF["Hugging Face"]:::external
    end

    %% Data and Control Flows
    ArchConfig -->|load hyperparameters & architecture| Pretrain
    PreConfig -->|load hyperparameters & architecture| Pretrain
    ArchConfig -->|load hyperparameters & architecture| Evaluate
    PreConfig -->|load hyperparameters & architecture| Evaluate

    Pretrain -->|invoke| ARCBuild
    Pretrain -->|invoke| MazeBuild
    Pretrain -->|invoke| SudokuBuild
    ARCBuild --> RawData
    MazeBuild --> RawData
    SudokuBuild --> RawData

    RawData --> Loader
    Loader --> Pretrain
    Loader --> Evaluate

    Pretrain --> HRMAct
    HRMAct --> Layers
    HRMAct --> Embedding
    HRMAct --> Losses
    HRMAct --> ModelCommon

    HRMAct -->|communicates| CUDA
    Pretrain -->|stores| Checkpoint
    Checkpoint --> HF
    Pretrain -->|logs metrics| WandB

    Evaluate --> Checkpoint
    Evaluate --> Loader
    Evaluate --> HRMAct
    Evaluate -->|logs metrics| WandB
    Evaluate --> Notebook

    RawData --> Npyjs
    Npyjs --> Visualizer

    %% Click Events
    click ArchConfig "https://github.com/larocs/hrm/blob/main/config/arch/hrm_v1.yaml"
    click PreConfig "https://github.com/larocs/hrm/blob/main/config/cfg_pretrain.yaml"
    click ARCBuild "https://github.com/larocs/hrm/blob/main/dataset/build_arc_dataset.py"
    click MazeBuild "https://github.com/larocs/hrm/blob/main/dataset/build_maze_dataset.py"
    click SudokuBuild "https://github.com/larocs/hrm/blob/main/dataset/build_sudoku_dataset.py"
    click RawData "https://github.com/larocs/hrm/tree/main/dataset/raw-data/"
    click Loader "https://github.com/larocs/hrm/blob/main/dataset/common.py"
    click HRMAct "https://github.com/larocs/hrm/blob/main/models/hrm/hrm_act_v1.py"
    click Layers "https://github.com/larocs/hrm/blob/main/models/layers.py"
    click Embedding "https://github.com/larocs/hrm/blob/main/models/sparse_embedding.py"
    click Losses "https://github.com/larocs/hrm/blob/main/models/losses.py"
    click ModelCommon "https://github.com/larocs/hrm/blob/main/models/common.py"
    click Pretrain "https://github.com/larocs/hrm/blob/main/pretrain.py"
    click Evaluate "https://github.com/larocs/hrm/blob/main/evaluate.py"
    click Notebook "https://github.com/larocs/hrm/blob/main/arc_eval.ipynb"
    click Visualizer "https://github.com/larocs/hrm/blob/main/assets/puzzle_visualizer.html"
    click Npyjs "https://github.com/larocs/hrm/blob/main/assets/npyjs.js"
    click Checkpoint "https://github.com/larocs/hrm/tree/main/checkpoints/"

    %% Styles
    classDef code fill:#D0E6FF,stroke:#0366d6,stroke-width:1px;
    classDef external fill:#E2F0D9,stroke:#2E7D32,stroke-width:1px;
    classDef artifact fill:#FFF3CD,stroke:#CC9A06,stroke-width:1px;
```
