How larocs/hrm works
Project type and purpose
• This is a research-oriented machine-learning codebase implementing the “Hierarchical Reasoning Model” (HRM). It provides end-to-end pipelines for (a) data preparation (ARC, Sudoku, maze), (b) model definition and training (two-level recurrent HRM), and (c) evaluation/visualization (W&B, puzzle_visualizer.html, arc_eval.ipynb).
File structure
• Top-level scripts:
• pretrain.py – orchestrates training (single- or multi-GPU via torchrun)
• evaluate.py – evaluation entry point
• arc_eval.ipynb – notebook for ARC evaluation analysis
• config/arch/*.yaml – YAML configurations for different HRM architectures
• dataset/ – data-building scripts (ARC, maze, Sudoku), plus raw-data subfolders
• models/ – HRM implementation (high-level and low-level modules), layers, losses, embeddings
• utils/ – shared helper functions
• requirements.txt & CUDA/FlashAttention setup instructions in README
README insights
• Two-module recurrent architecture: slow “high level” planner and fast “low level” executor
• No CoT data or pretraining; single forward pass reasoning
• Uses PyTorch, CUDA extensions (FlashAttention), Weights & Biases for logging
Detailed callouts to capture essentials
• Dataset builders as independent processes (can be parallelized or scheduled)
• Multi-GPU training orchestration via torchrun
• Checkpoint lifecycle: create, store, evaluate
• Evaluation split: automated script vs. interactive notebook vs. HTML visualization