How sapientinc/hrm works
Project type and purpose
• This is a machine-learning research codebase for the “Hierarchical Reasoning Model” (HRM), not a web or enterprise full-stack app.
• It provides:
• Dataset builders for puzzles (ARC, Sudoku, Maze)
• Model definitions and losses
• Training orchestration (pretrain.py)
• Evaluation scripts (evaluate.py + arc_eval.ipynb)
• Visualization tool (puzzle_visualizer.html)
• Configuration management for different architectures and pre-training settings
File structure
• Top-level directories and responsibilities:
• config/ ← YAML specs for model architectures and training
• dataset/ ← raw data + builders for ARC, Sudoku, Maze
• models/ ← HRM modules (high-level planner, low-level module), layers, losses, embeddings
• utils/ ← shared functions
• assets/ ← static JS for dataset visualization
• Key scripts:
• pretrain.py ← main training driver using PyTorch + CUDA extensions
• evaluate.py ← checkpoint evaluation on held-out puzzles
• arc_eval.ipynb ← notebook for final ARC metrics
• Support files:
• puzzle_visualizer.html ← interactive dataset browser
• requirements.txt, .gitmodules, .gitignore
Extract architectural insights from the README
• Two interdependent recurrent modules (high-level abstract planner, low-level detail processor)
• Single forward pass hierarchical design (27M parameters)
• Integration with PyTorch, CUDA, FlashAttention, Weights & Biases (W&B)
a. Main components
• Data Layer
– Raw puzzle repositories (ARC-AGI, ARC-AGI-2, ConceptARC)
– Dataset builders (build_arc_dataset.py, build_sudoku_dataset.py, build_maze_dataset.py)
– Processed datasets on disk
• Model Layer
– HRM high-level module (models/hrm/hrm_act_v1.py)
– HRM low-level module and shared components (models/common.py, layers, sparse_embedding)
– Loss functions (models/losses.py)
• Training Orchestrator
– pretrain.py
– Hyperparameter/config loader (config/*.yaml)
– CUDA extensions (FlashAttention)
• Evaluation & Visualization
– evaluate.py
– arc_eval.ipynb
– puzzle_visualizer.html
• External Services & Infrastructure
– Weights & Biases for logging
– GPU compute cluster (CUDA toolkit)
– Python environment (requirements.txt)
b. Relationships & data flow
1. Raw data → Dataset builders → Processed dataset
2. Processed dataset → DataLoader → Training script
3. Training script + config → HRM model → GPU → Checkpoint outputs
4. Metrics → W&B API
5. Checkpoints + dataset → evaluate.py → metrics, which arc_eval.ipynb consumes
6. Processed dataset → puzzle_visualizer.html (static asset)
c. Architectural patterns & design principles
• Modular separation: dataset, model, utils, config
• Config-driven orchestration (YAML + CLI overrides)
• Single-pass hierarchical recurrence (high- vs. low-level modules)
d. Key technologies/frameworks
• PyTorch with CUDA extensions (FlashAttention)
• Hydra-style config management in config/
• Weights & Biases integration
• Jupyter notebook for post-processing (arc_eval.ipynb)
• Static HTML/JS for dataset exploration (puzzle_visualizer.html)