How argo-navis-lab/scalelab works
ScaleLab is a measurement lab for small-model reinforcement learning. The principal workflow configures and prices an experiment, constructs a transformer and arithmetic task, runs warm-start or budget-limited RL training, records checkpoints and telemetry, grades runs, then analyzes and exports results. The graph preserves the documented sweep/worker, cost-model, and analysis capabilities. Edges reflect README statements or shown source/index references; several subsystem internals were not sampled, so their links are documented rather than inferred from names.
Main entry points: experiment.py Run orchestration · cli.py Command interface · loop.py RL training loop · costing.py Run cost estimate
Components 26
Experiment runs
- Experiment config
src/scalelab/core/config.py - Run orchestration
src/scalelab/rl/experiment.py - RL training loop
src/scalelab/rl/loop.py - Warm-start loop
src/scalelab/rl/warmstart_loop.py - Answer rollouts
src/scalelab/rl/rollout.py - Policy optimization
src/scalelab/rl/trainer.py
Model and tasks
- Transformer model
src/scalelab/models/transformer.py - Arithmetic tasks
src/scalelab/tasks/arithmetic.py - Digit tokenizer
src/scalelab/tasks/tokenizer.py - Evaluation sets
src/scalelab/tasks/evalsets.py
Run records
- Run registry
src/scalelab/core/registry.py - Checkpoint state
src/scalelab/core/checkpoint.py - Run storage
src/scalelab/core/storage.py - Metric sink
src/scalelab/telemetry/sink.py - Anomaly detection
src/scalelab/telemetry/detectors.py
Cost modeling
- Run cost estimate
src/scalelab/rl/costing.py - RL step model
src/scalelab/costmodel/rlstep.py - Operation model
src/scalelab/costmodel/ops.py - Hardware specs
src/scalelab/costmodel/specs.py
Analysis and planning
- Cost reports
src/scalelab/analysis/costreport.py - Scaling analysis
src/scalelab/analysis/scaling.py - Statistical analysis
src/scalelab/analysis/stats.py - Dataset export
src/scalelab/analysis/export.py - Sweep planning
src/scalelab/core/sweep.py
Other components
- Researcher
- Command interface
src/scalelab/cli.py
Connections 27
- Researcher → Command interface: invokes
README.md - Command interface → Experiment config: loads config
src/scalelab/cli.py - Command interface → Run orchestration: starts run
src/scalelab/cli.py - Run orchestration → Transformer model: constructs
src/scalelab/rl/experiment.py - Run orchestration → Arithmetic tasks: constructs
src/scalelab/rl/experiment.py - Run orchestration → Evaluation sets: loads sets
src/scalelab/rl/experiment.py - Arithmetic tasks → Digit tokenizer: encodes prompts
src/scalelab/tasks/arithmetic.py - Run orchestration → RL training loop: dispatches RL
src/scalelab/rl/experiment.py - Run orchestration → Warm-start loop: dispatches warm start
src/scalelab/rl/experiment.py - RL training loop → Answer rollouts: samples answers
src/scalelab/rl/loop.py - RL training loop → Policy optimization: optimizes policy
src/scalelab/rl/loop.py - RL training loop → Checkpoint state: saves state
src/scalelab/rl/loop.py - Checkpoint state → Run storage: writes artifacts
src/scalelab/core/checkpoint.py - Run orchestration → Run registry: records run
README.md - Run orchestration → Metric sink: records metrics
src/scalelab/rl/experiment.py - Metric sink → Run storage: writes metrics
src/scalelab/telemetry/sink.py - Run orchestration → Anomaly detection: monitors anomalies
src/scalelab/rl/experiment.py - Command interface → Run cost estimate: prices run
README.md - Run cost estimate → RL step model: predicts step
src/scalelab/rl/costing.py - RL step model → Operation model: uses op timings
src/scalelab/costmodel/rlstep.py - Run cost estimate → Hardware specs: uses device spec
src/scalelab/rl/costing.py - Command interface → Cost reports: reports costs
src/scalelab/cli.py - Cost reports → RL step model: reads predictions
src/scalelab/analysis/costreport.py - Command interface → Scaling analysis: fits scaling
src/scalelab/cli.py - Scaling analysis → Statistical analysis: uses statistics
src/scalelab/analysis/scaling.py - Command interface → Dataset export: exports runs
src/scalelab/cli.py - Command interface → Sweep planning: plans sweeps
README.md