How gfnorg/chunk-gfn works
Project Type and Purpose
• This is a research-oriented Python package for reinforcement learning (RL) and GFlowNet experiments.
• It uses PyTorch Lightning for training loops, Hydra for configuration, SLURM (via sbatch scripts) for cluster scheduling, and optionally WandB or local file logging.
Main Components
A. Entry Point & Config
– main.py + run_all.sh + sbatch_scripts/
– Hydra config tree (configs/)
B. Experiment Orchestrator
– PyTorch Lightning Trainer
– Hydra instantiation (chunkgfn/utils/instantiators.py)
C. Core Modules (chunkgfn/)
1. environment/ : defines MDP or generative environment
2. models/ : neural nets (MLP, RNN, GNN, Transformer, encoders)
3. algo/ : RL & GFlowNet algorithms (A2C, SAC, OptionCritic, TB-GFN, etc.)
4. replay_buffer/: stores transitions for off-policy updates
5. schedulers/ : LR and temperature schedules
6. rewards/ : loss or reward functions (MSE, etc.)
7. utils/ : logging, caching, Hydra + PyLogger helpers
D. Data & Artifacts
– data/ : raw datasets, download scripts
– logs/ : checkpoint and TensorBoard/WandB outputs
E. External Services
– SLURM (job submission via sbatch scripts)
– WandB or local file logger
– Filesystem (datasets, logs, mode pickles)
Data Flows and Interactions
1. User/CI invokes python main.py or calls an sbatch wrapper
2. Hydra reads YAML configs → builds a config object
3. Instantiators create Environment, Model, Algorithm, ReplayBuffer, Scheduler
4. Lightning Trainer kicks off training loop:
a. At each step, Trainer → algorithm.sample_action(state)
b. Environment returns next_state, reward → stored in ReplayBuffer
c. Algorithm updates Model parameters (backprop via PyTorch Lightning)
5. Trainer logs metrics & checkpoints to logs/ and WandB
6. At the end: results and artifacts in logs/, optional downstream analysis via notebooks/
Architectural Patterns & Design Principles
• Modular plugin architecture (Hydra + dynamic instantiation)
• Separation of concerns:
– Environments (MDP definitions) vs. Algorithms (update rules) vs. Models (NN architectures)
• Experiment reproducibility: config files per experiment, SLURM scripts
• Extensibility: add new envs, algos, reward modules by dropping files in chunkgfn/\*