How saisurag/irl2grid works
Interpretable RL2Grid trains and evaluates reinforcement-learning policies for Grid2Op power-grid operations. The principal workflow starts in main.py, creates vectorized environments, dispatches a selected algorithm, and evaluates and checkpoints its policy. Interpretable methods include tree optimization, distillation, VIPER, and symbolic regression; benchmark neural-RL methods provide comparison policies and teachers. The README also describes standalone checkpoint evaluation, fidelity analysis, and policy visualization. Edges reflect calls or references shown in excerpts, the SOURCE INDEX, or README; unsampled subsystem wiring is omitted.
Main entry points: main.py Training dispatcher · dtpo/core.py DTPO tree learner · eval.py Policy evaluator · s_reinforce/core.py S-REINFORCE learner
Components 28
Experiment entry
- Training dispatcher
main.py
Grid environment
- Environment setup
env/utils.py - Grid configuration
env/config.py - Grid dynamics and rewards
env/cmdp.py
Policy learning
- DTPO tree learner
alg/dtpo/core.py - Tree policy
alg/dtpo/agent.py - Policy distillation
alg/distill.py - VIPER learner
alg/viper/core.py - DAgger learner
alg/dagger/core.py - S-REINFORCE learner
alg/s_reinforce/core.py - DQN baseline
alg/dqn/core.py - PPO baseline
alg/ppo/core.py - SAC baseline
alg/sac/core.py - TD3 baseline
alg/td3/core.py - Lagrangian PPO
alg/lagr_ppo/core.py - P3O learner
alg/p3o/core.py - Symbolic agent
alg/s_reinforce/agent.py
Evaluation and support
- Action curation
common/action_reduction.py - Policy evaluator
env/eval.py - Checkpoint storage
common/checkpoint.py - Run metrics
common/logger.py - Checkpoint evaluation
eval_ckpt.py - Interpretable evaluation
honest_eval_any.py - Fidelity evaluation
fidelity_eval.py - Tree visualization
viz_dtpo_tree.py
Results analysis
- Result analysis
Final-Results/analysis - Result visuals
Final-Results/analysis/make_visuals_out.py
Other components
- Experiment runner
Connections 34
- Experiment runner → Training dispatcher: starts run
README.md - Training dispatcher → Environment setup: creates environments
main.py - Training dispatcher → Grid configuration: uses configuration
main.py - Training dispatcher → DTPO tree learner: dispatches
main.py - Training dispatcher → VIPER learner: dispatches
main.py - Training dispatcher → DAgger learner: dispatches
main.py - Training dispatcher → S-REINFORCE learner: dispatches
main.py - Training dispatcher → DQN baseline: dispatches
main.py - Training dispatcher → PPO baseline: dispatches
main.py - Training dispatcher → SAC baseline: dispatches
main.py - Training dispatcher → TD3 baseline: dispatches
main.py - Training dispatcher → Lagrangian PPO: dispatches
main.py - DTPO tree learner → Tree policy: uses policy
alg/dtpo/core.py - DTPO tree learner → Policy distillation: uses distillation
alg/dtpo/core.py - Policy distillation → Action curation: curates actions
alg/distill.py - VIPER learner → DQN baseline: builds oracle
alg/viper/core.py - VIPER learner → PPO baseline: builds oracle
alg/viper/core.py - S-REINFORCE learner → Symbolic agent: creates agent
alg/s_reinforce/core.py - DTPO tree learner → Policy evaluator: evaluates policy
alg/dtpo/core.py - S-REINFORCE learner → Policy evaluator: evaluates policy
alg/s_reinforce/core.py - DTPO tree learner → Checkpoint storage: saves checkpoints
alg/dtpo/core.py - PPO baseline → Checkpoint storage: saves checkpoints
alg/ppo/core.py - DTPO tree learner → Run metrics: logs metrics
alg/dtpo/core.py - S-REINFORCE learner → Run metrics: logs metrics
alg/s_reinforce/core.py - Environment setup → Grid dynamics and rewards: uses grid helpers
env/utils.py - Checkpoint evaluation → Policy evaluator: evaluates checkpoint
eval_ckpt.py - Interpretable evaluation → Policy evaluator: evaluates policy
honest_eval_any.py - Interpretable evaluation → Checkpoint evaluation: loads policy
honest_eval_any.py - Experiment runner → Checkpoint evaluation: rescoring run
README.md - Experiment runner → Fidelity evaluation: checks agreement
README.md - Experiment runner → Tree visualization: visualizes policy
README.md - Experiment runner → Result analysis: analyzes results
README.md - Result analysis → Result visuals: generates visuals
Final-Results/analysis/make_visuals_out.py - P3O learner → Policy evaluator: evaluates policy
alg/p3o/core.py