# saisurag/irl2grid architecture

> Architecture diagram of the GitHub repository saisurag/irl2grid, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/saisurag/irl2grid
- Repository: https://github.com/saisurag/irl2grid

## Overview

Interpretable RL2Grid trains and evaluates reinforcement-learning policies for Grid2Op power-grid operations. The principal workflow starts in `main.py`, creates vectorized environments, dispatches a selected algorithm, and evaluates and checkpoints its policy. Interpretable methods include tree optimization, distillation, VIPER, and symbolic regression; benchmark neural-RL methods provide comparison policies and teachers. The README also describes standalone checkpoint evaluation, fidelity analysis, and policy visualization. Edges reflect calls or references shown in excerpts, the SOURCE INDEX, or README; unsampled subsystem wiring is omitted.

Main entry points: [`main.py`](https://github.com/saisurag/irl2grid/blob/main/main.py) (Training dispatcher), [`alg/dtpo/core.py`](https://github.com/saisurag/irl2grid/blob/main/alg/dtpo/core.py) (DTPO tree learner), [`env/eval.py`](https://github.com/saisurag/irl2grid/blob/main/env/eval.py) (Policy evaluator), [`alg/s_reinforce/core.py`](https://github.com/saisurag/irl2grid/blob/main/alg/s_reinforce/core.py) (S-REINFORCE learner)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["Experiment entry"]
  node_main["Training dispatcher<br/>[main.py]"]
end

subgraph group_environment["Grid environment"]
  node_envsetup["Environment setup<br/>[utils.py]"]
  node_gridconfig["Grid configuration<br/>[config.py]"]
  node_gridlogic["Grid dynamics and rewards<br/>[cmdp.py]"]
end

subgraph group_learning["Policy learning"]
  node_dtpo["DTPO tree learner<br/>[core.py]"]
  node_dtpoagent["Tree policy<br/>[agent.py]"]
  node_distill["Policy distillation<br/>[distill.py]"]
  node_viper["VIPER learner<br/>[core.py]"]
  node_dagger["DAgger learner<br/>[core.py]"]
  node_sreinforce["S-REINFORCE learner<br/>[core.py]"]
  node_dqn["DQN baseline<br/>[core.py]"]
  node_ppo["PPO baseline<br/>[core.py]"]
  node_sac["SAC baseline<br/>[core.py]"]
  node_td3["TD3 baseline<br/>[core.py]"]
  node_lagrppo["Lagrangian PPO<br/>[core.py]"]
  node_p3o["P3O learner<br/>[core.py]"]
  node_sreinforceagent["Symbolic agent<br/>[agent.py]"]
end

subgraph group_support["Evaluation and support"]
  node_actionreduce["Action curation"]
  node_evaluator["Policy evaluator<br/>[eval.py]"]
  node_checkpoint[("Checkpoint storage<br/>[checkpoint.py]")]
  node_logger["Run metrics<br/>[logger.py]"]
  node_evalentry["Checkpoint evaluation<br/>[eval_ckpt.py]"]
  node_honesteval["Interpretable evaluation<br/>[honest_eval_any.py]"]
  node_fidelity["Fidelity evaluation<br/>[fidelity_eval.py]"]
  node_treeviz["Tree visualization<br/>[viz_dtpo_tree.py]"]
end

subgraph group_results["Results analysis"]
  node_analysis["Result analysis"]
  node_visuals["Result visuals"]
end

node_researcher(("Experiment runner"))

node_researcher -->|"starts run"| node_main
node_main -->|"creates environments"| node_envsetup
node_main -->|"uses configuration"| node_gridconfig
node_main -->|"dispatches"| node_dtpo
node_main -->|"dispatches"| node_viper
node_main -->|"dispatches"| node_dagger
node_main -->|"dispatches"| node_sreinforce
node_main -->|"dispatches"| node_dqn
node_main -->|"dispatches"| node_ppo
node_main -->|"dispatches"| node_sac
node_main -->|"dispatches"| node_td3
node_main -->|"dispatches"| node_lagrppo
node_dtpo -->|"uses policy"| node_dtpoagent
node_dtpo -->|"uses distillation"| node_distill
node_distill -->|"curates actions"| node_actionreduce
node_viper -->|"builds oracle"| node_dqn
node_viper -->|"builds oracle"| node_ppo
node_sreinforce -->|"creates agent"| node_sreinforceagent
node_dtpo -->|"evaluates policy"| node_evaluator
node_sreinforce -->|"evaluates policy"| node_evaluator
node_dtpo -->|"saves checkpoints"| node_checkpoint
node_ppo -->|"saves checkpoints"| node_checkpoint
node_dtpo -->|"logs metrics"| node_logger
node_sreinforce -->|"logs metrics"| node_logger
node_envsetup -->|"uses grid helpers"| node_gridlogic
node_evalentry -->|"evaluates checkpoint"| node_evaluator
node_honesteval -->|"evaluates policy"| node_evaluator
node_honesteval -->|"loads policy"| node_evalentry
node_researcher -->|"rescoring run"| node_evalentry
node_researcher -->|"checks agreement"| node_fidelity
node_researcher -->|"visualizes policy"| node_treeviz
node_researcher -->|"analyzes results"| node_analysis
node_analysis -->|"generates visuals"| node_visuals
node_p3o -->|"evaluates policy"| node_evaluator

click node_main "https://github.com/saisurag/irl2grid/blob/main/main.py"
click node_envsetup "https://github.com/saisurag/irl2grid/blob/main/env/utils.py"
click node_gridconfig "https://github.com/saisurag/irl2grid/blob/main/env/config.py"
click node_gridlogic "https://github.com/saisurag/irl2grid/blob/main/env/cmdp.py"
click node_dtpo "https://github.com/saisurag/irl2grid/blob/main/alg/dtpo/core.py"
click node_dtpoagent "https://github.com/saisurag/irl2grid/blob/main/alg/dtpo/agent.py"
click node_distill "https://github.com/saisurag/irl2grid/blob/main/alg/distill.py"
click node_viper "https://github.com/saisurag/irl2grid/blob/main/alg/viper/core.py"
click node_dagger "https://github.com/saisurag/irl2grid/blob/main/alg/dagger/core.py"
click node_sreinforce "https://github.com/saisurag/irl2grid/blob/main/alg/s_reinforce/core.py"
click node_dqn "https://github.com/saisurag/irl2grid/blob/main/alg/dqn/core.py"
click node_ppo "https://github.com/saisurag/irl2grid/blob/main/alg/ppo/core.py"
click node_sac "https://github.com/saisurag/irl2grid/blob/main/alg/sac/core.py"
click node_td3 "https://github.com/saisurag/irl2grid/blob/main/alg/td3/core.py"
click node_lagrppo "https://github.com/saisurag/irl2grid/blob/main/alg/lagr_ppo/core.py"
click node_p3o "https://github.com/saisurag/irl2grid/blob/main/alg/p3o/core.py"
click node_sreinforceagent "https://github.com/saisurag/irl2grid/blob/main/alg/s_reinforce/agent.py"
click node_actionreduce "https://github.com/saisurag/irl2grid/blob/main/common/action_reduction.py"
click node_evaluator "https://github.com/saisurag/irl2grid/blob/main/env/eval.py"
click node_checkpoint "https://github.com/saisurag/irl2grid/blob/main/common/checkpoint.py"
click node_logger "https://github.com/saisurag/irl2grid/blob/main/common/logger.py"
click node_evalentry "https://github.com/saisurag/irl2grid/blob/main/eval_ckpt.py"
click node_honesteval "https://github.com/saisurag/irl2grid/blob/main/honest_eval_any.py"
click node_fidelity "https://github.com/saisurag/irl2grid/blob/main/fidelity_eval.py"
click node_treeviz "https://github.com/saisurag/irl2grid/blob/main/viz_dtpo_tree.py"
click node_analysis "https://github.com/saisurag/irl2grid/tree/main/Final-Results/analysis"
click node_visuals "https://github.com/saisurag/irl2grid/blob/main/Final-Results/analysis/make_visuals_out.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main toneBlue
class node_envsetup,node_gridconfig,node_gridlogic toneAmber
class node_dtpo,node_dtpoagent,node_distill,node_viper,node_dagger,node_sreinforce,node_dqn,node_ppo,node_sac,node_td3,node_lagrppo,node_p3o,node_sreinforceagent toneMint
class node_actionreduce,node_evaluator,node_checkpoint,node_logger,node_evalentry,node_honesteval,node_fidelity,node_treeviz toneRose
class node_analysis,node_visuals,node_researcher toneIndigo
```

## Components

### Experiment entry

- [Training dispatcher](https://github.com/saisurag/irl2grid/blob/main/main.py): `main.py`

### Grid environment

- [Environment setup](https://github.com/saisurag/irl2grid/blob/main/env/utils.py): `env/utils.py`
- [Grid configuration](https://github.com/saisurag/irl2grid/blob/main/env/config.py): `env/config.py`
- [Grid dynamics and rewards](https://github.com/saisurag/irl2grid/blob/main/env/cmdp.py): `env/cmdp.py`

### Policy learning

- [DTPO tree learner](https://github.com/saisurag/irl2grid/blob/main/alg/dtpo/core.py): `alg/dtpo/core.py`
- [Tree policy](https://github.com/saisurag/irl2grid/blob/main/alg/dtpo/agent.py): `alg/dtpo/agent.py`
- [Policy distillation](https://github.com/saisurag/irl2grid/blob/main/alg/distill.py): `alg/distill.py`
- [VIPER learner](https://github.com/saisurag/irl2grid/blob/main/alg/viper/core.py): `alg/viper/core.py`
- [DAgger learner](https://github.com/saisurag/irl2grid/blob/main/alg/dagger/core.py): `alg/dagger/core.py`
- [S-REINFORCE learner](https://github.com/saisurag/irl2grid/blob/main/alg/s_reinforce/core.py): `alg/s_reinforce/core.py`
- [DQN baseline](https://github.com/saisurag/irl2grid/blob/main/alg/dqn/core.py): `alg/dqn/core.py`
- [PPO baseline](https://github.com/saisurag/irl2grid/blob/main/alg/ppo/core.py): `alg/ppo/core.py`
- [SAC baseline](https://github.com/saisurag/irl2grid/blob/main/alg/sac/core.py): `alg/sac/core.py`
- [TD3 baseline](https://github.com/saisurag/irl2grid/blob/main/alg/td3/core.py): `alg/td3/core.py`
- [Lagrangian PPO](https://github.com/saisurag/irl2grid/blob/main/alg/lagr_ppo/core.py): `alg/lagr_ppo/core.py`
- [P3O learner](https://github.com/saisurag/irl2grid/blob/main/alg/p3o/core.py): `alg/p3o/core.py`
- [Symbolic agent](https://github.com/saisurag/irl2grid/blob/main/alg/s_reinforce/agent.py): `alg/s_reinforce/agent.py`

### Evaluation and support

- [Action curation](https://github.com/saisurag/irl2grid/blob/main/common/action_reduction.py): `common/action_reduction.py`
- [Policy evaluator](https://github.com/saisurag/irl2grid/blob/main/env/eval.py): `env/eval.py`
- [Checkpoint storage](https://github.com/saisurag/irl2grid/blob/main/common/checkpoint.py): `common/checkpoint.py`
- [Run metrics](https://github.com/saisurag/irl2grid/blob/main/common/logger.py): `common/logger.py`
- [Checkpoint evaluation](https://github.com/saisurag/irl2grid/blob/main/eval_ckpt.py): `eval_ckpt.py`
- [Interpretable evaluation](https://github.com/saisurag/irl2grid/blob/main/honest_eval_any.py): `honest_eval_any.py`
- [Fidelity evaluation](https://github.com/saisurag/irl2grid/blob/main/fidelity_eval.py): `fidelity_eval.py`
- [Tree visualization](https://github.com/saisurag/irl2grid/blob/main/viz_dtpo_tree.py): `viz_dtpo_tree.py`

### Results analysis

- [Result analysis](https://github.com/saisurag/irl2grid/tree/main/Final-Results/analysis): `Final-Results/analysis`
- [Result visuals](https://github.com/saisurag/irl2grid/blob/main/Final-Results/analysis/make_visuals_out.py): `Final-Results/analysis/make_visuals_out.py`

### Other components

- **Experiment runner**

## Connections

- Experiment runner → Training dispatcher: starts run (evidence: `README.md`)
- Training dispatcher → Environment setup: creates environments (evidence: `main.py`)
- Training dispatcher → Grid configuration: uses configuration (evidence: `main.py`)
- Training dispatcher → DTPO tree learner: dispatches (evidence: `main.py`)
- Training dispatcher → VIPER learner: dispatches (evidence: `main.py`)
- Training dispatcher → DAgger learner: dispatches (evidence: `main.py`)
- Training dispatcher → S-REINFORCE learner: dispatches (evidence: `main.py`)
- Training dispatcher → DQN baseline: dispatches (evidence: `main.py`)
- Training dispatcher → PPO baseline: dispatches (evidence: `main.py`)
- Training dispatcher → SAC baseline: dispatches (evidence: `main.py`)
- Training dispatcher → TD3 baseline: dispatches (evidence: `main.py`)
- Training dispatcher → Lagrangian PPO: dispatches (evidence: `main.py`)
- DTPO tree learner → Tree policy: uses policy (evidence: `alg/dtpo/core.py`)
- DTPO tree learner → Policy distillation: uses distillation (evidence: `alg/dtpo/core.py`)
- Policy distillation → Action curation: curates actions (evidence: `alg/distill.py`)
- VIPER learner → DQN baseline: builds oracle (evidence: `alg/viper/core.py`)
- VIPER learner → PPO baseline: builds oracle (evidence: `alg/viper/core.py`)
- S-REINFORCE learner → Symbolic agent: creates agent (evidence: `alg/s_reinforce/core.py`)
- DTPO tree learner → Policy evaluator: evaluates policy (evidence: `alg/dtpo/core.py`)
- S-REINFORCE learner → Policy evaluator: evaluates policy (evidence: `alg/s_reinforce/core.py`)
- DTPO tree learner → Checkpoint storage: saves checkpoints (evidence: `alg/dtpo/core.py`)
- PPO baseline → Checkpoint storage: saves checkpoints (evidence: `alg/ppo/core.py`)
- DTPO tree learner → Run metrics: logs metrics (evidence: `alg/dtpo/core.py`)
- S-REINFORCE learner → Run metrics: logs metrics (evidence: `alg/s_reinforce/core.py`)
- Environment setup → Grid dynamics and rewards: uses grid helpers (evidence: `env/utils.py`)
- Checkpoint evaluation → Policy evaluator: evaluates checkpoint (evidence: `eval_ckpt.py`)
- Interpretable evaluation → Policy evaluator: evaluates policy (evidence: `honest_eval_any.py`)
- Interpretable evaluation → Checkpoint evaluation: loads policy (evidence: `honest_eval_any.py`)
- Experiment runner → Checkpoint evaluation: rescoring run (evidence: `README.md`)
- Experiment runner → Fidelity evaluation: checks agreement (evidence: `README.md`)
- Experiment runner → Tree visualization: visualizes policy (evidence: `README.md`)
- Experiment runner → Result analysis: analyzes results (evidence: `README.md`)
- Result analysis → Result visuals: generates visuals (evidence: `Final-Results/analysis/make_visuals_out.py`)
- P3O learner → Policy evaluator: evaluates policy (evidence: `alg/p3o/core.py`)
