# dpaletti/pop architecture

> Architecture diagram of the GitHub repository dpaletti/pop, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dpaletti/pop
- Repository: https://github.com/dpaletti/pop

## Overview

##### Project type and purpose
- This is a Python-based hierarchical Multi-Agent Reinforcement Learning (MARL) system built on top of the Grid2Op power-grid environment.
- Its goal is to control a simulated power network by deploying substation agents, grouping them into dynamic communities, and using managers at multiple levels to select and execute grid-control actions.

##### File structure
- Top-level:
- architectures/ & runs/: TOML files describing various network and agent configurations for training and evaluation.
- pop/: core library—contains agents, community-detection, multiagent system logic, network definitions, configs, and the entrypoint (main.py).
- scripts (.sh, .ipynb): training/evaluation helpers and scratchbooks.
- pyproject.toml & poetry.lock: dependency and build configuration.
- In pop/:
- agents/: GCN-based substation agents, exploration modules (ε-greedy, RND, episodic), replay buffer, manager orchestrators (ray_* variants for distributed training).
- community_detection/: dynamic Louvain-based community detection.
- multiagent_system/: DPOMDP abstractions—base_pop, community managers, dictatorship penalizers, reward distributor, head manager logic.
- networks/: PyTorch (or analogous) modules defining graph convolutional layers, dueling nets, serialization, and a parser for architecture TOML specs.
- configs/: type aliases and run_config tying TOML parameters to Python objects.

##### Key insights from the README
- Three hierarchical actors:
a) Substation Agent (local decision on its neighborhood)
b) Community Manager (selects one substation agent per community)
c) Head Manager (chooses one community’s action to execute)
- Dynamic community detection via Dynamo (extension of Louvain).
- Execution driven by CLI (`pop/main.py`) reading run_files/*.toml.

##### Detail without over-fragmentation
- Every layer (local/community/global) and major module (detection, replay, networks).
- Do not split substation agents into submodules unless you want to call out exploration vs policy network separately; grouping them is fine.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% External Systems
    Env["Grid2Op Simulator"]:::external

    %% Configuration
    Config["Configuration Loader\n(TOML Parser)"]:::config

    %% Controller
    Controller["Training Loop / Controller\n(pop/main.py)"]:::controller

    %% Agents Layer
    subgraph "Local Layer"
        SubAgents["Substation Agents\n(N parallel)\nGCN policy + exploration"]:::agent
        Replay["Replay Buffer"]:::data
    end

    %% Community Detection
    Detector["Community Detector\n(Dynamo Louvain)"]:::detector

    %% Community Managers Layer
    subgraph "Community Layer"
        ComManagers["Community Managers\n(one per community)"]:::manager
    end

    %% Head Manager Layer
    Head["Head Manager\n(Expert POP)"]:::head

    %% Data Flows
    Env -->|"observations"| SubAgents
    SubAgents -->|"experience"| Replay
    SubAgents -->|"graph state"| Detector
    Detector -->|"communities"| ComManagers
    SubAgents -->|"proposed actions"| ComManagers
    ComManagers -->|"selected actions & summaries"| Head
    Head -->|"execute action"| Env

    %% Training Updates
    Controller -.->|"update weights"| SubAgents
    Controller -.->|"update weights"| ComManagers
    Controller -.->|"update weights"| Head
    Controller -.->|"sample experience"| Replay

    %% Configuration Flows
    Config -->|"hyperparams & specs"| Controller
    Config -->|"architectures"| SubAgents
    Config -->|"settings"| Detector
    Config -->|"settings"| ComManagers
    Config -->|"settings"| Head

    %% Click Events
    click SubAgents "https://github.com/dpaletti/pop/blob/main/pop/agents/base_gcn_agent.py"
    click Replay "https://github.com/dpaletti/pop/blob/main/pop/agents/replay_buffer.py"
    click Detector "https://github.com/dpaletti/pop/blob/main/pop/community_detection/community_detector.py"
    click ComManagers "https://github.com/dpaletti/pop/blob/main/pop/multiagent_system/base_pop.py"
    click Head "https://github.com/dpaletti/pop/blob/main/pop/multiagent_system/expert_pop.py"
    click Config "https://github.com/dpaletti/pop/blob/main/pop/configs/run_config.py"
    click Controller "https://github.com/dpaletti/pop/blob/main/pop/main.py"

    %% Styles
    classDef external fill:#f9f,stroke:#333,stroke-width:2px
    classDef config fill:#fffbcc,stroke:#333,stroke-width:1px
    classDef controller fill:#ffe6e6,stroke:#333,stroke-width:1px
    classDef agent fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef data fill:#e2f0d9,stroke:#333,stroke-width:1px,stroke-dasharray: 5 5
    classDef detector fill:#d1e7dd,stroke:#333,stroke-width:1px
    classDef manager fill:#d4edda,stroke:#333,stroke-width:1px
    classDef head fill:#f8d7da,stroke:#333,stroke-width:1px
```
