How dpaletti/pop works
Project type and purpose
• This is a Python-based hierarchical Multi-Agent Reinforcement Learning (MARL) system built on top of the Grid2Op power-grid environment.
• Its goal is to control a simulated power network by deploying substation agents, grouping them into dynamic communities, and using managers at multiple levels to select and execute grid-control actions.
File structure
• Top-level:
• architectures/ & runs/: TOML files describing various network and agent configurations for training and evaluation.
• pop/: core library—contains agents, community-detection, multiagent system logic, network definitions, configs, and the entrypoint (main.py).
• scripts (.sh, .ipynb): training/evaluation helpers and scratchbooks.
• pyproject.toml & poetry.lock: dependency and build configuration.
• In pop/:
• agents/: GCN-based substation agents, exploration modules (ε-greedy, RND, episodic), replay buffer, manager orchestrators (ray_* variants for distributed training).
• community_detection/: dynamic Louvain-based community detection.
• multiagent_system/: DPOMDP abstractions—base_pop, community managers, dictatorship penalizers, reward distributor, head manager logic.
• networks/: PyTorch (or analogous) modules defining graph convolutional layers, dueling nets, serialization, and a parser for architecture TOML specs.
• configs/: type aliases and run_config tying TOML parameters to Python objects.
Key insights from the README
• Three hierarchical actors:
a) Substation Agent (local decision on its neighborhood)
b) Community Manager (selects one substation agent per community)
c) Head Manager (chooses one community’s action to execute)
• Dynamic community detection via Dynamo (extension of Louvain).
• Execution driven by CLI (pop/main.py) reading run_files/*.toml.
Detail without over-fragmentation
• Every layer (local/community/global) and major module (detection, replay, networks).
• Do not split substation agents into submodules unless you want to call out exploration vs policy network separately; grouping them is fine.