# mwasifanwar/multi_agent_rl architecture

> Architecture diagram of the GitHub repository mwasifanwar/multi_agent_rl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mwasifanwar/multi_agent_rl
- Repository: https://github.com/mwasifanwar/multi_agent_rl

## Overview

##### Project type and purpose
– Type: A Python library/framework for coordinated multi-agent reinforcement learning.
– Purpose: Provide implementations of popular multi-agent RL algorithms (IQL, MADDPG, MAPPO, QMIX) plus core training infrastructure, sample scripts, and utilities for evaluation and visualization.

##### File structure
– Top-level directories and files:
- algorithms/ – algorithm implementations (iql.py, maddpg.py, mappo.py, qmix.py)
- core/ – foundational RL components (agent_manager.py, experience_replay.py, multi_agent_env.py, policy_network.py, value_network.py)
- examples/ – ready-to-run training scripts (competitive_training.py, cooperative_training.py, mixed_training.py)
- main.py – command-line entry point
- utils/ – helpers (training_utils.py, evaluation_utils.py, visualization_utils.py)
- setup.py, requirements.txt – packaging and dependencies

– Patterns:
- Modular “plugin” style: each algorithm implements a common interface.
- Clear separation of concerns: core RL loop vs. algorithm logic vs. orchestration vs. utilities.

##### README for additional insights
– Very brief, but confirms coordinated multi-agent, collaborative and competitive scenarios.

## Diagram (Mermaid)

```mermaid
graph TB
    %% Entry-Point / CLI Layer
    subgraph "Entry-Point / CLI Layer"
        direction TB
        CLI["main.py"]:::entry
        Examples["examples/"]:::entry
    end

    %% Configuration Manager
    Config["Configuration Manager"]:::core

    %% Core Framework Modules
    subgraph "Core Framework" 
        direction TB
        Env["MultiAgentEnv"]:::core
        AgentMgr["AgentManager"]:::core
        Replay["ExperienceReplay"]:::core
        PolicyNet["PolicyNetwork"]:::core
        ValueNet["ValueNetwork"]:::core
        Trainer["Trainer (RL Loop)"]:::core
    end

    %% Algorithms Plugins
    subgraph "Algorithm Plugins"
        direction TB
        IQL["IQL"]:::alg
        MADDPG["MADDPG"]:::alg
        MAPPO["MAPPO"]:::alg
        QMIX["QMIX"]:::alg
    end

    %% Utilities
    subgraph "Utilities"
        direction TB
        TrainUtil["Training Utilities"]:::util
        EvalUtil["Evaluation Utilities"]:::util
        VizUtil["Visualization Utilities"]:::util
    end

    %% Data Flow
    CLI -->|"runs"| Config
    Examples -->|"provides scripts"| Config
    Config -->|"initializes"| Trainer

    Trainer -->|"reset()/step()"| Env
    Env -->|"observations"| AgentMgr
    AgentMgr -->|"select_actions()"| PolicyNet
    PolicyNet -->|"actions"| Env
    Env -->|"next_states,rewards"| Replay
    Trainer -->|"sample() from Replay"| Replay
    Replay -->|"batches"| IQL
    Replay -->|"batches"| MADDPG
    Replay -->|"batches"| MAPPO
    Replay -->|"batches"| QMIX

    IQL -->|"update()"| PolicyNet
    MADDPG -->|"update()"| PolicyNet
    MAPPO -->|"update()"| PolicyNet
    QMIX -->|"update()"| ValueNet

    %% Utilities connections
    Trainer -->|"log metrics"| TrainUtil
    Trainer -->|"evaluate"| EvalUtil
    Trainer -->|"plot results"| VizUtil

    %% Click Events
    click CLI "https://github.com/mwasifanwar/multi_agent_rl/blob/main/main.py"
    click Examples "https://github.com/mwasifanwar/multi_agent_rl/tree/main/examples/"
    click Config "https://github.com/mwasifanwar/multi_agent_rl/blob/main/main.py"
    click Env "https://github.com/mwasifanwar/multi_agent_rl/blob/main/core/multi_agent_env.py"
    click AgentMgr "https://github.com/mwasifanwar/multi_agent_rl/blob/main/core/agent_manager.py"
    click Replay "https://github.com/mwasifanwar/multi_agent_rl/blob/main/core/experience_replay.py"
    click PolicyNet "https://github.com/mwasifanwar/multi_agent_rl/blob/main/core/policy_network.py"
    click ValueNet "https://github.com/mwasifanwar/multi_agent_rl/blob/main/core/value_network.py"
    click IQL "https://github.com/mwasifanwar/multi_agent_rl/blob/main/algorithms/iql.py"
    click MADDPG "https://github.com/mwasifanwar/multi_agent_rl/blob/main/algorithms/maddpg.py"
    click MAPPO "https://github.com/mwasifanwar/multi_agent_rl/blob/main/algorithms/mappo.py"
    click QMIX "https://github.com/mwasifanwar/multi_agent_rl/blob/main/algorithms/qmix.py"
    click TrainUtil "https://github.com/mwasifanwar/multi_agent_rl/blob/main/utils/training_utils.py"
    click EvalUtil "https://github.com/mwasifanwar/multi_agent_rl/blob/main/utils/evaluation_utils.py"
    click VizUtil "https://github.com/mwasifanwar/multi_agent_rl/blob/main/utils/visualization_utils.py"

    %% Styles
    classDef entry fill:#FFA500,stroke:#333,stroke-width:1px
    classDef core fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef alg fill:#90EE90,stroke:#333,stroke-width:1px
    classDef util fill:#FFFACD,stroke:#333,stroke-width:1px
```
