# pr-shukla/maddpg-keras architecture

> Architecture diagram of the GitHub repository pr-shukla/maddpg-keras, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pr-shukla/maddpg-keras
- Repository: https://github.com/pr-shukla/maddpg-keras

## Overview

##### Project Type and Purpose
- This is a machine learning project focused on implementing the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm using TensorFlow’s Keras API.
- Its purpose is to provide an easily understandable, customizable implementation for multi-agent reinforcement learning. It supports training in a competitive 2 pursuer – 1 evader environment with the possibility to extend to any number of agents.
- Key papers, references, and training/prediction environments are mentioned in the README.

##### Analyzing the File Structure
- Top-Level Files:
▸ config.py – Contains training hyperparameters (e.g., STD_DEV, GAMMA, TAU).
▸ train.py – Main training script to execute the training loop.
▸ README.md and LICENSE – Project documentation and licensing.
▸ requirements.txt – Lists project dependencies.
- Directories:
▸ maddpg – Holds core algorithm components:
- buffer.py: Manages the experience replay and gradient calculation and model updates.
- model.py: Builds the neural network architectures for actor and critic.
- noise.py: Generates random noise for exploration.
▸ env – Contains code for simulating the environment:
- env.py: Defines the training environment.
- env_predict.py: Defines the prediction/testing environment.
▸ saved_models – Holds pre-trained models for actors and critics (and corresponding target networks) for quick evaluation.

##### Key Components
a. Environment Module:
- Training Environment (from env/env.py) – defines state, action spaces, step logic, and reward computation.
- Prediction Environment (from env/env_predict.py) – similar, but tailored for testing/prediction.
b. MADDPG Algorithm Module:
- Buffer (maddpg/buffer.py) – Innovation area where experiences are stored and gradient updates occur.
- Model (maddpg/model.py) – Constructs the neural networks for actor and critic.
- Noise (maddpg/noise.py) – Adds exploration behavior to the actions.
c. Main Execution Flow:
- Training Module (train.py) – Orchestrates interaction between the Environment and MADDPG modules.
- Prediction Module (predict.py) – Uses saved_models to test the trained networks.
d. Supporting Components:
- config.py – Defines hyperparameters and configurations.
- saved_models – Storage component for pre-trained models, reflecting the cyclical nature of training, evaluation, and possibly periodic saving of new models.

##### Relationships and Interactions
- The Training Module (train.py) orchestrates the overall process by using:
– The Environment to simulate interactions and generate states/rewards.
– The MADDPG modules (buffer, model, noise) to compute actions, update the models, and learn from past experiences.
- Configurations set in config.py influence both training and prediction processes.

##### Architectural Patterns
- Modular Design: Components (environment, algorithm, configuration, persistent storage) are clearly separated.
- Pipeline Flow: Data flows from the environment to the training algorithm (experience buffer and gradient update) and subsequently to the saved model checkpointing, then back into the prediction/testing environment.
- Extensibility: The design supports easy customizations – for example, by creating a new env.py file to simulate different environments.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration and Persistence
    Config["Configuration (config.py)"]:::config
    Models["Pre-trained Models"]:::storage

    %% Environment Modules
    subgraph "Environment Layer"
        TE["Training Environment (env.py)"]:::environment
        PE["Prediction Environment (env_predict.py)"]:::environment
    end

    %% MADDPG Core Modules
    subgraph "MADDPG Core"
        Buffer["Buffer"]:::core
        Model["Model"]:::core
        Noise["Noise"]:::core
    end

    %% Main Execution Flow
    subgraph "Execution Flow"
        TO["Training Orchestrator (train.py)"]:::execution
        PEcu["Prediction Executor (predict.py)"]:::execution
    end

    %% Data Flow Relationships
    TE -->|"Generates experience"| TO
    TO -->|"Uses"| Buffer
    TO -->|"Uses"| Model
    TO -->|"Uses"| Noise
    Buffer -- "Processes experience" --> Model
    Noise -- "Adds exploration" --> TO
    Model -->|"Saves updated models"| Models
    Config -->|"Configures"| TO
    Config -->|"Configures"| PEcu
    Models -->|"Provides trained networks"| PEcu
    PEcu -->|"Evaluates network"| PE

    %% Click Events
    click TE "https://github.com/pr-shukla/maddpg-keras/blob/main/env/env.py"
    click PE "https://github.com/pr-shukla/maddpg-keras/blob/main/env/env_predict.py"
    click Buffer "https://github.com/pr-shukla/maddpg-keras/blob/main/maddpg/buffer.py"
    click Model "https://github.com/pr-shukla/maddpg-keras/blob/main/maddpg/model.py"
    click Noise "https://github.com/pr-shukla/maddpg-keras/blob/main/maddpg/noise.py"
    click TO "https://github.com/pr-shukla/maddpg-keras/blob/main/train.py"
    click PEcu "https://github.com/pr-shukla/maddpg-keras/blob/main/predict.py"
    click Config "https://github.com/pr-shukla/maddpg-keras/blob/main/config.py"
    click Models "https://github.com/pr-shukla/maddpg-keras/tree/main/saved_models"

    %% Styles
    classDef environment fill:#AEDFF7,stroke:#333,stroke-width:2px;
    classDef core fill:#C8FACC,stroke:#333,stroke-width:2px;
    classDef execution fill:#FFF2B2,stroke:#333,stroke-width:2px;
    classDef config fill:#F7D6AE,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
    classDef storage fill:#F8C8DC,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
```
