# tinyyana/maze_agent_rl architecture

> Architecture diagram of the GitHub repository tinyyana/maze_agent_rl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tinyyana/maze_agent_rl
- Repository: https://github.com/tinyyana/maze_agent_rl

## Overview

##### Project type and purpose
- This is a research‐oriented deep reinforcement learning (RL) project combining Procedural Content Generation via RL (PCGRL).
- Purpose: Train a “Maze Master” agent (using PPO from Stable Baselines3) to dynamically modify a maze (walls, monsters, exit) so that a simulated or human player remains in a “flow” difficulty window.

##### File structure
Top‐level directories and files:
- agents/ – agent implementations
- astar_bot.py (A* player simulator)
- maze_master.py (the RL agent logic, imported by train.py)
- envs/ – Gymnasium environment definitions
- maze_env.py (core state/action/reward loop)
- maze_generator.py (initial maze creation via DFS)
- rendering.py (Pygame visualizer)
- config.py – global constants (reward weights, time limits, modes)
- train.py – script that instantiates the env & PPO model, trains & checkpoints “maze_master_ppo.zip”
- main.py – loads saved model, launches interactive demo (Pygame)
- evaluate.py – loads a model, runs evaluation episodes, writes CSV reports under evaluation_logs/
- experiment_runner.py – batch‐runs training & evaluation experiments, writes logs under experiment_log/
- models/, tensorboard_logs/ – persistent storage for model artifacts and TFEvent logs
- .gitignore, requirements.txt, LICENSE, README.md – meta/config files

##### README for additional insights
- Tech stack: Python 3.12+, Gymnasium environment, Stable Baselines3 PPO, Pygame rendering, TensorBoard for logging
- Workflow: train.py → tensorboard_logs & models; main.py for live demo; evaluate.py & experiment_runner.py for batch stats

##### Components
a. Main components
1. Configuration Module (config.py)
2. Maze Environment (envs/maze_env.py + maze_generator.py + rendering.py)
3. Player Simulators
- A* Bot (agents/astar_bot.py)
- Human Input (via main.py + Pygame)
4. Maze Master RL Agent (Stable Baselines3 PPO wrapper)
5. Training Pipeline (train.py)
6. Evaluation Pipeline (evaluate.py + experiment_runner.py)
7. Artifact Stores
- Model Store (models/)
- TensorBoard Logs (tensorboard_logs/)
- CSV and Text Reports (evaluation_logs/, experiment_log/)
b. Relationships & interactions
- train.py reads config.py, instantiates MazeEnv, wraps it in SB3 PPO, steps through episodes →
- Sends data (states, actions, rewards) to TensorBoard →
- Saves checkpoints to models/
- evaluate.py & experiment_runner.py load a checkpoint from models/, run MazeEnv + A* Bot or human mode, collect metrics → write CSV/text reports
- main.py loads checkpoint + MazeEnv + rendering.py → Pygame window for interactive demo (human or A* mode via config.py)
c. Architectural patterns & principles
- Separation of concerns: environment code (envs/) is decoupled from training (train.py), evaluation (evaluate.py), and UI (main.py)
- Modular config: single source of truth for hyperparameters and modes
- Experiment reproducibility: distinct directories for logs, models, raw & summary CSVs
d. Key technologies/frameworks
- Gymnasium environment interface
- Stable Baselines3 PPO algorithm
- Pygame for rendering
- TensorBoard for live metric visualization

- Training Pipeline → TensorBoard Logs (write metrics)
- Training Pipeline → Model Store (checkpoint model .zip)
- A* Bot (agents/astar_bot.py) feeding into Env
- CSV/Text Reports (evaluation_logs/, experiment_log/)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Configuration
    Config["Configuration Module<br/>(config.py)"]:::code

    %% Environment
    subgraph "Maze Environment" 
        direction TB
        MazeEnv["MazeEnv<br/>(envs/maze_env.py)"]:::code
        MazeGen["MazeGenerator<br/>(envs/maze_generator.py)"]:::code
        Rendering["Rendering<br/>(envs/rendering.py)"]:::code
    end

    %% Agent
    Agent["Maze Master RL Agent<br/>(agents/maze_master.py)"]:::code

    %% Training Pipeline
    Train["Training Pipeline<br/>(train.py)"]:::script

    %% Evaluation Pipeline
    subgraph "Evaluation Pipeline"
        direction TB
        EvalSingle["Single-model Eval<br/>(evaluate.py)"]:::script
        EvalBatch["Batch Experiments<br/>(experiment_runner.py)"]:::script
    end

    %% Player Simulator
    AStar["A* Bot Simulator<br/>(agents/astar_bot.py)"]:::code

    %% Main Demo
    Main["Interactive Demo<br/>(main.py)"]:::script
    Pygame["Pygame Window"]:::external

    %% Storage
    Models["Model Store<br/>(models/)"]:::storage
    TBLogs["TensorBoard Logs<br/>(tensorboard_logs/)"]:::storage
    EvalReports["Evaluation Reports<br/>(evaluation_logs/)"]:::storage
    ExpLogs["Experiment Summaries<br/>(experiment_log/)"]:::storage

    %% Flows
    Config -->|"load settings"| MazeEnv
    Config -->|"load settings"| Agent

    MazeGen -->|"initialize maze"| MazeEnv
    MazeEnv -->|"observation"| Agent
    Agent -->|"action"| MazeEnv
    MazeEnv -->|"render state"| Rendering

    Agent -->|"training loop"| Train
    Train -->|"write metrics"| TBLogs
    Train -->|"save checkpoint"| Models

    EvalSingle -->|"load model"| Models
    EvalBatch -->|"load model"| Models
    EvalSingle -->|"run episodes via A*"| MazeEnv
    EvalBatch -->|"run episodes via A*"| MazeEnv
    AStar -->|"simulate player"| MazeEnv
    EvalSingle -->|"write CSV/text"| EvalReports
    EvalBatch -->|"write experiment logs"| ExpLogs

    Main -->|"load model"| Models
    Main -->|"invoke env"| MazeEnv
    Main -->|"visualize via"| Rendering
    Rendering -->|"display"| Pygame
    Main -->|"player input|human/A*"| MazeEnv

    %% Click Events
    click Config "https://github.com/tinyyana/maze_agent_rl/blob/main/config.py"
    click MazeEnv "https://github.com/tinyyana/maze_agent_rl/blob/main/envs/maze_env.py"
    click MazeGen "https://github.com/tinyyana/maze_agent_rl/blob/main/envs/maze_generator.py"
    click Rendering "https://github.com/tinyyana/maze_agent_rl/blob/main/envs/rendering.py"
    click Agent "https://github.com/tinyyana/maze_agent_rl/blob/main/agents/maze_master.py"
    click Train "https://github.com/tinyyana/maze_agent_rl/blob/main/train.py"
    click EvalSingle "https://github.com/tinyyana/maze_agent_rl/blob/main/evaluate.py"
    click EvalBatch "https://github.com/tinyyana/maze_agent_rl/blob/main/experiment_runner.py"
    click AStar "https://github.com/tinyyana/maze_agent_rl/blob/main/agents/astar_bot.py"
    click Main "https://github.com/tinyyana/maze_agent_rl/blob/main/main.py"
    click Models "https://github.com/tinyyana/maze_agent_rl/tree/main/models/"
    click TBLogs "https://github.com/tinyyana/maze_agent_rl/tree/main/tensorboard_logs/"
    click EvalReports "https://github.com/tinyyana/maze_agent_rl/tree/main/evaluation_logs/"
    click ExpLogs "https://github.com/tinyyana/maze_agent_rl/tree/main/experiment_log/"

    %% Styles
    classDef code fill:#D0E6FF,stroke:#1B4F72,color:#1B2631
    classDef script fill:#D5F5E3,stroke:#196F3D,color:#145A32
    classDef storage fill:#FDEBD0,stroke:#BA4A00,color:#6E2C00
    classDef external fill:#F4ECF7,stroke:#6C3483,color:#4A235A,stroke-dasharray: 5 5
```
