# ulrikisdahl/muzero-breakout architecture

> Architecture diagram of the GitHub repository ulrikisdahl/muzero-breakout, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ulrikisdahl/muzero-breakout
- Repository: https://github.com/ulrikisdahl/muzero-breakout

## Overview

##### Project type and purpose
- This is a research-style reinforcement learning (RL) implementation of the MuZero algorithm applied to a vectorized Breakout environment. It is not a typical web or mobile app but a computational pipeline combining environment simulation, search (MCTS), neural networks, and training orchestration.

##### File structure
- Top-level files:
- README.md – high-level overview and how to run
- config.yaml – hyperparameter and training configuration
- train_torch.py – main training script / controller
- replay_buffer.py – experience storage and sampling
- utils.py – miscellaneous helper functions
- Directory “environment/”: custom vectorized Breakout implementation (parallel environment wrapper)
- Directory “src/”: core algorithmic modules
- mcts.py – Monte Carlo Tree Search in latent space
- networks.py – MuZero networks (representation, dynamics, prediction)
- Directory “static/”: output artifacts (GIF, plots) – useful for visualization but not part of the runtime

##### README for additional insights
- Running: “python3 train_torch.py”, logs piped to TensorBoard
- Describes deviations from MuZero paper (action mask removal, target network, replay buffer size)
- References to TensorBoard, PyTorch, vectorized environments

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Components
    TC["Training Controller<br/>(train_torch.py)"]:::compute
    CFG["config.yaml"]:::config
    UT["Configuration Loader<br/>(utils.py)"]:::config
    ENV["Environment Module<br/>(parallel_breakout.py)"]:::io
    MCTS["MCTS Module<br/>(src/mcts.py)"]:::compute
    NET["Neural Networks Module<br/>(src/networks.py)"]:::compute
    TN["Target Network"]:::compute
    RB["Replay Buffer<br/>(replay_buffer.py)"]:::datastore
    LOG["Logging & Visualization<br/>(static/)"]:::io

    %% Data Flows
    TC -->|"read hyperparams"| CFG
    CFG -->|"parsed by"| UT
    UT -->|"provides params"| TC

    TC -->|"step / reset"| ENV
    ENV -->|"batch of observations"| TC

    TC -->|"root state"| MCTS
    MCTS -->|"rep, dyn, pred calls"| NET
    NET -->|"features & predictions"| MCTS
    MCTS -->|"selected action"| TC

    TC -->|"store experience"| RB
    TC -->|"sample minibatch"| RB
    RB -->|"batch data"| NET
    NET -->|"compute losses & update"| NET
    NET -.->|"copy weights every N"| TN

    TC -->|"log metrics & images"| LOG

    %% Loop indicator
    TC --> TC

    %% Click Events
    click TC "https://github.com/ulrikisdahl/muzero-breakout/blob/main/train_torch.py"
    click CFG "https://github.com/ulrikisdahl/muzero-breakout/blob/main/config.yaml"
    click UT "https://github.com/ulrikisdahl/muzero-breakout/blob/main/utils.py"
    click ENV "https://github.com/ulrikisdahl/muzero-breakout/blob/main/environment/parallel_breakout.py"
    click MCTS "https://github.com/ulrikisdahl/muzero-breakout/blob/main/src/mcts.py"
    click NET "https://github.com/ulrikisdahl/muzero-breakout/blob/main/src/networks.py"
    click TN "https://github.com/ulrikisdahl/muzero-breakout/blob/main/src/networks.py"
    click RB "https://github.com/ulrikisdahl/muzero-breakout/blob/main/replay_buffer.py"
    click LOG "https://github.com/ulrikisdahl/muzero-breakout/tree/main/static/"

    %% Styles
    classDef compute fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef datastore fill:#90EE90,stroke:#333,stroke-width:1px
    classDef io fill:#FFA500,stroke:#333,stroke-width:1px
    classDef config fill:#D3D3D3,stroke:#333,stroke-width:1px
```
