# colingoonth/pokemon-rl architecture

> Architecture diagram of the GitHub repository colingoonth/pokemon-rl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/colingoonth/pokemon-rl
- Repository: https://github.com/colingoonth/pokemon-rl

## Overview

pokerl trains a PPO agent to earn Pokémon Red’s first gym badge, then supports checkpoint evaluation and watching/recording rollouts. Training loads YAML parameters, constructs vectorized Gym environments, gathers emulator experience, and updates an actor-critic; evaluation and watch paths run trained policies in the same environment. The README describes the project as early scaffolding, despite the training and evaluation entry points present in the sampled source. Reward versions and RAM interpretation are core domain layers; unsampled details are omitted where wiring is not evidenced.

Main entry points: [`pokerl/env/make.py`](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/make.py) (Vector Environments), [`pokerl/scripts/train.py`](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/scripts/train.py) (Train Entry), [`pokerl/agent/ppo.py`](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/agent/ppo.py) (PPO Trainer), [`pokerl/env/pokemon_red_env.py`](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/pokemon_red_env.py) (Pokémon Red Env)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Training"]
  node_train_cli["Train Entry<br/>[train.py]"]
  node_config["PPO Config<br/>[config.py]"]
  node_dist["Distributed Setup<br/>[dist.py]"]
  node_logging["Training Logs<br/>[logging.py]"]
end

subgraph group_game["Game Environment"]
  node_vector_env["Vector Environments<br/>[make.py]"]
  node_env["Pokémon Red Env<br/>[pokemon_red_env.py]"]
  node_wrappers["Frame Stacking<br/>[wrappers.py]"]
  node_reward["Reward Versions<br/>[rewards.py]"]
  node_ram["RAM Mapping<br/>[ram_map.py]"]
end

subgraph group_policy["Policy Learning"]
  node_ppo["PPO Trainer<br/>[ppo.py]"]
  node_network["Actor-Critic<br/>[networks.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_rollout["Episode Evaluation<br/>[rollout.py]"]
  node_watch["Watch / Record<br/>[watch.py]"]
end

node_user(("Researcher"))
node_pyboy["PyBoy Emulator"]
node_rom["Game ROM / State"]
node_checkpoint["Model Checkpoint"]

node_user -->|"starts training"| node_train_cli
node_train_cli -->|"loads settings"| node_config
node_train_cli -->|"initializes ranks"| node_dist
node_train_cli -->|"builds environments"| node_vector_env
node_train_cli -->|"starts trainer"| node_ppo
node_config -->|"constructs config"| node_ppo
node_vector_env -->|"constructs instances"| node_env
node_vector_env -->|"wraps observations"| node_wrappers
node_vector_env -->|"creates reward"| node_reward
node_env -->|"computes reward"| node_reward
node_reward -->|"reads game state"| node_ram
node_env -->|"steps emulator"| node_pyboy
node_env -->|"loads ROM / state"| node_rom
node_ppo -->|"collects experience"| node_vector_env
node_ppo -->|"runs policy"| node_network
node_ppo -->|"coordinates ranks"| node_dist
node_ppo -->|"records metrics"| node_logging
node_train_cli -->|"saves weights"| node_checkpoint
node_user -->|"evaluates policy"| node_rollout
node_rollout -->|"creates environment"| node_vector_env
node_rollout -->|"loads and runs"| node_network
node_rollout -->|"loads weights"| node_checkpoint
node_user -->|"watches rollout"| node_watch
node_watch -->|"creates environment"| node_vector_env
node_watch -->|"loads and runs"| node_network
node_watch -->|"loads weights"| node_checkpoint

click node_train_cli "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/scripts/train.py"
click node_config "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/infra/config.py"
click node_dist "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/infra/dist.py"
click node_vector_env "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/make.py"
click node_env "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/pokemon_red_env.py"
click node_wrappers "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/wrappers.py"
click node_reward "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/rewards.py"
click node_ram "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/ram_map.py"
click node_ppo "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/agent/ppo.py"
click node_network "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/agent/networks.py"
click node_logging "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/infra/logging.py"
click node_rollout "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/eval/rollout.py"
click node_watch "https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/eval/watch.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_cli,node_config,node_dist,node_logging toneBlue
class node_vector_env,node_env,node_wrappers,node_reward,node_ram toneAmber
class node_ppo,node_network toneMint
class node_rollout,node_watch toneRose
class node_user,node_pyboy,node_rom,node_checkpoint toneIndigo
```

## Components

### Training

- [Train Entry](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/scripts/train.py): `pokerl/scripts/train.py`
- [PPO Config](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/infra/config.py): `pokerl/infra/config.py`
- [Distributed Setup](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/infra/dist.py): `pokerl/infra/dist.py`
- [Training Logs](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/infra/logging.py): `pokerl/infra/logging.py`

### Game Environment

- [Vector Environments](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/make.py): `pokerl/env/make.py`
- [Pokémon Red Env](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/pokemon_red_env.py): `pokerl/env/pokemon_red_env.py`
- [Frame Stacking](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/wrappers.py): `pokerl/env/wrappers.py`
- [Reward Versions](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/rewards.py): `pokerl/env/rewards.py`
- [RAM Mapping](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/env/ram_map.py): `pokerl/env/ram_map.py`

### Policy Learning

- [PPO Trainer](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/agent/ppo.py): `pokerl/agent/ppo.py`
- [Actor-Critic](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/agent/networks.py): `pokerl/agent/networks.py`

### Evaluation

- [Episode Evaluation](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/eval/rollout.py): `pokerl/eval/rollout.py`
- [Watch / Record](https://github.com/colingoonth/pokemon-rl/blob/main/pokerl/eval/watch.py): `pokerl/eval/watch.py`

### Other components

- **Researcher**
- **PyBoy Emulator**
- **Game ROM / State**
- **Model Checkpoint**

## Connections

- Researcher → Train Entry: starts training (evidence: `pokerl/scripts/train.py`)
- Train Entry → PPO Config: loads settings (evidence: `pokerl/scripts/train.py`)
- Train Entry → Distributed Setup: initializes ranks (evidence: `pokerl/scripts/train.py`)
- Train Entry → Vector Environments: builds environments (evidence: `pokerl/scripts/train.py`)
- Train Entry → PPO Trainer: starts trainer (evidence: `pokerl/scripts/train.py`)
- PPO Config → PPO Trainer: constructs config (evidence: `pokerl/infra/config.py`)
- Vector Environments → Pokémon Red Env: constructs instances (evidence: `pokerl/env/make.py`)
- Vector Environments → Frame Stacking: wraps observations (evidence: `pokerl/env/make.py`)
- Vector Environments → Reward Versions: creates reward (evidence: `pokerl/env/make.py`)
- Pokémon Red Env → Reward Versions: computes reward (evidence: `pokerl/env/pokemon_red_env.py`)
- Reward Versions → RAM Mapping: reads game state (evidence: `pokerl/env/rewards.py`)
- Pokémon Red Env → PyBoy Emulator: steps emulator (evidence: `pokerl/env/pokemon_red_env.py`)
- Pokémon Red Env → Game ROM / State: loads ROM / state (evidence: `pokerl/env/pokemon_red_env.py`)
- PPO Trainer → Vector Environments: collects experience (evidence: `pokerl/agent/ppo.py`)
- PPO Trainer → Actor-Critic: runs policy (evidence: `pokerl/agent/ppo.py`)
- PPO Trainer → Distributed Setup: coordinates ranks (evidence: `pokerl/agent/ppo.py`)
- PPO Trainer → Training Logs: records metrics (evidence: `pokerl/agent/ppo.py`)
- Train Entry → Model Checkpoint: saves weights (evidence: `pokerl/scripts/train.py`)
- Researcher → Episode Evaluation: evaluates policy (evidence: `pokerl/eval/rollout.py`)
- Episode Evaluation → Vector Environments: creates environment (evidence: `pokerl/eval/rollout.py`)
- Episode Evaluation → Actor-Critic: loads and runs (evidence: `pokerl/eval/rollout.py`)
- Episode Evaluation → Model Checkpoint: loads weights (evidence: `pokerl/eval/rollout.py`)
- Researcher → Watch / Record: watches rollout (evidence: `pokerl/eval/watch.py`)
- Watch / Record → Vector Environments: creates environment (evidence: `pokerl/eval/watch.py`)
- Watch / Record → Actor-Critic: loads and runs (evidence: `pokerl/eval/watch.py`)
- Watch / Record → Model Checkpoint: loads weights (evidence: `pokerl/eval/watch.py`)
