# majdzidan-dev/reinforcement-learning-proj architecture

> Architecture diagram of the GitHub repository majdzidan-dev/reinforcement-learning-proj, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/majdzidan-dev/reinforcement-learning-proj
- Repository: https://github.com/majdzidan-dev/reinforcement-learning-proj

## Overview

RLProject trains and compares four SARSA agents on Gymnasium Taxi-v4, using default, sparse, and custom rewards across five seeds. Its principal flow is experiment orchestration, agent training, metric collection, and comparison plotting; separate workflows sweep hyperparameters and record gameplay videos. The graph preserves those documented capabilities, but internals of the metrics and sweep runners were not sampled, so their internal relationships are not expanded.

Main entry points: [`create_animation.py`](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/create_animation.py) (Gameplay videos), [`main.py`](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/main.py) (Run experiments), [`helpers.py`](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/helpers.py) (Training helpers), [`algorithms/PSSARSA.py`](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/PSSARSA.py) (Prioritized SARSA)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_orchestration["Experiment orchestration"]
  node_main["Run experiments<br/>[main.py]"]
  node_metrics["Seed metrics<br/>[metrics_test.py]"]
  node_sweeps["Parameter sweeps<br/>[sweep_test.py]"]
  node_trainer["Training helpers<br/>[helpers.py]"]
  node_metric_wrapper["Episode metrics<br/>[metrics_wrapper.py]"]
end

subgraph group_learning["SARSA agents"]
  node_sarsa["SARSA<br/>[SARSA.py]"]
  node_sarsa_gen["Generalized SARSA<br/>[SARSAGen.py]"]
  node_ps_sarsa["Prioritized SARSA<br/>[PSSARSA.py]"]
  node_ps_gen["Combined SARSA<br/>[PSSARSAGen.py]"]
end

subgraph group_environment["Taxi rewards"]
  node_taxi["Gymnasium Taxi"]
  node_sparse["Sparse rewards<br/>[Sparse.py]"]
  node_custom1["Custom rewards 1<br/>[Custom1.py]"]
  node_custom2["Custom rewards 2<br/>[Custom2.py]"]
end

subgraph group_outputs["Analysis and media"]
  node_plots["Comparison plots<br/>[draw_metrics.py]"]
  node_animation["Gameplay videos"]
end

node_user(("Engineer"))

node_user -->|"runs"| node_main
node_main -->|"runs"| node_metrics
node_main -->|"draws comparisons"| node_plots
node_main -->|"runs"| node_sweeps
node_main -->|"records gameplay"| node_animation
node_trainer -->|"wraps environment"| node_metric_wrapper
node_sarsa_gen -->|"extends"| node_sarsa
node_ps_sarsa -->|"extends"| node_sarsa
node_ps_gen -->|"extends"| node_ps_sarsa
node_animation -->|"trains with"| node_trainer
node_animation -->|"instantiates"| node_ps_gen
node_animation -->|"creates environment"| node_taxi
node_animation -->|"wraps with"| node_sparse
node_animation -->|"wraps with"| node_custom1
node_animation -->|"wraps with"| node_custom2
node_custom1 -->|"wraps"| node_taxi
node_custom2 -->|"wraps"| node_taxi

click node_main "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/main.py"
click node_metrics "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/metrics_test.py"
click node_sweeps "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/sweep_test.py"
click node_trainer "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/helpers.py"
click node_metric_wrapper "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/metrics_wrapper.py"
click node_sarsa "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/SARSA.py"
click node_sarsa_gen "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/SARSAGen.py"
click node_ps_sarsa "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/PSSARSA.py"
click node_ps_gen "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/PSSARSAGen.py"
click node_sparse "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/rewards/Sparse.py"
click node_custom1 "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/rewards/Custom1.py"
click node_custom2 "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/rewards/Custom2.py"
click node_plots "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/draw_metrics.py"
click node_animation "https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/create_animation.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_metrics,node_sweeps,node_trainer,node_metric_wrapper toneBlue
class node_sarsa,node_sarsa_gen,node_ps_sarsa,node_ps_gen toneAmber
class node_taxi,node_sparse,node_custom1,node_custom2 toneMint
class node_plots,node_animation toneRose
class node_user toneIndigo
```

## Components

### Experiment orchestration

- [Run experiments](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/main.py): `main.py`
- [Seed metrics](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/metrics_test.py): `metrics_test.py`
- [Parameter sweeps](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/sweep_test.py): `sweep_test.py`
- [Training helpers](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/helpers.py): `helpers.py`
- [Episode metrics](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/metrics_wrapper.py): `metrics_wrapper.py`

### SARSA agents

- [SARSA](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/SARSA.py): `algorithms/SARSA.py`
- [Generalized SARSA](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/SARSAGen.py): `algorithms/SARSAGen.py`
- [Prioritized SARSA](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/PSSARSA.py): `algorithms/PSSARSA.py`
- [Combined SARSA](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/algorithms/PSSARSAGen.py): `algorithms/PSSARSAGen.py`

### Taxi rewards

- **Gymnasium Taxi**
- [Sparse rewards](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/rewards/Sparse.py): `rewards/Sparse.py`
- [Custom rewards 1](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/rewards/Custom1.py): `rewards/Custom1.py`
- [Custom rewards 2](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/rewards/Custom2.py): `rewards/Custom2.py`

### Analysis and media

- [Comparison plots](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/draw_metrics.py): `draw_metrics.py`
- [Gameplay videos](https://github.com/majdzidan-dev/reinforcement-learning-proj/blob/main/create_animation.py): `create_animation.py`

### Other components

- **Engineer**

## Connections

- Engineer → Run experiments: runs (evidence: `main.py`)
- Run experiments → Seed metrics: runs (evidence: `main.py`)
- Run experiments → Comparison plots: draws comparisons (evidence: `main.py`)
- Run experiments → Parameter sweeps: runs (evidence: `main.py`)
- Run experiments → Gameplay videos: records gameplay (evidence: `main.py`)
- Training helpers → Episode metrics: wraps environment (evidence: `helpers.py`)
- Generalized SARSA → SARSA: extends (evidence: `algorithms/SARSAGen.py`)
- Prioritized SARSA → SARSA: extends (evidence: `algorithms/PSSARSA.py`)
- Combined SARSA → Prioritized SARSA: extends (evidence: `algorithms/PSSARSAGen.py`)
- Gameplay videos → Training helpers: trains with (evidence: `create_animation.py`)
- Gameplay videos → Combined SARSA: instantiates (evidence: `create_animation.py`)
- Gameplay videos → Gymnasium Taxi: creates environment (evidence: `create_animation.py`)
- Gameplay videos → Sparse rewards: wraps with (evidence: `create_animation.py`)
- Gameplay videos → Custom rewards 1: wraps with (evidence: `create_animation.py`)
- Gameplay videos → Custom rewards 2: wraps with (evidence: `create_animation.py`)
- Custom rewards 1 → Gymnasium Taxi: wraps (evidence: `rewards/Custom1.py`)
- Custom rewards 2 → Gymnasium Taxi: wraps (evidence: `rewards/Custom2.py`)
