# urbanintelligence/wpi-ds551-fall26 architecture

> Architecture diagram of the GitHub repository urbanintelligence/wpi-ds551-fall26, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/urbanintelligence/wpi-ds551-fall26
- Repository: https://github.com/urbanintelligence/wpi-ds551-fall26

## Overview

This repository contains reinforcement-learning course projects: tabular dynamic programming, Monte Carlo and temporal-difference learning, and a deep-Q-learning Atari project. The principal workflow is to run or call an algorithm against an environment and obtain a policy, value estimate, or learned agent. The sampled algorithm bodies are largely assignment stubs, so the graph shows documented interfaces and visible orchestration rather than implying completed learning behavior. Project 3 source supports its setup and evaluation wiring; some agent internals and other project READMEs were not sampled.

Main entry points: [`Project3/main.py`](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/main.py) (Run orchestration), [`Project3/agent_dqn.py`](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/agent_dqn.py) (DQN agent), [`Project1/mdp_dp.py`](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project1/mdp_dp.py) (DP algorithms), [`Project3/test.py`](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/test.py) (Episode evaluation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_dp["Dynamic Programming"]
  node_dp_api["DP algorithms<br/>[mdp_dp.py]"]
  node_dp_env[("Gymnasium MDP")]
end

subgraph group_tabular["Tabular Learning"]
  node_mc_api["Monte Carlo<br/>[mc.py]"]
  node_blackjack[("Blackjack env")]
  node_td_api["TD control<br/>[td.py]"]
  node_cliff[("Cliffwalking env")]
end

subgraph group_dqn["Atari DQN"]
  node_main["Run orchestration<br/>[main.py]"]
  node_environment["Environment adapter<br/>[environment.py]"]
  node_wrappers["Atari preprocessing<br/>[atari_wrapper.py]"]
  node_gym[("Gymnasium Atari")]
  node_agent["DQN agent<br/>[agent_dqn.py]"]
  node_base_agent["Agent interface<br/>[agent.py]"]
  node_network["Q network<br/>[dqn_model.py]"]
  node_evaluation["Episode evaluation<br/>[test.py]"]
end

node_learner(("Course user"))
node_cli(("CLI user"))

node_learner -->|"calls algorithms"| node_dp_api
node_dp_api -->|"reads dynamics"| node_dp_env
node_dp_api -->|"runs episodes"| node_dp_env
node_learner -->|"calls algorithms"| node_mc_api
node_mc_api -->|"samples episodes"| node_blackjack
node_learner -->|"calls algorithms"| node_td_api
node_td_api -->|"samples episodes"| node_cliff
node_cli -->|"starts run"| node_main
node_main -->|"creates"| node_environment
node_environment -->|"builds Atari env"| node_wrappers
node_wrappers -->|"wraps and steps"| node_gym
node_main -->|"creates"| node_agent
node_agent -->|"inherits"| node_base_agent
node_agent -.->|"imports model"| node_network
node_main -->|"evaluates agent"| node_evaluation
node_evaluation -->|"requests actions"| node_agent
node_evaluation -->|"runs episodes"| node_environment

click node_dp_api "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project1/mdp_dp.py"
click node_mc_api "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project2/Project2-1/mc.py"
click node_td_api "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project2/Project2-2/td.py"
click node_main "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/main.py"
click node_environment "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/environment.py"
click node_wrappers "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/atari_wrapper.py"
click node_agent "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/agent_dqn.py"
click node_base_agent "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/agent.py"
click node_network "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/dqn_model.py"
click node_evaluation "https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/test.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dp_api,node_dp_env,node_learner,node_cli toneBlue
class node_mc_api,node_blackjack,node_td_api,node_cliff toneAmber
class node_main,node_environment,node_wrappers,node_gym,node_agent,node_base_agent,node_network,node_evaluation toneMint
```

## Components

### Dynamic Programming

- [DP algorithms](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project1/mdp_dp.py): `Project1/mdp_dp.py`
- **Gymnasium MDP**

### Tabular Learning

- [Monte Carlo](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project2/Project2-1/mc.py): `Project2/Project2-1/mc.py`
- **Blackjack env**
- [TD control](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project2/Project2-2/td.py): `Project2/Project2-2/td.py`
- **Cliffwalking env**

### Atari DQN

- [Run orchestration](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/main.py): `Project3/main.py`
- [Environment adapter](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/environment.py): `Project3/environment.py`
- [Atari preprocessing](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/atari_wrapper.py): `Project3/atari_wrapper.py`
- **Gymnasium Atari**
- [DQN agent](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/agent_dqn.py): `Project3/agent_dqn.py`
- [Agent interface](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/agent.py): `Project3/agent.py`
- [Q network](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/dqn_model.py): `Project3/dqn_model.py`
- [Episode evaluation](https://github.com/urbanintelligence/wpi-ds551-fall26/blob/main/Project3/test.py): `Project3/test.py`

### Other components

- **Course user**
- **CLI user**

## Connections

- Course user → DP algorithms: calls algorithms
- DP algorithms → Gymnasium MDP: reads dynamics
- DP algorithms → Gymnasium MDP: runs episodes
- Course user → Monte Carlo: calls algorithms
- Monte Carlo → Blackjack env: samples episodes
- Course user → TD control: calls algorithms
- TD control → Cliffwalking env: samples episodes
- CLI user → Run orchestration: starts run
- Run orchestration → Environment adapter: creates
- Environment adapter → Atari preprocessing: builds Atari env
- Atari preprocessing → Gymnasium Atari: wraps and steps
- Run orchestration → DQN agent: creates
- DQN agent → Agent interface: inherits
- DQN agent → Q network: imports model
- Run orchestration → Episode evaluation: evaluates agent
- Episode evaluation → DQN agent: requests actions
- Episode evaluation → Environment adapter: runs episodes