# nicolassalat/dspro2 architecture

> Architecture diagram of the GitHub repository nicolassalat/dspro2, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nicolassalat/dspro2
- Repository: https://github.com/nicolassalat/dspro2

## Overview

This repository is a Mario Kart Wii image-based reinforcement-learning agent. Its main workflow captures gameplay frames, selects driving actions with a distributional Q-network, and trains from replayed experience; a separate greedy evaluation loop reports rewards and finish rate. The graph also preserves the emulator-side state/action interface and reward shaping. Training and evaluation source excerpts show their dependencies, but the unsampled Dolphin environment implementation limits claims about its internal runtime wiring.

Main entry points: [`btr/networks/btr.py`](https://github.com/nicolassalat/dspro2/blob/main/btr/networks/btr.py) (BTR Network), [`scripts/eval/evaluate.py`](https://github.com/nicolassalat/dspro2/blob/main/scripts/eval/evaluate.py) (Evaluation Loop), [`scripts/train/training_process.py`](https://github.com/nicolassalat/dspro2/blob/main/scripts/train/training_process.py) (Training Process), [`scripts/train/neural_agent.py`](https://github.com/nicolassalat/dspro2/blob/main/scripts/train/neural_agent.py) (Neural Agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_environment["Emulator Environment"]
  node_dolphin_env["Dolphin Environment<br/>[dolphin_env.py]"]
  node_memory["Game Memory<br/>[game_memory.py]"]
  node_actions["Action Space<br/>[actions.py]"]
  node_capture["Frame Capture<br/>[dolphin_capture.py]"]
  node_reward["Reward Shaping<br/>[reward_function.py]"]
end

subgraph group_agent["Agent Training"]
  node_training["Training Process"]
  node_neural_agent["Neural Agent<br/>[neural_agent.py]"]
  node_replay[("Replay Buffer<br/>[neural_agent.py]")]
end

subgraph group_network["Q Network"]
  node_btr["BTR Network<br/>[btr.py]"]
  node_encoder["IMPALA Encoder<br/>[impala_encoder.py]"]
  node_base_encoder["Encoder Base<br/>[base_encoder.py]"]
  node_iqn["IQN Head<br/>[iqn.py]"]
  node_value_stream["Value Streams<br/>[value_stream.py]"]
  node_noisy["Noisy Layers<br/>[noisy_linear.py]"]
end

subgraph group_evaluation["Evaluation"]
  node_eval_launcher["Evaluation Launcher<br/>[start_eval.py]"]
  node_evaluate["Evaluation Loop<br/>[evaluate.py]"]
end

node_user(("Operator"))
node_dolphin["Dolphin Emulator"]

node_user -->|"starts training"| node_training
node_training -->|"uses frames"| node_capture
node_training -->|"computes reward"| node_reward
node_training -->|"trains agent"| node_neural_agent
node_neural_agent -->|"stores transitions"| node_replay
node_neural_agent -->|"runs policy"| node_btr
node_btr -->|"encodes frames"| node_encoder
node_encoder -->|"extends"| node_base_encoder
node_btr -->|"embeds quantiles"| node_iqn
node_btr -->|"combines streams"| node_value_stream
node_value_stream -->|"uses layers"| node_noisy
node_btr -->|"resets noise"| node_noisy
node_user -->|"starts evaluation"| node_eval_launcher
node_eval_launcher -->|"launches loop"| node_evaluate
node_evaluate -->|"captures frames"| node_capture
node_evaluate -->|"scores progress"| node_reward
node_evaluate -->|"selects actions"| node_neural_agent
node_evaluate -->|"accepts emulator links"| node_dolphin
node_dolphin -.->|"hosts environment"| node_dolphin_env
node_dolphin_env -->|"reads state"| node_memory
node_dolphin_env -->|"maps controls"| node_actions

click node_dolphin_env "https://github.com/nicolassalat/dspro2/blob/main/scripts/env/dolphin_env.py"
click node_memory "https://github.com/nicolassalat/dspro2/blob/main/scripts/env/game_memory.py"
click node_actions "https://github.com/nicolassalat/dspro2/blob/main/scripts/env/actions.py"
click node_capture "https://github.com/nicolassalat/dspro2/blob/main/scripts/env/dolphin_capture.py"
click node_reward "https://github.com/nicolassalat/dspro2/blob/main/scripts/env/reward_function.py"
click node_training "https://github.com/nicolassalat/dspro2/blob/main/scripts/train/training_process.py"
click node_neural_agent "https://github.com/nicolassalat/dspro2/blob/main/scripts/train/neural_agent.py"
click node_replay "https://github.com/nicolassalat/dspro2/blob/main/scripts/train/neural_agent.py"
click node_btr "https://github.com/nicolassalat/dspro2/blob/main/btr/networks/btr.py"
click node_encoder "https://github.com/nicolassalat/dspro2/blob/main/btr/encoders/impala_encoder.py"
click node_base_encoder "https://github.com/nicolassalat/dspro2/blob/main/btr/encoders/base_encoder.py"
click node_iqn "https://github.com/nicolassalat/dspro2/blob/main/btr/networks/iqn.py"
click node_value_stream "https://github.com/nicolassalat/dspro2/blob/main/btr/networks/value_stream.py"
click node_noisy "https://github.com/nicolassalat/dspro2/blob/main/btr/layers/noisy_linear.py"
click node_eval_launcher "https://github.com/nicolassalat/dspro2/blob/main/scripts/eval/start_eval.py"
click node_evaluate "https://github.com/nicolassalat/dspro2/blob/main/scripts/eval/evaluate.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dolphin_env,node_memory,node_actions,node_capture,node_reward toneBlue
class node_training,node_neural_agent,node_replay toneAmber
class node_btr,node_encoder,node_base_encoder,node_iqn,node_value_stream,node_noisy toneMint
class node_eval_launcher,node_evaluate toneRose
class node_user,node_dolphin toneIndigo
```

## Components

### Emulator Environment

- [Dolphin Environment](https://github.com/nicolassalat/dspro2/blob/main/scripts/env/dolphin_env.py): `scripts/env/dolphin_env.py`
- [Game Memory](https://github.com/nicolassalat/dspro2/blob/main/scripts/env/game_memory.py): `scripts/env/game_memory.py`
- [Action Space](https://github.com/nicolassalat/dspro2/blob/main/scripts/env/actions.py): `scripts/env/actions.py`
- [Frame Capture](https://github.com/nicolassalat/dspro2/blob/main/scripts/env/dolphin_capture.py): `scripts/env/dolphin_capture.py`
- [Reward Shaping](https://github.com/nicolassalat/dspro2/blob/main/scripts/env/reward_function.py): `scripts/env/reward_function.py`

### Agent Training

- [Training Process](https://github.com/nicolassalat/dspro2/blob/main/scripts/train/training_process.py): `scripts/train/training_process.py`
- [Neural Agent](https://github.com/nicolassalat/dspro2/blob/main/scripts/train/neural_agent.py): `scripts/train/neural_agent.py`
- [Replay Buffer](https://github.com/nicolassalat/dspro2/blob/main/scripts/train/neural_agent.py): `scripts/train/neural_agent.py`

### Q Network

- [BTR Network](https://github.com/nicolassalat/dspro2/blob/main/btr/networks/btr.py): `btr/networks/btr.py`
- [IMPALA Encoder](https://github.com/nicolassalat/dspro2/blob/main/btr/encoders/impala_encoder.py): `btr/encoders/impala_encoder.py`
- [Encoder Base](https://github.com/nicolassalat/dspro2/blob/main/btr/encoders/base_encoder.py): `btr/encoders/base_encoder.py`
- [IQN Head](https://github.com/nicolassalat/dspro2/blob/main/btr/networks/iqn.py): `btr/networks/iqn.py`
- [Value Streams](https://github.com/nicolassalat/dspro2/blob/main/btr/networks/value_stream.py): `btr/networks/value_stream.py`
- [Noisy Layers](https://github.com/nicolassalat/dspro2/blob/main/btr/layers/noisy_linear.py): `btr/layers/noisy_linear.py`

### Evaluation

- [Evaluation Launcher](https://github.com/nicolassalat/dspro2/blob/main/scripts/eval/start_eval.py): `scripts/eval/start_eval.py`
- [Evaluation Loop](https://github.com/nicolassalat/dspro2/blob/main/scripts/eval/evaluate.py): `scripts/eval/evaluate.py`

### Other components

- **Operator**
- **Dolphin Emulator**

## Connections

- Operator → Training Process: starts training (evidence: `README.md`)
- Training Process → Frame Capture: uses frames (evidence: `scripts/train/training_process.py`)
- Training Process → Reward Shaping: computes reward (evidence: `scripts/train/training_process.py`)
- Training Process → Neural Agent: trains agent (evidence: `scripts/train/training_process.py`)
- Neural Agent → Replay Buffer: stores transitions (evidence: `scripts/train/neural_agent.py`)
- Neural Agent → BTR Network: runs policy (evidence: `scripts/train/neural_agent.py`)
- BTR Network → IMPALA Encoder: encodes frames (evidence: `btr/networks/btr.py`)
- IMPALA Encoder → Encoder Base: extends (evidence: `btr/encoders/impala_encoder.py`)
- BTR Network → IQN Head: embeds quantiles (evidence: `btr/networks/btr.py`)
- BTR Network → Value Streams: combines streams (evidence: `btr/networks/btr.py`)
- Value Streams → Noisy Layers: uses layers (evidence: `btr/networks/value_stream.py`)
- BTR Network → Noisy Layers: resets noise (evidence: `btr/networks/btr.py`)
- Operator → Evaluation Launcher: starts evaluation (evidence: `README.md`)
- Evaluation Launcher → Evaluation Loop: launches loop (evidence: `README.md`)
- Evaluation Loop → Frame Capture: captures frames (evidence: `scripts/eval/evaluate.py`)
- Evaluation Loop → Reward Shaping: scores progress (evidence: `scripts/eval/evaluate.py`)
- Evaluation Loop → Neural Agent: selects actions (evidence: `scripts/eval/evaluate.py`)
- Evaluation Loop → Dolphin Emulator: accepts emulator links (evidence: `scripts/eval/evaluate.py`)
- Dolphin Emulator → Dolphin Environment: hosts environment (evidence: `README.md`)
- Dolphin Environment → Game Memory: reads state (evidence: `scripts/env/dolphin_env.py`)
- Dolphin Environment → Action Space: maps controls (evidence: `scripts/env/dolphin_env.py`)
