# evansnyanney/rl4se architecture

> Architecture diagram of the GitHub repository evansnyanney/rl4se, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/evansnyanney/rl4se
- Repository: https://github.com/evansnyanney/rl4se

## Overview

##### Project type and purpose
- This is a Reinforcement-Learning training pipeline (“RL4SE”) that applies the Proximal Policy Optimization (PPO) algorithm (via Stable-Baselines3) to the OpenAI Gymnasium LunarLander-v3 environment.
- It is not a web‐app or microservices suite, but a single Python-based ML pipeline whose goal is to train, track, and save an RL agent.

##### File structure
- Top level
- CS4900GameAI/evansnyanney/
– train_ppo.py: main training orchestration script
– utils.py: helper routines (e.g., video recording, model loading/saving)
– requirements.txt: Python dependencies (Gymnasium, Stable-Baselines3, WandB, etc.)
- performance_metrics/: holds charts and images produced post-training
- README.md: high-level project overview
- Git LFS configured for large video files
- No separate “frontend” or “backend” folders—it's a single-process ML workflow.

##### README for additional insights
- Uses WandB for experiment tracking and visualization
- Records videos of the agent landing and stores them via Git LFS
- Modular structure for future enhancements (clear separation of training logic vs. utility code)

a. Main components
1. CLI / Entry Point (train_ppo.py)
2. Configuration Loader (hyperparameters, WandB project settings)
3. Gymnasium Environment (LunarLander-v3)
4. Stable-Baselines3 PPO Agent
5. Utils Module (video recorder, model‐save/load routines)
6. WandB Service (external)
7. Local File Storage
- Model Artifacts (.zip/.pkl)
- Videos (.mp4 managed via Git LFS)
- performance_metrics/ images
b. Relationships and data flow
- train_ppo.py reads config → initializes WandB → creates Gym environment → injects env into PPO Agent
- At each training step:
– PPO Agent samples actions from policy, steps the environment → receives observations/rewards
– Metrics (reward, loss, entropy, etc.) are streamed to WandB
– Optionally, utils module captures video frames and writes to local disk
- On checkpoint or completion:
– Model artifacts saved locally
– WandB final snapshot saved remotely
– Local performance_metrics/ populated after evaluation scripts run
c. Architectural patterns & principles
- Modular layering: separation between “Orchestrator” (train_ppo.py), “Core Algorithm” (Stable-Baselines3), “Environment” (Gym), and “Utilities” (utils.py)
- Externalization of experimentation concerns: logging (WandB) and media storage (Git LFS) are decoupled from core training logic
- Single-process pipeline with clear I/O boundaries
d. Key technologies & libraries
- Python 3.7+
- Gymnasium (LunarLander-v3)
- Stable-Baselines3 (PPO implementation)
- Weights & Biases (WandB) for metrics & code snapshots
- Git LFS for large video files

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Documentation and Config
    Doc["README.md"]:::doc
    Req["requirements.txt"]:::config
    GitLFS[".gitattributes"]:::configFile

    %% Core Modules
    subgraph "Core Pipeline"
        direction TB
        CLI["train_ppo.py"]:::orchestrator
        Utils["utils.py"]:::utilsModule
        Env["Gym Env\n(LunarLander-v3)"]:::environment
        Agent["PPO Agent\n(Stable-Baselines3)"]:::agentModule
    end

    %% External Services and Storage
    WandB["WandB Service"]:::externalService
    LocalStorage["Local Storage\n(models, videos)"]:::storage
    Performance["performance_metrics/"]:::storage

    %% Data Flows
    Doc --> Req
    Req -->|"loads config"| CLI
    CLI -->|"imports utilities"| Utils
    CLI -->|"initializes WandB"| WandB
    CLI -->|"creates environment"| Env
    CLI -->|"initializes agent"| Agent

    Agent -->|"actions"| Env
    Env -->|"observations\n& rewards"| Agent
    Agent -->|"logs metrics"| WandB

    Utils -->|"records video (.mp4)"| LocalStorage
    CLI -->|"saves models\n(.zip/.pkl)"| LocalStorage
    LocalStorage -->|"managed via Git LFS"| GitLFS

    CLI -->|"post-evaluation"| Performance

    %% Click Events
    click CLI "https://github.com/evansnyanney/rl4se/blob/main/CS4900GameAI/evansnyanney/train_ppo.py"
    click Utils "https://github.com/evansnyanney/rl4se/blob/main/CS4900GameAI/evansnyanney/utils.py"
    click Req "https://github.com/evansnyanney/rl4se/blob/main/CS4900GameAI/evansnyanney/requirements.txt"
    click Performance "https://github.com/evansnyanney/rl4se/tree/main/performance_metrics/"
    click GitLFS "https://github.com/evansnyanney/rl4se/blob/main/.gitattributes"
    click Doc "https://github.com/evansnyanney/rl4se/blob/main/README.md"

    %% Styles
    classDef orchestrator fill:#D0E6FF,stroke:#0366D6,color:#0366D6,stroke-width:2px;
    classDef utilsModule fill:#E6F7D4,stroke:#28A745,color:#155724,stroke-width:2px;
    classDef config fill:#FFF5BA,stroke:#FFC107,color:#856404,stroke-width:2px;
    classDef configFile fill:#FFE5EC,stroke:#FF4081,color:#C2185B,stroke-width:2px;
    classDef environment fill:#F0F4FF,stroke:#5A67D8,color:#2C5282,stroke-width:2px,shape:hexagon;
    classDef agentModule fill:#FFF0E6,stroke:#DD6B20,color:#7B341E,stroke-width:2px;
    classDef externalService fill:#E3F2FD,stroke:#1976D2,color:#0D47A1,stroke-width:2px,shape:cloud;
    classDef storage fill:#E8F5E9,stroke:#388E3C,color:#1B5E20,stroke-width:2px,shape:cylinder;
    classDef doc fill:#F3E5F5,stroke:#8E24AA,color:#4A148C,stroke-width:2px;
```
