# park-jihoo/rl_til architecture

> Architecture diagram of the GitHub repository park-jihoo/rl_til, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/park-jihoo/rl_til
- Repository: https://github.com/park-jihoo/rl_til

## Overview

##### Project Type and Purpose
- This is a multi-course, multi-assignment repository focused on reinforcement learning. It includes implementations (e.g., Wordle RL agents, baby_robot experiments), a set of academic assignments from Stanford (ass1, ass2, ass3, and 2023 variants) and Cornell, plus a library of relevant research papers.
- There isn’t one monolithic application, but rather multiple experiments, environments, and algorithm implementations. The common goal is to demonstrate various RL methods (e.g., deep Q-learning, policy gradients, and variations used in simulated environments) applied to different tasks.

##### Analyzing the File Structure
- Top-Level Organization:
- “cornell-cs6789”: Contains resources for the Cornell course (likely including its own README) and possibly assignment instructions or code.
- “stanford-cs234”: Contains multiple assignment files and directories (ass1, ass2, ass2_2023, ass3, ass3_2023) with code, configurations, and experimental results.
- “implementation”: Holds independent projects such as “Wordle” (with RL agents implemented in dqn.py, policygradient.py, and human-based approaches) and “baby_robot” which appears to be a separate RL or simulation project.
- “papers”: Consists of markdown files for research papers on bandit algorithms and related RL topics.
- Within each course directory (especially Stanford CS234), the structure is further divided into assignments (e.g., different “ass” folders) that include:
- Configuration files (e.g., Makefiles, YAML files, configs)
- Core RL algorithm implementations (e.g., deep_q_learning, q_learning, ppo.py, policy_gradient.py)
- Utility functions and test environments
- Results directories that capture experiment data, logs, and model weights.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Courses and Assignments
    subgraph "Courses and Assignments"
        direction TB
        Cornell["Cornell CS6789"]:::course
        subgraph "Stanford CS234"
            direction TB
            A1["Assignment 1"]:::assignment
            A2["Assignment 2"]:::assignment
            A2_2023["Assignment 2 (2023)"]:::assignment
            A3["Assignment 3"]:::assignment
            A3_2023["Assignment 3 (2023)"]:::assignment
        end
    end

    %% Common RL Methods Node
    RLMethods["Common RL Approaches"]:::rl

    %% Implementations
    subgraph "Implementations"
        direction TB
        Wordle["Wordle RL Implementation"]:::implementation
        subgraph "Wordle Details"
            direction TB
            WordleSrc["RL Algorithms & Environment Interface"]:::implementation
            WordleGym["Wordle Gym Environment"]:::environment
        end
        BabyRobot["Baby_robot Project"]:::implementation
    end

    %% Research Papers
    Research["Research Papers"]:::research

    %% Connections between components
    Cornell -->|"influences"| RLMethods
    A1 -->|"influences"| RLMethods
    A2 -->|"influences"| RLMethods
    A2_2023 -->|"influences"| RLMethods
    A3 -->|"influences"| RLMethods
    A3_2023 -->|"influences"| RLMethods

    RLMethods -->|"implements"| Wordle
    RLMethods -->|"implements"| BabyRobot
    Research -->|"supports"| RLMethods

    Wordle -->|"utilizes"| WordleSrc
    WordleSrc -->|"interfaces"| WordleGym
    WordleGym -->|"feeds"| WordleSrc

    %% Click Events
    click Cornell "https://github.com/park-jihoo/rl_til/tree/main/cornell-cs6789/"
    click A1 "https://github.com/park-jihoo/rl_til/tree/main/stanford-cs234/assignments/ass1/"
    click A2 "https://github.com/park-jihoo/rl_til/tree/main/stanford-cs234/assignments/ass2/"
    click A2_2023 "https://github.com/park-jihoo/rl_til/tree/main/stanford-cs234/assignments/ass2_2023/"
    click A3 "https://github.com/park-jihoo/rl_til/tree/main/stanford-cs234/assignments/ass3/"
    click A3_2023 "https://github.com/park-jihoo/rl_til/tree/main/stanford-cs234/assignments/ass3_2023/"
    click Wordle "https://github.com/park-jihoo/rl_til/tree/main/implementation/Wordle/"
    click WordleSrc "https://github.com/park-jihoo/rl_til/tree/main/implementation/Wordle/src/"
    click WordleGym "https://github.com/park-jihoo/rl_til/tree/main/implementation/Wordle/src/wordle_gym/"
    click BabyRobot "https://github.com/park-jihoo/rl_til/tree/main/implementation/baby_robot/"
    click Research "https://github.com/park-jihoo/rl_til/tree/main/papers/"

    %% Styles
    classDef course fill:#FFC0CB,stroke:#333,stroke-width:2px;
    classDef assignment fill:#FFE4B5,stroke:#333,stroke-width:2px;
    classDef implementation fill:#90EE90,stroke:#333,stroke-width:2px;
    classDef environment fill:#ADD8E6,stroke:#333,stroke-width:2px;
    classDef research fill:#FFA07A,stroke:#333,stroke-width:2px;
    classDef rl fill:#D3D3D3,stroke:#333,stroke-width:2px;
```
