# katerakelly/oyster architecture

> Architecture diagram of the GitHub repository katerakelly/oyster, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/katerakelly/oyster
- Repository: https://github.com/katerakelly/oyster

## Overview

Step 1: Identify the Main Components
- Experiment Launcher & Config Manager: The scripts “launch.py” and “launch_experiment.py” use configuration files in the “configs” directory. These represent the entry point into the system, triggering training runs and experiment setups.
- Environment Definitions: Inside the “rlkit/envs” folder, you see environment scripts (e.g., ant.py, half_cheetah.py, walker wrappers) and related XML assets for MuJoCo simulation. These form the simulated worlds where the RL agent learns.
- RL Algorithm & Policy Modules: The heart of the implementation is in “rlkit/core”, “rlkit/policies”, “rlkit/samplers” and “rlkit/torch/sac”, where the RL algorithms (including SAC for continuous control) and associated policies and sampling methods are implemented.
- Data Management & Logging: “rlkit/data_management” handles replay buffers, normalization, and path building. The “rlkit/core/logger.py” is used for logging both progress and experimental statistics.
- Deep Learning Infrastructure: The “rlkit/torch” directory includes modules that define networks, helper functions for pytorch, and support for GPU acceleration.
- Additional Modules:
- “rand_param_envs” module: Provides environments with randomized parameters for tasks like Walker and Hopper.
- Docker Configuration: The “docker” folder’s Dockerfile and supportive scripts ensure that experiments can run in a containerized, GPU-enabled environment.
- Visualization & Evaluation: Extra scripts like “sim_policy.py” for evaluating and visualizing learned policies and a Jupyter notebook (“sparse-point-robot-vis.ipynb”) for interactive exploration complete the picture.

Step 2: Capture Relationships and Data Flow
- The launch scripts load experiment configurations (from “configs”) that specify which simulation environment (MuJoCo-based environments in “rlkit/envs” and “rand_param_envs”) and RL algorithm (from modules in “rlkit/torch/sac” and “rlkit/core/rl_algorithm.py”) to use.
- The Agent interacts with the Environment via samplers (in “rlkit/samplers”) that collect experience.
- Experience is stored and processed using data management components (replay buffers and normalizers).
- The Policy (defined in “rlkit/policies”) is updated by the RL algorithm’s learning procedure.
- During training, logging (via “rlkit/core/logger.py”) continuously outputs metrics that can later be visualized using external tools like “viskit”.
- Docker encapsulates the entire system for deployment on GPU-supported machines, while ensuring that external dependencies (e.g., MuJoCo) are correctly handled.

Step 3: Architectural Patterns & Design Principles
- Modularity: The project uses a layered architecture. Components are neatly decomposed into configuration, environment, data management, RL algorithms, and deep learning support.
- Separation of Concerns: There is a clear boundary between experiment setup (launchers, configs), simulation (envs, rand_param_envs), learning (core, torch, policies, samplers), and evaluation (simulation/visualization scripts).
- Extensibility: The per-environment setup and modular RL algorithm code allow for plugging in alternative policies, replay buffers, or even an encoder (as noted in the TODO for an RNN encoder and optional convolutional encoder).

For this meta-RL project:
a. Create nodes/blocks that represent:
- Experiment Launcher (launch.py & launch_experiment.py)
- Configuration Manager (configs/ JSON and Python files)
- Environment Simulator (rlkit/envs, rand_param_envs, and related XML assets)
- Learning Core (rlkit/core and rlkit/torch modules, including SAC agent)
- Data Management (replay buffers, logging, and normalization)
- Policy & Sampler (rlkit/policies, rlkit/samplers)
- Visualization/Evaluation (sim_policy.py, notebook)
- Docker Container (docker folder with Dockerfile and environment.yml)
- The flow from configurations to the Experiment Launcher.
- The interaction between the RL agent (learning core, policies, and samplers) and the Environment.
- The data flow from experience collection through replay buffers to RL updates.
- Feedback loops (e.g., logging and network snapshots during training).
- A third for the learning algorithms (core, torch, policies).
d. Text labels for each block:
- Not just generic names but indicate key roles (e.g., “MuJoCo Environment Wrapper”, “SAC Agent & RL Update Logic”, “Replay Buffer – Data Management”).
e. External Dependencies:

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Experiment Setup
    ExpLauncher["Experiment Launcher (launch.py & launch_experiment.py)"]:::classExp
    ConfigManager["Configuration Manager (configs/)"]:::classExp

    %% Environment Simulation
    EnvSim["MuJoCo Environment (rlkit/envs/, rand_param_envs/)"]:::classEnv

    %% Learning Core Modules
    Sampler["Samplers (rlkit/samplers/)"]:::classRL
    RLAgent["RL Agent & Policy (rlkit/core, rlkit/torch/sac, rlkit/policies)"]:::classRL

    %% Data Management & Logging
    DataMgmt["Data Management & Logging (rlkit/data_management, rlkit/core/logger.py)"]:::classData

    %% Deep Learning Infrastructure
    DLInfra["Deep Learning Infrastructure (rlkit/torch)"]:::classDL

    %% Docker Container Setup
    Docker["Docker Container Setup (docker/)"]:::classDocker

    %% Visualization & Evaluation Tools
    VizEval["Visualization & Evaluation (sim_policy.py, sparse-point-robot-vis.ipynb)"]:::classViz

    %% High-level Flow
    ConfigManager -->|"loads config"| ExpLauncher
    ExpLauncher -->|"triggers simulation"| EnvSim
    ExpLauncher -->|"initializes RL"| RLAgent

    %% Training Loop Flow
    EnvSim -->|"collects experience"| Sampler
    Sampler -->|"stores data"| DataMgmt
    DataMgmt -->|"provides data"| RLAgent
    RLAgent -->|"sends actions"| EnvSim

    %% Deep Learning dependency
    RLAgent -->|"uses neural nets"| DLInfra
    DLInfra -->|"supports computation"| RLAgent

    %% Auxiliary Connections
    DataMgmt -->|"logs metrics"| VizEval
    RLAgent -->|"evaluates policy"| VizEval

    %% Deployment container
    Docker -->|"encapsulates system"| ExpLauncher
    Docker -->|"encapsulates system"| EnvSim
    Docker -->|"encapsulates system"| RLAgent
    Docker -->|"encapsulates system"| DataMgmt
    Docker -->|"encapsulates system"| DLInfra

    %% Click Events for Experiment Setup
    click ExpLauncher "https://github.com/katerakelly/oyster/blob/master/launch.py"
    click ExpLauncher "https://github.com/katerakelly/oyster/blob/master/launch_experiment.py"
    click ConfigManager "https://github.com/katerakelly/oyster/tree/master/configs/"

    %% Click Events for Environment Definitions
    click EnvSim "https://github.com/katerakelly/oyster/tree/master/rlkit/envs/"
    click EnvSim "https://github.com/katerakelly/oyster/tree/master/rand_param_envs/"

    %% Click Events for RL Algorithms & Policies
    click RLAgent "https://github.com/katerakelly/oyster/blob/master/rlkit/core/rl_algorithm.py"
    click RLAgent "https://github.com/katerakelly/oyster/tree/master/rlkit/policies/"
    click RLAgent "https://github.com/katerakelly/oyster/tree/master/rlkit/torch/sac/"

    %% Click Events for Samplers
    click Sampler "https://github.com/katerakelly/oyster/tree/master/rlkit/samplers/"

    %% Click Events for Data Management & Logging
    click DataMgmt "https://github.com/katerakelly/oyster/tree/master/rlkit/data_management/"
    click DataMgmt "https://github.com/katerakelly/oyster/blob/master/rlkit/core/logger.py"

    %% Click Event for Deep Learning Infrastructure
    click DLInfra "https://github.com/katerakelly/oyster/tree/master/rlkit/torch/"

    %% Click Event for Docker Container Setup
    click Docker "https://github.com/katerakelly/oyster/tree/master/docker/"

    %% Click Events for Visualization & Evaluation
    click VizEval "https://github.com/katerakelly/oyster/blob/master/sim_policy.py"
    click VizEval "https://github.com/katerakelly/oyster/blob/master/sparse-point-robot-vis.ipynb"

    %% Styles
    classDef classExp fill:#add8e6,stroke:#000,stroke-width:2px;
    classDef classEnv fill:#90ee90,stroke:#000,stroke-width:2px;
    classDef classRL fill:#ffff99,stroke:#000,stroke-width:2px;
    classDef classData fill:#ffa07a,stroke:#000,stroke-width:2px;
    classDef classDL fill:#dda0dd,stroke:#000,stroke-width:2px;
    classDef classDocker fill:#d3d3d3,stroke:#000,stroke-width:2px;
    classDef classViz fill:#ffb6c1,stroke:#000,stroke-width:2px;
```
