# drcl-usc/ogmp architecture

> Architecture diagram of the GitHub repository drcl-usc/ogmp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/drcl-usc/ogmp
- Repository: https://github.com/drcl-usc/ogmp

## Overview

##### Project Type and Purpose
- This is a research‐oriented reinforcement‐learning (RL) codebase for “Oracle Guided Multimodal Policies” (OGMP) used in agile robot control tasks (parkour, dive, etc.).
- It implements custom PPO algorithms (algos/), neural networks (nn/), environment interfaces (dtsd/), oracle modules (oracles/), training/test entry points (train.py, test.py), experiment configurations (exp_confs/), analysis scripts (analysis/), and logging infrastructure (logs/).

##### High-Level Components
A. Configuration Layer
- exp_confs/*.yaml – experiment parameters (environment, horizon, hyperparameters)
- dtsd/confs/*.yaml – environment‐specific parameters (robot model, terrain)
B. Core RL Pipeline
- train.py
– Reads YAML configs → Instantiates Environment (dtsd/envs & mujoco) → Builds Oracle (oracles/) and Policy/Critic (nn/) → Uses PPO implementation (algos/) to collect rollouts, update networks → Writes model checkpoints, TensorBoard logs (logs/)
- test.py
– Loads best model + config → Runs episodes in render mode → Saves videos/metrics
C. Environment Abstraction
- dtsd/envs/src/*.py – Gym‐style wrappers: action, observation, reward, logging, frame recording
- dtsd/envs/sim/*.py – Task definitions (dive.py, parkour.py), simulation base classes (mujoco_sim_base.py)
- dtsd/envs/rsc/models/* – Robot Mujoco XML/YAML & mesh assets
D. Neural Networks & Oracle Modules
- nn/ – Policy (actor.py), Value (critic.py), AE/VAE encoders (lstm_ae.py, lstm_vae.py), training utilities (fit.py)
- oracles/ – Interface for ground‐truth oracles (e.g., LQR, Li‐oracle), optimisers, utility functions
E. Algorithm Implementation
- algos/ – Custom PPO variants (ppo.py, extract.py) adapted for recurrent policies
F. Analysis & Results
- analysis/ – Scripts to compute metrics (rollout statistics, LMSR tests) using trained policies
- logs/ – Structured folders per experiment storing actor.pt, critic.pt, TensorBoard event files, param lists, encoders, reconstructions
- media/ – Visual artifacts (GIFs, images) for demonstration

##### Relationships & Data Flow
1. User invokes train.py/test.py with `--exp_conf_path`.
2. Script loads experiment config (exp_confs/) and environment config (dtsd/confs/).
3. Environment factory in dtsd/envs constructs a Mujoco or Gym‐like environment.
4. Oracle module is selected per config to provide privileged data or guidance.
5. Neural network models (nn/) are instantiated (actor, critic, optional autoencoder).
6. PPO algorithm (algos/) orchestrates:
– Interact with env → collect rollouts → optionally consult oracle → compute advantages → update nn weights
7. After each update, models are checkpointed in logs/, and metrics are published to TensorBoard.
8. Once training concludes, test.py can load checkpoints and run evaluation episodes (rendered).
9. analysis/ scripts read policies and logs to produce final plots in results/.

##### Architectural Patterns & Principles
- Modular separation:
- Configuration vs. code
- Environment abstraction (dtsd/) vs. algorithm (algos/) vs. model (nn/) vs. oracle (oracles/)
- Pipeline pattern: config → environment → data collection → model update → logging → analysis
- Dependency injection: experiment configs drive which env, oracle, and network variants are used
- Reusability:
- analysis/ reuses trained policies and env wrappers
- oracles/ can be swapped in with minimal code change

##### Detailed Capture of Essential Elements
- Do not collapse modules into a single “RL code” block. Show each folder as its own component.
- Where oracles intervene in that loop.
- Where multiprocessing is used in analysis/ (forked policy copies).

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph "Configuration Layer"
        CFG1["Experiment Configs (exp_confs/*.yaml)"]:::config
        CFG2["Environment Configs (dtsd/confs/*.yaml)"]:::config
    end

    subgraph "Scripts"
        TRAIN["train.py"]:::code
        TEST["test.py"]:::code
        ANALYSIS["analysis/"]:::code
    end

    subgraph "Environment Layer"
        ENVSRC["Gym-style Wrappers (dtsd/envs/src/)"]:::code
        ENVSIM["Simulation Tasks & Base (dtsd/envs/sim/)"]:::code
        RSC["Robot Models & Meshes (dtsd/envs/rsc/models/)"]:::data
    end

    subgraph "Oracle Layer"
        ORACLE["oracles/"]:::code
    end

    subgraph "Model Layer"
        NN["nn/"]:::code
    end

    subgraph "Algorithm Layer"
        ALGOS["algos/"]:::code
    end

    subgraph "Logging & Media"
        LOGS["logs/"]:::data
        MEDIA["media/"]:::data
    end

    subgraph "External Dependencies"
        MUJ["Mujoco Physics Engine"]:::external
        PTTB["PyTorch & TensorBoard"]:::external
    end

    %% Flows
    CFG1 -->|loads| TRAIN
    CFG2 -->|loads| TRAIN
    TRAIN -->|instantiates| ENVSRC
    TRAIN -->|instantiates| ENVSIM
    TRAIN -->|instantiates| ORACLE
    TRAIN -->|instantiates| NN
    TRAIN -->|uses| ALGOS
    ALGOS -->|step/reset| ENVSRC
    ALGOS -->|step/reset| ENVSIM
    ENVSIM -->|requires| MUJ
    ALGOS -->|forward/backprop| NN
    ALGOS -->|checkpoints & logs| LOGS
    ALGOS -->|metrics| PTTB
    TRAIN -->|end→| LOGS

    TEST -->|loads configs| CFG1
    TEST -->|loads logs| LOGS
    TEST -->|runs env| ENVSRC
    TEST -->|runs env| ENVSIM
    TEST -->|saves| MEDIA

    ANALYSIS -->|reads| LOGS
    ANALYSIS -->|uses| NN
    ANALYSIS -->|outputs results| MEDIA

    %% Click events
    click CFG1 "https://github.com/drcl-usc/ogmp/tree/master/exp_confs/"
    click CFG2 "https://github.com/drcl-usc/ogmp/tree/master/dtsd/confs/"
    click TRAIN "https://github.com/drcl-usc/ogmp/blob/master/train.py"
    click TEST "https://github.com/drcl-usc/ogmp/blob/master/test.py"
    click ENVSRC "https://github.com/drcl-usc/ogmp/tree/master/dtsd/envs/src/"
    click ENVSIM "https://github.com/drcl-usc/ogmp/tree/master/dtsd/envs/sim/"
    click RSC "https://github.com/drcl-usc/ogmp/tree/master/dtsd/envs/rsc/models/"
    click NN "https://github.com/drcl-usc/ogmp/tree/master/nn/"
    click ORACLE "https://github.com/drcl-usc/ogmp/tree/master/oracles/"
    click ALGOS "https://github.com/drcl-usc/ogmp/tree/master/algos/"
    click ANALYSIS "https://github.com/drcl-usc/ogmp/tree/master/analysis/"
    click LOGS "https://github.com/drcl-usc/ogmp/tree/master/logs/"
    click MEDIA "https://github.com/drcl-usc/ogmp/tree/master/media/"

    %% Styling
    classDef config fill:#a8e6cf,stroke:#379683,color:#05386b;
    classDef code fill:#ffd3b6,stroke:#ff8b94,color:#333333;
    classDef data fill:#ffaaa5,stroke:#ff6f69,color:#333333;
    classDef external fill:#eeeeee,stroke:#cccccc,color:#666666;
```
