# nvlabs/drail architecture

> Architecture diagram of the GitHub repository nvlabs/drail, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvlabs/drail
- Repository: https://github.com/nvlabs/drail

## Overview

##### Main components
a. Configuration & CLI
– config.yaml & `configs/` directory (WandB sweep specs)
– `utils/wandb.sh` and other shell scripts
– Entry script: `drail/main.py`
b. Environment Layer
– `goal_prox` module (custom MuJoCo & MiniGrid envs)
– `d4rl` module (Maze2D, Bullet, Carla, Hand‐Manipulation Suite)
– `shape_env` (custom Sine/Trajectory envs)
– External Sims: MuJoCo, Bullet, CARLA
c. Expert Dataset Storage
– `./expert_datasets` (downloaded via `utils/expert_data.sh`)
d. Core Algorithm Layer
– DRAIL variants:
- Diffusion‐based discriminator: `drail/ddpm/*`
- Main DRAIL logic: `drail/drail.py`, `drail/drail_un.py`
- Policy extraction: `drail/get_policy.py`
– Baselines (via `rl-toolkit/rlf/algos/il`): GAIL, WAIL, BC, DP
– Additional RL routines (SAC in `pytorch_sac`)
e. Training & Orchestration
– `drail/main.py` invokes:
- Config loader → sets hyperparams → selects env & algorithm
- Loads expert dataset
- Constructs Diffusion model module & policy module
- Runs adversarial training loop (discriminator ↔ policy)
- Uses RL‐toolkit or PyTorch‐SAC for policy updates
f. Logging & Monitoring
– Weights & Biases (W&B) via `utils/wandb.sh` & `logger.py`
– Local saving of model checkpoints & plots

##### Relationships and data flows
– Configs → Main script → Environment instantiation & Expert‐data loader
– Expert Data + Environment interactions → Trajectories
– Trajectories feed both:
- Diffusion discriminator (drail/ddpm) → reward function
- Policy optimizer (RL‐toolkit or SAC)
– Discriminator outputs “diffusion reward” → policy update
– Policy rollouts → new trajectories → discriminator
– Throughout, hyperparameters & progress → W&B → Web UI

##### Architectural patterns
– Modular plugin pattern: new envs (goal_prox, d4rl, shape_env) and new algorithms (drail vs. GAIL vs. WAIL) swap in via configuration
– Pipeline pattern: data (expert & rollouts) flows uni‐ and bi‐directionally between discriminator and policy optimizer
– Layered separation:
- Env & Data Layer
- Algorithmic Layer (discriminator + optimizer)
- Orchestration Layer (main, config, logging)

##### Relevant technologies
– Python 3.8+, PyTorch, mujoco-py, Gym, Gym-Bullet, CARLA
– Weights & Biases (wandb)
– Shell scripts for setup & data preprocessing

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration & CLI
    subgraph "Configuration & CLI"
        cfgY["config.yaml"]:::config
        cfgs["configs/"]:::config
        sh_wandb["utils/wandb.sh"]:::config
        sh_setup["utils/setup.sh"]:::config
        sh_expert["utils/expert_data.sh"]:::config
        main_entry["drail/main.py"]:::orchestration
    end

    %% Environment Layer
    subgraph "Environment Layer"
        gp["goal_prox/"]:::env
        d4["d4rl/"]:::env
        se["shape_env/"]:::env
        ext1["MuJoCo"]:::external
        ext2["Bullet"]:::external
        ext3["CARLA"]:::external
    end

    %% Expert Dataset Storage
    subgraph "Expert Dataset Storage"
        ds["expert_datasets/"]:::data
    end

    %% Core Algorithm Layer
    subgraph "Core Algorithm Layer"
        subgraph "Discriminator Module"
            ddpm_cond["ddpm_condition.py"]:::algo
            ddpm_cust["ddpm_customize.py"]:::algo
            policy_mod["policy_model.py"]:::algo
        end
        subgraph "Policy Module"
            drail_mod["drail.py"]:::algo
            drail_un["drail_un.py"]:::algo
            get_pol["get_policy.py"]:::algo
        end
        subgraph "Baseline IL Algorithms"
            bc["bc.py"]:::algo
            gail["gail.py"]:::algo
            wail["wail.py"]:::algo
            dp["dp.py"]:::algo
        end
        subgraph "SAC Routines"
            sac_actor["actor.py"]:::algo
            sac_critic["critic.py"]:::algo
            sac_core["sac.py"]:::algo
            sac_train["train.py"]:::algo
        end
    end

    %% Logging & Monitoring
    subgraph "Logging & Monitoring"
        wb_sh["utils/wandb.sh"]:::logging
        logger_sac["pytorch_sac/logger.py"]:::logging
        wb_logger["rl-toolkit/rlf/rl/loggers/wb_logger.py"]:::logging
        wb_cloud["W&B Service"]:::external
    end

    %% Trajectories Node
    traj["Trajectories"]:::data

    %% Connections
    cfgY --> main_entry
    cfgs --> main_entry
    sh_wandb --> main_entry
    sh_setup --> main_entry
    sh_expert --> main_entry

    main_entry --> gp
    main_entry --> d4
    main_entry --> se
    main_entry --> ds

    gp --> traj
    d4 --> traj
    se --> traj
    ds --> traj

    traj --> ddpm_cond
    traj --> policy_mod

    ddpm_cond -->|"diffusion reward"| drail_mod
    ddpm_cust -->|"diffusion reward"| drail_mod
    policy_mod -->|"diffusion reward"| drail_mod

    drail_mod --> traj
    drail_un --> traj
    get_pol --> traj

    bc --> drail_mod
    gail --> drail_mod
    wail --> drail_mod
    dp --> drail_mod

    sac_actor --> sac_core
    sac_critic --> sac_core
    sac_core --> sac_train
    sac_train --> traj

    main_entry --> wb_sh
    main_entry --> logger_sac
    main_entry --> wb_logger
    wb_sh --> wb_cloud
    logger_sac --> wb_cloud
    wb_logger --> wb_cloud

    %% Click Events
    click cfgY "https://github.com/nvlabs/drail/blob/main/config.yaml"
    click cfgs "https://github.com/nvlabs/drail/tree/main/configs/"
    click sh_wandb "https://github.com/nvlabs/drail/blob/main/utils/wandb.sh"
    click sh_setup "https://github.com/nvlabs/drail/blob/main/utils/setup.sh"
    click sh_expert "https://github.com/nvlabs/drail/blob/main/utils/expert_data.sh"
    click main_entry "https://github.com/nvlabs/drail/blob/main/drail/main.py"
    click gp "https://github.com/nvlabs/drail/tree/main/goal_prox/"
    click d4 "https://github.com/nvlabs/drail/tree/main/d4rl/"
    click se "https://github.com/nvlabs/drail/tree/main/shape_env/"
    click ds "https://github.com/nvlabs/drail/tree/main/expert_datasets/"
    click ddpm_cond "https://github.com/nvlabs/drail/blob/main/drail/ddpm/ddpm_condition.py"
    click ddpm_cust "https://github.com/nvlabs/drail/blob/main/drail/ddpm/ddpm_customize.py"
    click policy_mod "https://github.com/nvlabs/drail/blob/main/drail/ddpm/policy_model.py"
    click drail_mod "https://github.com/nvlabs/drail/blob/main/drail/drail.py"
    click drail_un "https://github.com/nvlabs/drail/blob/main/drail/drail_un.py"
    click get_pol "https://github.com/nvlabs/drail/blob/main/drail/get_policy.py"
    click bc "https://github.com/nvlabs/drail/blob/main/rl-toolkit/rlf/algos/il/bc.py"
    click gail "https://github.com/nvlabs/drail/blob/main/rl-toolkit/rlf/algos/il/gail.py"
    click wail "https://github.com/nvlabs/drail/blob/main/rl-toolkit/rlf/algos/il/wail.py"
    click dp "https://github.com/nvlabs/drail/blob/main/rl-toolkit/rlf/algos/il/dp.py"
    click sac_actor "https://github.com/nvlabs/drail/blob/main/pytorch_sac/agent/actor.py"
    click sac_critic "https://github.com/nvlabs/drail/blob/main/pytorch_sac/agent/critic.py"
    click sac_core "https://github.com/nvlabs/drail/blob/main/pytorch_sac/agent/sac.py"
    click sac_train "https://github.com/nvlabs/drail/blob/main/pytorch_sac/train.py"
    click logger_sac "https://github.com/nvlabs/drail/blob/main/pytorch_sac/logger.py"
    click wb_logger "https://github.com/nvlabs/drail/blob/main/rl-toolkit/rlf/rl/loggers/wb_logger.py"

    %% Styles
    classDef config fill:#fddc5c,stroke:#333,stroke-width:1px
    classDef env fill:#c5fddc,stroke:#333,stroke-width:1px
    classDef data fill:#fddcae,stroke:#333,stroke-width:1px
    classDef algo fill:#c5d8fd,stroke:#333,stroke-width:1px
    classDef orchestration fill:#fef3c7,stroke:#333,stroke-width:1px
    classDef logging fill:#eac5fd,stroke:#333,stroke-width:1px
    classDef external fill:#fdaac5,stroke:#333,stroke-width:1px
```
