# scutrobotlab/wheeled-legged_rl architecture

> Architecture diagram of the GitHub repository scutrobotlab/wheeled-legged_rl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/scutrobotlab/wheeled-legged_rl
- Repository: https://github.com/scutrobotlab/wheeled-legged_rl

## Overview

This repository trains and evaluates wheeled-legged robot policies in Isaac Lab, with play-time keyboard control, visualization, trajectory recording, and deployment export. The main workflow is task selection and configuration, simulated interaction, rollout collection, policy optimization, then checkpoint-based playback. Distinct task, terrain, state-machine, and algorithm branches are retained, alongside robot-world components and analysis tools. The README documents the user-facing workflow and capabilities; sampled source excerpts and the SOURCE INDEX clarify selected internal relationships, while unsampled links are omitted or marked unverified.

Main entry points: [`scripts/rsl_rl/play.py`](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/rsl_rl/play.py) (Play CLI), [`source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env.py`](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env.py) (V14 Environment), [`source/agent_rl/agent_rl/rsl_rl/runners/on_constraint_policy_runner.py`](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/runners/on_constraint_policy_runner.py) (Policy Runner), [`source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py`](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py) (V14 Task Config)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User Workflows"]
  node_train_cli["Training CLI<br/>[train.py]"]
  node_play_cli["Play CLI<br/>[play.py]"]
  node_keyboard["Keyboard Control"]
end

subgraph group_tasks["Simulation Tasks"]
  node_task_cfg["V14 Task Config<br/>[env_cfg.py]"]
  node_task_env["V14 Environment<br/>[env.py]"]
  node_base_env_cfg["Base Task Config<br/>[env_cfg.py]"]
  node_mdp_functions["MDP Functions"]
  node_terrain_cfg["Terrain Catalog<br/>[terrains.py]"]
  node_terrain_commands["Terrain Commands"]
  node_terrain_tasks["Terrain Mapping"]
  node_sm_dispatch["State Machine Stack<br/>[manager.py]"]
  node_sm_states["Movement States"]
  node_mdp_sm["State Machine Manager<br/>[manager.py]"]
end

subgraph group_rl["Policy Learning"]
  node_runner["Policy Runner"]
  node_ppo_cfg["PPO Task Config<br/>[rsl_rl_ppo_cfg.py]"]
  node_np3o["NP3O Algorithm<br/>[np3o.py]"]
  node_actor_critic["BarlowTwins Policy"]
  node_other_algorithms["PPO Variants"]
  node_rollouts[("Rollout Storage<br/>[rollout_storage.py]")]
  node_policy_export["Policy Export<br/>[exporter.py]"]
end

subgraph group_world["Robot World"]
  node_world_actuators["Robot Actuators"]
  node_heightfield["Heightfield Terrain<br/>[height_field.py]"]
end

subgraph group_analysis["Analysis Export"]
  node_trace_capture["Trace Capture"]
  node_trace_export["HTML Export"]
  node_plotter["Realtime Plotter"]
end

node_engineer(("Engineer"))

node_engineer -->|"starts training"| node_train_cli
node_engineer -->|"starts playback"| node_play_cli
node_train_cli -->|"selects task"| node_task_cfg
node_play_cli -->|"selects task"| node_task_cfg
node_play_cli -->|"enables control"| node_keyboard
node_train_cli -.->|"trains with"| node_runner
node_play_cli -.->|"runs policy"| node_runner
node_task_cfg -->|"configures"| node_task_env
node_task_cfg -->|"extends configuration"| node_base_env_cfg
node_task_cfg -->|"selects terrain"| node_terrain_cfg
node_task_env -->|"uses task terms"| node_mdp_functions
node_task_env -->|"configures state machines"| node_sm_dispatch
node_sm_dispatch -->|"selects machines"| node_sm_states
node_mdp_sm -.->|"manages behavior"| node_sm_dispatch
node_task_env -.->|"uses terrain profiles"| node_terrain_commands
node_task_env -.->|"maps terrain IDs"| node_terrain_tasks
node_terrain_cfg -->|"defines terrain"| node_heightfield
node_runner -.->|"optimizes with"| node_np3o
node_runner -.->|"runs variants"| node_other_algorithms
node_np3o -->|"uses policy"| node_actor_critic
node_np3o -->|"writes transitions"| node_rollouts
node_runner -.->|"uses algorithm config"| node_ppo_cfg
node_runner -.->|"exports policy"| node_policy_export
node_task_env -.->|"drives actuators"| node_world_actuators
node_play_cli -->|"records play data"| node_trace_capture
node_trace_capture -->|"exports HTML"| node_trace_export
node_play_cli -->|"shows live plots"| node_plotter

click node_train_cli "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/rsl_rl/train.py"
click node_play_cli "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/rsl_rl/play.py"
click node_keyboard "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/rsl_rl/keyboard_controller.py"
click node_task_cfg "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py"
click node_task_env "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env.py"
click node_base_env_cfg "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe25_v3/env_cfg.py"
click node_mdp_functions "https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_tasks/agent_tasks/manager/mdp/isaaclab"
click node_terrain_cfg "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/isaaclab/terrains.py"
click node_terrain_commands "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/terrain/terrain_command_manager.py"
click node_terrain_tasks "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/terrain/terrain_task_manager.py"
click node_sm_dispatch "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/state_machines/manager.py"
click node_sm_states "https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_tasks/agent_tasks/direct/wheelbipe/state_machines"
click node_mdp_sm "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/state_machine/manager.py"
click node_runner "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/runners/on_constraint_policy_runner.py"
click node_ppo_cfg "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/agents/rsl_rl_ppo_cfg.py"
click node_np3o "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/algorithms/np3o.py"
click node_actor_critic "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/modules/actor_critic_balowtwins.py"
click node_other_algorithms "https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_rl/agent_rl/rsl_rl/algorithms"
click node_rollouts "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/storage/rollout_storage.py"
click node_policy_export "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/utils/exporter.py"
click node_world_actuators "https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_world/agent_world/actuators"
click node_heightfield "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_world/agent_world/terrains/height_field.py"
click node_trace_capture "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/utils/velocity_trace_html.py"
click node_trace_export "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/utils/export_velocity_trace_html.py"
click node_plotter "https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/utils/realtime_plotter.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_cli,node_play_cli,node_keyboard toneBlue
class node_task_cfg,node_task_env,node_base_env_cfg,node_mdp_functions,node_terrain_cfg,node_terrain_commands,node_terrain_tasks,node_sm_dispatch,node_sm_states,node_mdp_sm toneAmber
class node_runner,node_ppo_cfg,node_np3o,node_actor_critic,node_other_algorithms,node_rollouts,node_policy_export toneMint
class node_world_actuators,node_heightfield toneRose
class node_trace_capture,node_trace_export,node_plotter,node_engineer toneIndigo
```

## Components

### User Workflows

- [Training CLI](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/rsl_rl/train.py): `scripts/rsl_rl/train.py`
- [Play CLI](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/rsl_rl/play.py): `scripts/rsl_rl/play.py`
- [Keyboard Control](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/rsl_rl/keyboard_controller.py): `scripts/rsl_rl/keyboard_controller.py`

### Simulation Tasks

- [V14 Task Config](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py): `source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py`
- [V14 Environment](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env.py): `source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env.py`
- [Base Task Config](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe25_v3/env_cfg.py): `source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe25_v3/env_cfg.py`
- [MDP Functions](https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_tasks/agent_tasks/manager/mdp/isaaclab): `source/agent_tasks/agent_tasks/manager/mdp/isaaclab`
- [Terrain Catalog](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/isaaclab/terrains.py): `source/agent_tasks/agent_tasks/manager/mdp/isaaclab/terrains.py`
- [Terrain Commands](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/terrain/terrain_command_manager.py): `source/agent_tasks/agent_tasks/manager/mdp/terrain/terrain_command_manager.py`
- [Terrain Mapping](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/terrain/terrain_task_manager.py): `source/agent_tasks/agent_tasks/manager/mdp/terrain/terrain_task_manager.py`
- [State Machine Stack](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/state_machines/manager.py): `source/agent_tasks/agent_tasks/direct/wheelbipe/state_machines/manager.py`
- [Movement States](https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_tasks/agent_tasks/direct/wheelbipe/state_machines): `source/agent_tasks/agent_tasks/direct/wheelbipe/state_machines`
- [State Machine Manager](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/manager/mdp/state_machine/manager.py): `source/agent_tasks/agent_tasks/manager/mdp/state_machine/manager.py`

### Policy Learning

- [Policy Runner](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/runners/on_constraint_policy_runner.py): `source/agent_rl/agent_rl/rsl_rl/runners/on_constraint_policy_runner.py`
- [PPO Task Config](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_tasks/agent_tasks/direct/wheelbipe/agents/rsl_rl_ppo_cfg.py): `source/agent_tasks/agent_tasks/direct/wheelbipe/agents/rsl_rl_ppo_cfg.py`
- [NP3O Algorithm](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/algorithms/np3o.py): `source/agent_rl/agent_rl/rsl_rl/algorithms/np3o.py`
- [BarlowTwins Policy](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/modules/actor_critic_balowtwins.py): `source/agent_rl/agent_rl/rsl_rl/modules/actor_critic_balowtwins.py`
- [PPO Variants](https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_rl/agent_rl/rsl_rl/algorithms): `source/agent_rl/agent_rl/rsl_rl/algorithms`
- [Rollout Storage](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/storage/rollout_storage.py): `source/agent_rl/agent_rl/rsl_rl/storage/rollout_storage.py`
- [Policy Export](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_rl/agent_rl/rsl_rl/utils/exporter.py): `source/agent_rl/agent_rl/rsl_rl/utils/exporter.py`

### Robot World

- [Robot Actuators](https://github.com/scutrobotlab/wheeled-legged_rl/tree/main/source/agent_world/agent_world/actuators): `source/agent_world/agent_world/actuators`
- [Heightfield Terrain](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/source/agent_world/agent_world/terrains/height_field.py): `source/agent_world/agent_world/terrains/height_field.py`

### Analysis Export

- [Trace Capture](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/utils/velocity_trace_html.py): `scripts/utils/velocity_trace_html.py`
- [HTML Export](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/utils/export_velocity_trace_html.py): `scripts/utils/export_velocity_trace_html.py`
- [Realtime Plotter](https://github.com/scutrobotlab/wheeled-legged_rl/blob/main/scripts/utils/realtime_plotter.py): `scripts/utils/realtime_plotter.py`

### Other components

- **Engineer**

## Connections

- Engineer → Training CLI: starts training
- Engineer → Play CLI: starts playback
- Training CLI → V14 Task Config: selects task
- Play CLI → V14 Task Config: selects task
- Play CLI → Keyboard Control: enables control
- Training CLI → Policy Runner: trains with
- Play CLI → Policy Runner: runs policy
- V14 Task Config → V14 Environment: configures (evidence: `source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py`)
- V14 Task Config → Base Task Config: extends configuration (evidence: `source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py`)
- V14 Task Config → Terrain Catalog: selects terrain (evidence: `source/agent_tasks/agent_tasks/direct/wheelbipe/wheelbipe_V14/env_cfg.py`)
- V14 Environment → MDP Functions: uses task terms
- V14 Environment → State Machine Stack: configures state machines
- State Machine Stack → Movement States: selects machines (evidence: `source/agent_tasks/agent_tasks/direct/wheelbipe/state_machines/manager.py`)
- State Machine Manager → State Machine Stack: manages behavior
- V14 Environment → Terrain Commands: uses terrain profiles
- V14 Environment → Terrain Mapping: maps terrain IDs
- Terrain Catalog → Heightfield Terrain: defines terrain (evidence: `source/agent_tasks/agent_tasks/manager/mdp/isaaclab/terrains.py`)
- Policy Runner → NP3O Algorithm: optimizes with
- Policy Runner → PPO Variants: runs variants (evidence: `source/agent_rl/agent_rl/rsl_rl/runners/on_constraint_policy_runner.py`)
- NP3O Algorithm → BarlowTwins Policy: uses policy (evidence: `source/agent_rl/agent_rl/rsl_rl/algorithms/np3o.py`)
- NP3O Algorithm → Rollout Storage: writes transitions (evidence: `source/agent_rl/agent_rl/rsl_rl/algorithms/np3o.py`)
- Policy Runner → PPO Task Config: uses algorithm config
- Policy Runner → Policy Export: exports policy
- V14 Environment → Robot Actuators: drives actuators
- Play CLI → Trace Capture: records play data
- Trace Capture → HTML Export: exports HTML
- Play CLI → Realtime Plotter: shows live plots
