# hitszcjh/rmua_rl architecture

> Architecture diagram of the GitHub repository hitszcjh/rmua_rl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hitszcjh/rmua_rl
- Repository: https://github.com/hitszcjh/rmua_rl

## Overview

RMUA RL combines batched quadrotor simulation with two policy-development routes: differentiable-physics training and rsl_rl reinforcement learning, followed by evaluation and ROS-oriented deployment. The central RL path wraps the simulator as a vector environment, collects rollouts, updates policies, and evaluates or exports them. The repository also includes vision/recurrent policy options and deployment-side state estimation and flight-control components. Some deployment wiring and unsampled modules are represented only where documented or explicitly referenced; uncertain connections are omitted.

Main entry points: [`RL/rsl_rl/rsl_rl/runners/on_policy_runner.py`](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/runners/on_policy_runner.py) (On-Policy Runner), [`RL/learn.py`](https://github.com/hitszcjh/rmua_rl/blob/master/RL/learn.py) (RL Training Entry), [`Env/utils.py`](https://github.com/hitszcjh/rmua_rl/blob/master/Env/utils.py) (Environment Factory), [`DiffPhys/learn.py`](https://github.com/hitszcjh/rmua_rl/blob/master/DiffPhys/learn.py) (DiffPhys Training)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_simulation["Simulation"]
  node_env_factory["Environment Factory<br/>[utils.py]"]
  node_quad_env["CUDA Quadrotor<br/>[env_cuda.py]"]
end

subgraph group_diffphys["Differentiable Physics"]
  node_diff_learn["DiffPhys Training<br/>[learn.py]"]
  node_diff_model["Vision Dynamics Policy<br/>[model.py]"]
  node_diff_play["DiffPhys Evaluation<br/>[play.py]"]
end

subgraph group_rl["RL Training"]
  node_rl_learn["RL Training Entry<br/>[learn.py]"]
  node_rl_play["RL Evaluation Entry<br/>[play.py]"]
  node_rl_wrapper["RL Environment Adapter<br/>[rl_env_wrapper.py]"]
  node_runner["On-Policy Runner"]
  node_ppo["PPO Algorithm<br/>[ppo.py]"]
  node_mdpo["MDPO Algorithm<br/>[mdpo.py]"]
  node_rollout[("Rollout Storage<br/>[rollout_storage.py]")]
  node_policy_modules["Policy Families"]
  node_vision_policy["Vision Recurrent Policy"]
  node_networks["Network Components"]
  node_logger["Training Logger<br/>[logger.py]"]
end

subgraph group_deployment["ROS Deployment"]
  node_fast_lio["LiDAR Inertial Mapping<br/>[laserMapping.cpp]"]
  node_imu_processing["IMU Processing<br/>[IMU_Processing.hpp]"]
  node_preprocess["Point Cloud Preprocessing<br/>[preprocess.h]"]
  node_odom_filter["IMU GPS Filter<br/>[eskf.cpp]"]
  node_flight_control["PX4 Controller<br/>[px4controller.cpp]"]
  node_diff_controller["Diff Policy Controller"]
  node_rl_controller["RL Policy Controller"]
end

node_user(("Engineer"))
node_ros(("ROS / PX4 Runtime"))

node_user -->|"starts training"| node_diff_learn
node_diff_learn -->|"creates simulator"| node_env_factory
node_diff_learn -.->|"trains policy"| node_diff_model
node_user -->|"starts evaluation"| node_diff_play
node_diff_play -->|"creates simulator"| node_env_factory
node_env_factory -->|"constructs"| node_quad_env
node_user -->|"starts training"| node_rl_learn
node_rl_learn -->|"builds environment"| node_env_factory
node_rl_learn -->|"wraps simulator"| node_rl_wrapper
node_rl_learn -->|"starts runner"| node_runner
node_rl_wrapper -->|"steps simulator"| node_quad_env
node_runner -->|"constructs algorithm"| node_ppo
node_runner -->|"constructs algorithm"| node_mdpo
node_runner -->|"selects policy"| node_policy_modules
node_runner -->|"uses storage"| node_rollout
node_runner -->|"records metrics"| node_logger
node_ppo -->|"reads batches"| node_rollout
node_mdpo -->|"uses storage"| node_rollout
node_policy_modules -->|"exports policy"| node_vision_policy
node_vision_policy -->|"uses components"| node_networks
node_user -->|"starts evaluation"| node_rl_play
node_rl_play -->|"creates environment"| node_rl_wrapper
node_rl_play -->|"loads policy"| node_runner
node_fast_lio -->|"uses IMU processing"| node_imu_processing
node_fast_lio -->|"uses preprocessing"| node_preprocess
node_ros -.->|"runs controller"| node_flight_control
node_ros -.->|"runs controller"| node_diff_controller
node_ros -.->|"runs controller"| node_rl_controller

click node_diff_learn "https://github.com/hitszcjh/rmua_rl/blob/master/DiffPhys/learn.py"
click node_diff_model "https://github.com/hitszcjh/rmua_rl/blob/master/DiffPhys/model.py"
click node_diff_play "https://github.com/hitszcjh/rmua_rl/blob/master/DiffPhys/play.py"
click node_env_factory "https://github.com/hitszcjh/rmua_rl/blob/master/Env/utils.py"
click node_quad_env "https://github.com/hitszcjh/rmua_rl/blob/master/Env/env_cuda.py"
click node_rl_learn "https://github.com/hitszcjh/rmua_rl/blob/master/RL/learn.py"
click node_rl_play "https://github.com/hitszcjh/rmua_rl/blob/master/RL/play.py"
click node_rl_wrapper "https://github.com/hitszcjh/rmua_rl/blob/master/RL/rl_env_wrapper.py"
click node_runner "https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/runners/on_policy_runner.py"
click node_ppo "https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/algorithms/ppo.py"
click node_mdpo "https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/algorithms/mdpo.py"
click node_rollout "https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/storage/rollout_storage.py"
click node_policy_modules "https://github.com/hitszcjh/rmua_rl/tree/master/RL/rsl_rl/rsl_rl/modules"
click node_vision_policy "https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/modules/vision_actor_critic.py"
click node_networks "https://github.com/hitszcjh/rmua_rl/tree/master/RL/rsl_rl/rsl_rl/networks"
click node_logger "https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/utils/logger.py"
click node_fast_lio "https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp"
click node_imu_processing "https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/FAST_LIO/src/IMU_Processing.hpp"
click node_preprocess "https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/FAST_LIO/src/preprocess.h"
click node_odom_filter "https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/imu_gps_odometry/src/eskf.cpp"
click node_flight_control "https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/basic/src/px4controller.cpp"
click node_diff_controller "https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/basic/src/uav_diff_controller.cpp"
click node_rl_controller "https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/basic/src/uav_rl_controller.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_env_factory,node_quad_env toneBlue
class node_diff_learn,node_diff_model,node_diff_play toneAmber
class node_rl_learn,node_rl_play,node_rl_wrapper,node_runner,node_ppo,node_mdpo,node_rollout,node_policy_modules,node_vision_policy,node_networks,node_logger toneMint
class node_fast_lio,node_imu_processing,node_preprocess,node_odom_filter,node_flight_control,node_diff_controller,node_rl_controller toneRose
class node_user,node_ros toneIndigo
```

## Components

### Simulation

- [Environment Factory](https://github.com/hitszcjh/rmua_rl/blob/master/Env/utils.py): `Env/utils.py`
- [CUDA Quadrotor](https://github.com/hitszcjh/rmua_rl/blob/master/Env/env_cuda.py): `Env/env_cuda.py`

### Differentiable Physics

- [DiffPhys Training](https://github.com/hitszcjh/rmua_rl/blob/master/DiffPhys/learn.py): `DiffPhys/learn.py`
- [Vision Dynamics Policy](https://github.com/hitszcjh/rmua_rl/blob/master/DiffPhys/model.py): `DiffPhys/model.py`
- [DiffPhys Evaluation](https://github.com/hitszcjh/rmua_rl/blob/master/DiffPhys/play.py): `DiffPhys/play.py`

### RL Training

- [RL Training Entry](https://github.com/hitszcjh/rmua_rl/blob/master/RL/learn.py): `RL/learn.py`
- [RL Evaluation Entry](https://github.com/hitszcjh/rmua_rl/blob/master/RL/play.py): `RL/play.py`
- [RL Environment Adapter](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rl_env_wrapper.py): `RL/rl_env_wrapper.py`
- [On-Policy Runner](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/runners/on_policy_runner.py): `RL/rsl_rl/rsl_rl/runners/on_policy_runner.py`
- [PPO Algorithm](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/algorithms/ppo.py): `RL/rsl_rl/rsl_rl/algorithms/ppo.py`
- [MDPO Algorithm](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/algorithms/mdpo.py): `RL/rsl_rl/rsl_rl/algorithms/mdpo.py`
- [Rollout Storage](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/storage/rollout_storage.py): `RL/rsl_rl/rsl_rl/storage/rollout_storage.py`
- [Policy Families](https://github.com/hitszcjh/rmua_rl/tree/master/RL/rsl_rl/rsl_rl/modules): `RL/rsl_rl/rsl_rl/modules`
- [Vision Recurrent Policy](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/modules/vision_actor_critic.py): `RL/rsl_rl/rsl_rl/modules/vision_actor_critic.py`
- [Network Components](https://github.com/hitszcjh/rmua_rl/tree/master/RL/rsl_rl/rsl_rl/networks): `RL/rsl_rl/rsl_rl/networks`
- [Training Logger](https://github.com/hitszcjh/rmua_rl/blob/master/RL/rsl_rl/rsl_rl/utils/logger.py): `RL/rsl_rl/rsl_rl/utils/logger.py`

### ROS Deployment

- [LiDAR Inertial Mapping](https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp): `deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp`
- [IMU Processing](https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/FAST_LIO/src/IMU_Processing.hpp): `deployments/rmua_ws/src/FAST_LIO/src/IMU_Processing.hpp`
- [Point Cloud Preprocessing](https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/FAST_LIO/src/preprocess.h): `deployments/rmua_ws/src/FAST_LIO/src/preprocess.h`
- [IMU GPS Filter](https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/imu_gps_odometry/src/eskf.cpp): `deployments/rmua_ws/src/imu_gps_odometry/src/eskf.cpp`
- [PX4 Controller](https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/basic/src/px4controller.cpp): `deployments/rmua_ws/src/basic/src/px4controller.cpp`
- [Diff Policy Controller](https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/basic/src/uav_diff_controller.cpp): `deployments/rmua_ws/src/basic/src/uav_diff_controller.cpp`
- [RL Policy Controller](https://github.com/hitszcjh/rmua_rl/blob/master/deployments/rmua_ws/src/basic/src/uav_rl_controller.cpp): `deployments/rmua_ws/src/basic/src/uav_rl_controller.cpp`

### Other components

- **Engineer**
- **ROS / PX4 Runtime**

## Connections

- Engineer → DiffPhys Training: starts training (evidence: `README.md`)
- DiffPhys Training → Environment Factory: creates simulator (evidence: `DiffPhys/learn.py`)
- DiffPhys Training → Vision Dynamics Policy: trains policy
- Engineer → DiffPhys Evaluation: starts evaluation (evidence: `README.md`)
- DiffPhys Evaluation → Environment Factory: creates simulator (evidence: `DiffPhys/play.py`)
- Environment Factory → CUDA Quadrotor: constructs (evidence: `Env/utils.py`)
- Engineer → RL Training Entry: starts training (evidence: `README.md`)
- RL Training Entry → Environment Factory: builds environment (evidence: `RL/learn.py`)
- RL Training Entry → RL Environment Adapter: wraps simulator (evidence: `RL/learn.py`)
- RL Training Entry → On-Policy Runner: starts runner (evidence: `RL/learn.py`)
- RL Environment Adapter → CUDA Quadrotor: steps simulator (evidence: `RL/rl_env_wrapper.py`)
- On-Policy Runner → PPO Algorithm: constructs algorithm (evidence: `RL/rsl_rl/rsl_rl/runners/on_policy_runner.py`)
- On-Policy Runner → MDPO Algorithm: constructs algorithm (evidence: `RL/rsl_rl/rsl_rl/runners/on_policy_runner.py`)
- On-Policy Runner → Policy Families: selects policy (evidence: `RL/rsl_rl/rsl_rl/runners/on_policy_runner.py`)
- On-Policy Runner → Rollout Storage: uses storage (evidence: `RL/rsl_rl/rsl_rl/runners/on_policy_runner.py`)
- On-Policy Runner → Training Logger: records metrics (evidence: `RL/rsl_rl/rsl_rl/runners/on_policy_runner.py`)
- PPO Algorithm → Rollout Storage: reads batches (evidence: `RL/rsl_rl/rsl_rl/algorithms/ppo.py`)
- MDPO Algorithm → Rollout Storage: uses storage (evidence: `RL/rsl_rl/rsl_rl/algorithms/mdpo.py`)
- Policy Families → Vision Recurrent Policy: exports policy (evidence: `RL/rsl_rl/rsl_rl/modules/__init__.py`)
- Vision Recurrent Policy → Network Components: uses components (evidence: `RL/rsl_rl/rsl_rl/modules/vision_actor_critic.py`)
- Engineer → RL Evaluation Entry: starts evaluation (evidence: `README.md`)
- RL Evaluation Entry → RL Environment Adapter: creates environment (evidence: `RL/play.py`)
- RL Evaluation Entry → On-Policy Runner: loads policy (evidence: `RL/play.py`)
- LiDAR Inertial Mapping → IMU Processing: uses IMU processing (evidence: `deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp`)
- LiDAR Inertial Mapping → Point Cloud Preprocessing: uses preprocessing (evidence: `deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp`)
- ROS / PX4 Runtime → PX4 Controller: runs controller
- ROS / PX4 Runtime → Diff Policy Controller: runs controller
- ROS / PX4 Runtime → RL Policy Controller: runs controller
