How hitszcjh/rmua_rl works
RMUA RL combines batched quadrotor simulation with two policy-development routes: differentiable-physics training and rsl_rl reinforcement learning, followed by evaluation and ROS-oriented deployment. The central RL path wraps the simulator as a vector environment, collects rollouts, updates policies, and evaluates or exports them. The repository also includes vision/recurrent policy options and deployment-side state estimation and flight-control components. Some deployment wiring and unsampled modules are represented only where documented or explicitly referenced; uncertain connections are omitted.
Main entry points: on_policy_runner.py On-Policy Runner · RL/learn.py RL Training Entry · utils.py Environment Factory · DiffPhys/learn.py DiffPhys Training
Components 25
Simulation
- Environment Factory
Env/utils.py - CUDA Quadrotor
Env/env_cuda.py
Differentiable Physics
- DiffPhys Training
DiffPhys/learn.py - Vision Dynamics Policy
DiffPhys/model.py - DiffPhys Evaluation
DiffPhys/play.py
RL Training
- RL Training Entry
RL/learn.py - RL Evaluation Entry
RL/play.py - RL Environment Adapter
RL/rl_env_wrapper.py - On-Policy Runner
RL/rsl_rl/rsl_rl/runners/on_policy_runner.py - PPO Algorithm
RL/rsl_rl/rsl_rl/algorithms/ppo.py - MDPO Algorithm
RL/rsl_rl/rsl_rl/algorithms/mdpo.py - Rollout Storage
RL/rsl_rl/rsl_rl/storage/rollout_storage.py - Policy Families
RL/rsl_rl/rsl_rl/modules - Vision Recurrent Policy
RL/rsl_rl/rsl_rl/modules/vision_actor_critic.py - Network Components
RL/rsl_rl/rsl_rl/networks - Training Logger
RL/rsl_rl/rsl_rl/utils/logger.py
ROS Deployment
- LiDAR Inertial Mapping
deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp - IMU Processing
deployments/rmua_ws/src/FAST_LIO/src/IMU_Processing.hpp - Point Cloud Preprocessing
deployments/rmua_ws/src/FAST_LIO/src/preprocess.h - IMU GPS Filter
deployments/rmua_ws/src/imu_gps_odometry/src/eskf.cpp - PX4 Controller
deployments/rmua_ws/src/basic/src/px4controller.cpp - Diff Policy Controller
deployments/rmua_ws/src/basic/src/uav_diff_controller.cpp - RL Policy Controller
deployments/rmua_ws/src/basic/src/uav_rl_controller.cpp
Other components
- Engineer
- ROS / PX4 Runtime
Connections 28
- Engineer → DiffPhys Training: starts training
README.md - DiffPhys Training → Environment Factory: creates simulator
DiffPhys/learn.py - DiffPhys Training → Vision Dynamics Policy: trains policy
- Engineer → DiffPhys Evaluation: starts evaluation
README.md - DiffPhys Evaluation → Environment Factory: creates simulator
DiffPhys/play.py - Environment Factory → CUDA Quadrotor: constructs
Env/utils.py - Engineer → RL Training Entry: starts training
README.md - RL Training Entry → Environment Factory: builds environment
RL/learn.py - RL Training Entry → RL Environment Adapter: wraps simulator
RL/learn.py - RL Training Entry → On-Policy Runner: starts runner
RL/learn.py - RL Environment Adapter → CUDA Quadrotor: steps simulator
RL/rl_env_wrapper.py - On-Policy Runner → PPO Algorithm: constructs algorithm
RL/rsl_rl/rsl_rl/runners/on_policy_runner.py - On-Policy Runner → MDPO Algorithm: constructs algorithm
RL/rsl_rl/rsl_rl/runners/on_policy_runner.py - On-Policy Runner → Policy Families: selects policy
RL/rsl_rl/rsl_rl/runners/on_policy_runner.py - On-Policy Runner → Rollout Storage: uses storage
RL/rsl_rl/rsl_rl/runners/on_policy_runner.py - On-Policy Runner → Training Logger: records metrics
RL/rsl_rl/rsl_rl/runners/on_policy_runner.py - PPO Algorithm → Rollout Storage: reads batches
RL/rsl_rl/rsl_rl/algorithms/ppo.py - MDPO Algorithm → Rollout Storage: uses storage
RL/rsl_rl/rsl_rl/algorithms/mdpo.py - Policy Families → Vision Recurrent Policy: exports policy
RL/rsl_rl/rsl_rl/modules/__init__.py - Vision Recurrent Policy → Network Components: uses components
RL/rsl_rl/rsl_rl/modules/vision_actor_critic.py - Engineer → RL Evaluation Entry: starts evaluation
README.md - RL Evaluation Entry → RL Environment Adapter: creates environment
RL/play.py - RL Evaluation Entry → On-Policy Runner: loads policy
RL/play.py - LiDAR Inertial Mapping → IMU Processing: uses IMU processing
deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp - LiDAR Inertial Mapping → Point Cloud Preprocessing: uses preprocessing
deployments/rmua_ws/src/FAST_LIO/src/laserMapping.cpp - ROS / PX4 Runtime → PX4 Controller: runs controller
- ROS / PX4 Runtime → Diff Policy Controller: runs controller
- ROS / PX4 Runtime → RL Policy Controller: runs controller