How xuan-waves/ur3e_rl works
This is an experimental UR3e insertion stack spanning VR teleoperation, impedance-controlled execution, demonstration collection, SmolVLA behavior cloning, and gated RLT refinement. The documented workflow moves from demonstrations to policy training and rollout, then through gate/token/intervention training to closed-loop trials. Runtime internals are represented by the ROS nodes, robot API, kinematics, safety, and impedance controller. Several learning subsystems are documented but unsampled; their documented capabilities are shown without inferring internal calls.
Main entry points: nodes.py ROS control nodes · rollout_ur3e_smolvla_no_rotvec_rtc.py No-rotvec VLA rollout · collect_ur3e_vr_impedance.py Impedance data collection · ur3e_api.py Robot API
Components 22
Robot runtime
- Hardware entrypoint
scripts/hardware/ur3e_vr_servoj_ros2.py - ROS control nodes
real_teleop/nodes.py - Robot API
mujoco_env/hardware/ur3e_api.py
Demonstrations and VLA
- Impedance data collection
scripts/collect_data/collect_ur3e_vr_impedance.py - LeRobot demonstrations
scripts/collect_data/lerobot_writer.py - SmolVLA training
scripts/train_smolvla_no_rotvec/train_ur3e_smolvla_no_rotvec.py - No-rotvec VLA rollout
scripts/rollout_smolvla_no_rotvec/rollout_ur3e_smolvla_no_rotvec_rtc.py
RLT refinement
- RL gate
scripts/rlt_gate/train_rlt_gate.py - Gate evaluation and monitor
scripts/rlt_gate/live_rlt_gate_monitor.py - Stage1 token learning
scripts/rlt_token/train_rlt_stage1.py - HIL interventions
scripts/rlt_train/collect_rlt_interventions.py - Stage2 actor critic
scripts/rlt_train/train_hil_serl_stage2.py - Ready-gate trial loop
scripts/rollout_smolvla_no_rotvec/rollout_ur3e_smolvla_no_rotvec_ready_loop.py - Ready-gate training and evaluation
scripts/ready_gate/train_ready_gate.py
Teleoperation core
- Kinematics and IK
real_teleop/kinematics.py - Safety limiter
real_teleop/safety.py - Impedance control
real_teleop/impedance/controller.py - Robot and VR messages
real_teleop/messages.py
Other components
- VR operator
- XRoboToolkit
- RealSense cameras
- UR3e and gripper
Connections 26
- VR operator → XRoboToolkit: provides VR input
- XRoboToolkit → ROS control nodes: supplies teleoperation
- Hardware entrypoint → ROS control nodes: starts nodes
scripts/hardware/ur3e_vr_servoj_ros2.py - ROS control nodes → Robot API: controls robot
real_teleop/nodes.py - ROS control nodes → Kinematics and IK: uses IK
real_teleop/nodes.py - ROS control nodes → Safety limiter: applies limits
real_teleop/nodes.py - ROS control nodes → Impedance control: runs impedance
real_teleop/nodes.py - ROS control nodes → Robot and VR messages: parses commands
real_teleop/nodes.py - Impedance control → Robot API: commands motion
real_teleop/impedance/controller.py - Robot API → UR3e and gripper: drives arm
- RealSense cameras → Impedance data collection: provides images
- VR operator → Impedance data collection: records demonstrations
- ROS control nodes → Impedance data collection: publishes robot state
- Impedance data collection → LeRobot demonstrations: writes episodes
scripts/collect_data/collect_ur3e_vr_impedance.py - LeRobot demonstrations → SmolVLA training: trains on
- SmolVLA training → No-rotvec VLA rollout: produces policy
- No-rotvec VLA rollout → ROS control nodes: publishes targets
- No-rotvec VLA rollout → RL gate: validates rollout
- RL gate → Gate evaluation and monitor: evaluates gate
scripts/rlt_gate/live_rlt_gate_monitor.py - No-rotvec VLA rollout → Stage1 token learning: provides frozen VLA
- Gate evaluation and monitor → HIL interventions: marks refinement phase
scripts/rlt_train/collect_rlt_interventions.py - Stage1 token learning → HIL interventions: guides token features
scripts/rlt_train/collect_rlt_interventions.py - HIL interventions → Stage2 actor critic: trains from buffers
- Stage2 actor critic → Ready-gate trial loop: refines policy actions
- Ready-gate training and evaluation → Ready-gate trial loop: supplies readiness gate
- Ready-gate trial loop → ROS control nodes: executes trials