How reiniscimurs/drl-robot-navigation-ros2 works
Project type and purpose
• A robotics full‐stack application combining:
– Gazebo simulation (virtual environment)
– ROS2 middleware for robot communication
– Deep Reinforcement Learning (TD3 & SAC agents) implemented in Python/PyTorch
– Visualization (RViz) and logging (Tensorboard)
File structure
• src/drl_navigation_ros2 – core Python DRL modules (SAC, TD3, replay_buffer, pretrain_utils, utils, ros_nodes)
• src/turtlebot3_description – URDF models and meshes for TurtleBot3 robots
• src/turtlebot3_simulations – Gazebo world definitions, robot plugins, launch files
• tests – unit tests for default behaviors
• pyproject.toml, colcon build files – build and packaging configuration
README insights
• ROS2 Foxy + Python3.8 + PyTorch + Tensorboard are core dependencies
• Training workflow: launch Gazebo → launch training script → monitor via RViz & Tensorboard
• ROS2 nodes and DRL code interoperate via ROS2 topics
Main components and interactions (to show
a. Gazebo Simulator
– Hosts the world (obstacles, robot model)
– Publishes simulated laser scans (/scan), odometry (/odom)
b. ROS2 Middleware (DDS)
– Manages topics, services, parameters
– Launch files (ros2_drl.launch.py) coordinate nodes
c. DRL Training Node (train.py)
– Contains agent loop: observe → compute action → publish cmd_vel
– Uses SAC and/or TD3 modules under src/drl_navigation_ros2/SAC and /TD3
– Interfaces to replay_buffer module to store transitions
– Calls pretrain_utils to load or initialize network weights
d. Replay Buffer (replay_buffer.py)
– Data store for (state, action, reward, next_state) tuples
e. Neural‐Network Modules
– SAC_actor.py, SAC_critic.py, TD3.py – define policy and value networks
f. Hardcoded Model Fallback (hardcoded_model.py)
– Provides baseline control policy if needed
g. Visualization & Logging
– RViz2 subscribes to robot state topics for 3D visualization
– Tensorboard reads logs in runs/ directory for training metrics
h. External Configuration
– data.yml (assets) for hyperparameters
– ROS parameters set via launch files
Architectural patterns and design principles
• Modular separation between:
– Simulation environment (Gazebo)
– Middleware (ROS2 nodes, topics)
– Learning algorithms (agent classes, replay buffer)
– Visualization/Logging (RViz, Tensorboard)
• Clear producer–consumer flow (Gazebo → DRL node → Gazebo)
• Dependency inversion via ROS2 topics, enabling hot‐swapping agents