How nvidia-omniverse/sample-ackermann-amr works
Project Type and Purpose
• This is an NVIDIA Omniverse Isaac Sim sample for training and evaluating an AI-driven Ackermann steering autonomous mobile robot (AMR).
• It combines:
• A 3D simulation environment (USD assets under assets/)
• Omniverse extensions implementing controller and trainer logic (exts/omni.sample.ackermann_amr_trainer)
• A ROS2-based evaluation/training client (ros2_f1_tenth_trainer)
• Utility tooling (tools/packman for dependency/bootstrap, link_app scripts)
• The README and tutorial explain how to spin up the sim, collect training data, train a policy, and run inference in ROS2.
File Structure
• assets/ → USD scene files, 3D models, environment definitions
• exts/omni.sample.ackermann_amr_trainer/ → Python package with:
· ackermann_amr_data.py (data collection)
· ackermann_amr_model.py & ackermann_amr_policy.py (model, policy definitions)
· trainer window + extension hooks
• ros2_f1_tenth_trainer/ → ROS2 Python nodes for teleop, model evaluation, and bridge to the trained policy (road_following_model.pth)
• tools/packman/ & tools/scripts/ → bootstrap and package management utilities
• pyproject.toml → Python packaging, dependencies
• link_app.{bat,sh} → helper scripts to link the project into Omniverse
README for Additional Insights
• Points to separate controller, trainer, and ROS packages with their own READMEs.
• Tutorial clarifies workflow:
a. Launch Omniverse Isaac Sim with the Ackermann extension
b. Collect data and train in-sim
c. Export a PyTorch model (.pth)
d. Evaluate using ROS2 nodes on a physical or simulated robot
a. Main Components
• Omniverse Isaac Sim Engine
• USD Asset Repository (assets/)
• Trainer Extension (exts/omni.sample.ackermann_amr_trainer)
• Controller Extension (ackermann_amr_controller – referenced, similar structure)
• ROS2 Evaluation Nodes (ros2_f1_tenth_trainer)
• Model Storage (road_following_model.pth)
• Dependency Manager (tools/packman, PyPI via pyproject.toml)
b. Relationships & Interactions
1. Isaac Sim loads USD assets and the Trainer Extension
2. Trainer Extension steps through sim frames → collects sensor observations → ackermann_amr_data module
3. Observations fed to ackermann_amr_model + policy → outputs steering commands
4. Steering commands applied in sim → new observations → RL loop
5. Trained model serialized to disk (road_following_model.pth)
6. ROS2 evaluation nodes load .pth → subscribe to robot topics → publish drive commands
7. packman and link_app scripts bootstrap environment and resolve dependencies
c. Architectural Patterns & Principles
• Modular extension architecture (Omniverse extensions as plugins)
• Clear separation of simulation (C++/USD engine) vs. Python control logic
• Data-driven RL pipeline (data collection → model training → inference)
• Standard ROS2 pub/sub for real-world integration
d. Key Technologies
• NVIDIA Omniverse Isaac Sim (USD, extension API)
• Python 3 + pyproject.toml packaging
• PyTorch (.pth model)
• ROS2 (rclpy, topics, nodes)
• Packman (Omniverse dependency manager)