# nvidia-omniverse/sample-ackermann-amr architecture

> Architecture diagram of the GitHub repository nvidia-omniverse/sample-ackermann-amr, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia-omniverse/sample-ackermann-amr
- Repository: https://github.com/nvidia-omniverse/sample-ackermann-amr

## Overview

##### Project Type and Purpose
- This is an NVIDIA Omniverse Isaac Sim sample for training and evaluating an AI-driven Ackermann steering autonomous mobile robot (AMR).
- It combines:
- A 3D simulation environment (USD assets under assets/)
- Omniverse extensions implementing controller and trainer logic (exts/omni.sample.ackermann_amr_trainer)
- A ROS2-based evaluation/training client (ros2_f1_tenth_trainer)
- Utility tooling (tools/packman for dependency/bootstrap, link_app scripts)
- The README and tutorial explain how to spin up the sim, collect training data, train a policy, and run inference in ROS2.

##### File Structure
- assets/ → USD scene files, 3D models, environment definitions
- exts/omni.sample.ackermann_amr_trainer/ → Python package with:
· ackermann_amr_data.py (data collection)
· ackermann_amr_model.py & ackermann_amr_policy.py (model, policy definitions)
· trainer window + extension hooks
- ros2_f1_tenth_trainer/ → ROS2 Python nodes for teleop, model evaluation, and bridge to the trained policy (road_following_model.pth)
- tools/packman/ & tools/scripts/ → bootstrap and package management utilities
- pyproject.toml → Python packaging, dependencies
- link_app.{bat,sh} → helper scripts to link the project into Omniverse

##### README for Additional Insights
- Points to separate controller, trainer, and ROS packages with their own READMEs.
- Tutorial clarifies workflow:
a. Launch Omniverse Isaac Sim with the Ackermann extension
b. Collect data and train in-sim
c. Export a PyTorch model (.pth)
d. Evaluate using ROS2 nodes on a physical or simulated robot

a. Main Components
- Omniverse Isaac Sim Engine
- USD Asset Repository (assets/)
- Trainer Extension (exts/omni.sample.ackermann_amr_trainer)
- Controller Extension (ackermann_amr_controller – referenced, similar structure)
- ROS2 Evaluation Nodes (ros2_f1_tenth_trainer)
- Model Storage (road_following_model.pth)
- Dependency Manager (tools/packman, PyPI via pyproject.toml)
b. Relationships & Interactions
1. Isaac Sim loads USD assets and the Trainer Extension
2. Trainer Extension steps through sim frames → collects sensor observations → ackermann_amr_data module
3. Observations fed to ackermann_amr_model + policy → outputs steering commands
4. Steering commands applied in sim → new observations → RL loop
5. Trained model serialized to disk (road_following_model.pth)
6. ROS2 evaluation nodes load .pth → subscribe to robot topics → publish drive commands
7. packman and link_app scripts bootstrap environment and resolve dependencies
c. Architectural Patterns & Principles
- Modular extension architecture (Omniverse extensions as plugins)
- Clear separation of simulation (C++/USD engine) vs. Python control logic
- Data-driven RL pipeline (data collection → model training → inference)
- Standard ROS2 pub/sub for real-world integration
d. Key Technologies
- NVIDIA Omniverse Isaac Sim (USD, extension API)
- Python 3 + pyproject.toml packaging
- PyTorch (.pth model)
- ROS2 (rclpy, topics, nodes)
- Packman (Omniverse dependency manager)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Simulation Layer
    SimEngine["Omniverse Isaac Sim Engine"]:::sim
    Assets["USD Asset Repository"]:::storage
    click Assets "https://github.com/nvidia-omniverse/sample-ackermann-amr/tree/main/assets/"

    %% Extensions Layer
    subgraph "Trainer Extension" 
        ExtPackage["Trainer Extension Package"]:::ext
        click ExtPackage "https://github.com/nvidia-omniverse/sample-ackermann-amr/tree/main/exts/omni.sample.ackermann_amr_trainer/"
        Config["extension.toml (config)"]:::ext
        click Config "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/exts/omni.sample.ackermann_amr_trainer/config/extension.toml"
        DataModule["ackermann_amr_data.py"]:::ext
        click DataModule "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/exts/omni.sample.ackermann_amr_trainer/omni/sample/ackermann_amr_trainer/ackermann_amr_data.py"
        ModelDef["ackermann_amr_model.py"]:::ext
        click ModelDef "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/exts/omni.sample.ackermann_amr_trainer/omni/sample/ackermann_amr_trainer/ackermann_amr_model.py"
        PolicyImpl["ackermann_amr_policy.py"]:::ext
        click PolicyImpl "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/exts/omni.sample.ackermann_amr_trainer/omni/sample/ackermann_amr_trainer/ackermann_amr_policy.py"
        TrainerWindow["ackermann_trainer_window.py"]:::ext
        click TrainerWindow "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/exts/omni.sample.ackermann_amr_trainer/omni/sample/ackermann_amr_trainer/ackermann_trainer_window.py"
        ExtEntry["extension.py (entry point)"]:::ext
        click ExtEntry "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/exts/omni.sample.ackermann_amr_trainer/omni/sample/ackermann_amr_trainer/extension.py"
    end

    %% ML Loop
    SimEngine -->|"loads assets and extension"| Assets
    SimEngine -->|"loads extension"| ExtPackage
    ExtPackage -->|runs| ExtEntry
    ExtEntry -->|initializes UI| TrainerWindow
    TrainerWindow -->|collects sensors| DataModule
    DataModule -->|feeds observations| ModelDef
    ModelDef -->|computes policy| PolicyImpl
    PolicyImpl -->|outputs steering commands| SimEngine

    %% Model Storage
    ModelFile["road_following_model.pth"]:::storage
    click ModelFile "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/ros2_f1_tenth_trainer/road_following_model.pth"
    PolicyImpl -->|saves model| ModelFile

    %% ROS2 Evaluation Layer
    subgraph "ROS2 Evaluation Nodes"
        ROS2["ros2_f1_tenth_trainer"]:::ros
        click ROS2 "https://github.com/nvidia-omniverse/sample-ackermann-amr/tree/main/ros2_f1_tenth_trainer/"
        TeleopNode["teleop_ackermann_key.py"]:::ros
        click TeleopNode "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/ros2_f1_tenth_trainer/teleop_ackermann_key.py"
        ModelEvalNode["model_evaluate.py"]:::ros
        click ModelEvalNode "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/ros2_f1_tenth_trainer/model_evaluate.py"
        ROS2PolicyLoader["ackermann_amr_policy.py"]:::ros
        click ROS2PolicyLoader "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/ros2_f1_tenth_trainer/ackermann_amr_policy.py"
        ROS2ModelFile["road_following_model.pth"]:::ros
        click ROS2ModelFile "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/ros2_f1_tenth_trainer/road_following_model.pth"
    end

    ModelFile -->|loaded by| ROS2PolicyLoader
    ROS2PolicyLoader -->|publishes commands| SimEngine
    SimEngine -->|publishes topics| ROS2

    %% Tools & Bootstrap
    subgraph "Bootstrap & Dependencies"
        Packman["Packman (tools/packman)"]:::tools
        click Packman "https://github.com/nvidia-omniverse/sample-ackermann-amr/tree/main/tools/packman/"
        PackmanBootstrap["bootstrap scripts"]:::tools
        click PackmanBootstrap "https://github.com/nvidia-omniverse/sample-ackermann-amr/tree/main/tools/packman/bootstrap/"
        PackmanConfig["config.packman.xml"]:::tools
        click PackmanConfig "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/tools/packman/config.packman.xml"
        PyProject["pyproject.toml"]:::tools
        click PyProject "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/pyproject.toml"
        LinkScripts["link_app.sh/.bat"]:::tools
        click LinkScripts "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/link_app.sh and link_app.bat"
    end

    Packman -->|installs deps| SimEngine
    LinkScripts -->|links project| SimEngine

    %% Documentation
    README["README.md"]:::doc
    click README "https://github.com/nvidia-omniverse/sample-ackermann-amr/blob/main/README.md"

    README --> SimEngine
    README --> ROS2
    README --> Packman

    %% Styles
    classDef sim fill:#D0E6FF,stroke:#4A90E2,color:#000;
    classDef ext fill:#FFE0B2,stroke:#FB8C00,color:#000;
    classDef ros fill:#C8E6C9,stroke:#43A047,color:#000;
    classDef storage fill:#ECEFF1,stroke:#90A4AE,color:#000;
    classDef tools fill:#FFF9C4,stroke:#FBC02D,color:#000;
    classDef doc fill:#F3E5F5,stroke:#8E24AA,color:#000;
```
