# vsislab/robotamer4qmini architecture

> Architecture diagram of the GitHub repository vsislab/robotamer4qmini, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/vsislab/robotamer4qmini
- Repository: https://github.com/vsislab/robotamer4qmini

## Overview

##### Project Type and Purpose
- This is a full-stack reinforcement learning framework for bipedal/quadruped robot locomotion using NVIDIA Isaac Gym.
- It supports training (train.py), evaluation/playback (play.py), model export (export_pt2onnx.py), URDF tuning (tune_urdf.py), PID tuning (tune_pid.py), and houses experiments, assets, configs, models, and utilities.

##### Main Components (Blocks)
A. Environment & Simulation Layer
- Isaac Gym simulator (external service)
- env/ (gym_env_wrapper.py, legged_robot.py, tasks, utils/terrain, etc.)
- assets/urdf & meshes
B. Configuration Layer
- config/ (Base.py, BIRL.py)
- command-line overrides (train.py, play.py flags)
C. RL Algorithm Layer
- rl/alg/ppo.py
- rl/storage/rollout_storage.py
- rl/module/… (policy networks, common interfaces)
D. Model Architectures
- model/ (simple_policy.py, rnn.py, tcn.py)
- model/utils.py
E. Training & Execution Scripts
- train.py (training loop)
- play.py (evaluation loop)
- export_pt2onnx.py
- tune_urdf.py
- tune_pid.py
F. Experiment Management & Logging
- experiments/ (pretrained checkpoints, logs, onnx deploy)
- tensorboard, Excel reports
G. Utility Modules
- utils/ (common helpers, yaml, attrdict)
- env/utils (phase_modulator, math, helpers)
- rl/utils

1. train.py reads config → instantiates env wrapper → spawns Isaac Gym environments
2. At each iteration:
– env simulates robot (using URDF & meshes) → returns observations
– rl/alg/ppo policy (model/) consumes observations → outputs actions
– actions → env step → new observations + rewards
– rollout_storage logs transitions → ppo.update() → new policy weights
3. Checkpoints (policy.pt) & training metrics (tensorboard logs) → experiments/
4. play.py uses same env & policy (loads .pt) → renders/writes data/video
5. export_pt2onnx.py loads .pt → onnxruntime export → experiments/<name>/deploy
6. tune_urdf.py loads assets/urdf → viewer; tune_pid.py runs specialized tasks to optimize PID → writes Excel & logs

##### Architectural Patterns & Principles
- Modular separation: environment, algorithm, model architectures, configs, utilities
- Config-driven behavior: swap tasks, tuning modes, hyperparameters
- Plugin-like tasks in env/tasks for extensibility
- Clear I/O boundaries: env ↔ policy, scripts ↔ experiments folder

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Layer
    subgraph "Configuration Layer"
        direction TB
        CFG_Base["Base.py"]:::config
        CFG_BIRL["BIRL.py"]:::config
    end

    %% Scripts Layer
    subgraph "Scripts & Orchestration" 
        direction TB
        Train["train.py"]:::script
        Play["play.py"]:::script
        Export["export_pt2onnx.py"]:::script
        TuneURDF["tune_urdf.py"]:::script
        TunePID["tune_pid.py"]:::script
    end

    %% Simulation & Environment
    subgraph "Simulation Layer"
        direction TB
        IsaacGym["Isaac Gym Simulator"]:::external
        subgraph "Assets"
            direction TB
            URDF["q1.urdf"]:::core
            Meshes["Meshes (.STL)"]:::core
        end
        subgraph "Environment Wrapper & Tasks"
            direction TB
            Wrapper["gym_env_wrapper.py"]:::core
            Robot["legged_robot.py"]:::core
            subgraph "Tasks"
                direction TB
                BaseTask["base_task.py"]:::core
                BIRLTask["birl_task.py"]:::core
                NullTask["null_task.py"]:::core
            end
            subgraph "Env Utils"
                direction TB
                DelayDeque["delay_torch_deque.py"]:::core
                Helpers["helpers.py"]:::core
                Math["math.py"]:::core
                PhaseMod["phase_modulator.py"]:::core
                Terrain["terrain.py"]:::core
            end
        end
    end

    %% RL & Model Layer
    subgraph "RL Core"
        direction TB
        subgraph "RL Algorithm"
            direction TB
            PPO["ppo.py"]:::core
            Storage["rollout_storage.py"]:::core
            subgraph "RL Module Interfaces"
                direction TB
                Common["common.py"]:::core
                Cont["continuous.py"]:::core
                Disc["discrete.py"]:::core
            end
        end
        subgraph "Model Architectures"
            direction TB
            SimplePolicy["simple_policy.py"]:::core
            RNN["rnn.py"]:::core
            TCN["tcn.py"]:::core
            ModelUtils["utils.py"]:::core
        end
    end

    %% Experiment Management
    subgraph "Experiment Management & Logging"
        direction TB
        ExpQ2["experiments/q2"]:::store
        ExpPID["experiments/tune_pid"]:::store
        ExpURDF["experiments/tune_urdf"]:::store
    end

    %% Utility Modules
    subgraph "Utilities"
        direction TB
        UtilsMain["utils/"]:::util
        RLUtils["rl/utils/utils.py"]:::util
    end

    %% Connections
    CFG_Base -->|loads| Train
    CFG_BIRL -->|loads| Train
    Train -->|uses| IsaacGym
    Play -->|uses| IsaacGym
    Train -->|instantiates| Wrapper
    Play -->|instantiates| Wrapper

    IsaacGym -->|provides obs/rewards| Wrapper
    Wrapper -->|obs| PPO
    PPO -->|actions| Wrapper
    Wrapper -->|stores| Storage
    Storage -->|updates| PPO

    Train -->|writes checkpoints/logs| ExpQ2
    Play -->|reads policy| ExpQ2
    Export -->|loads policy| ExpQ2
    Export -->|exports ONNX| ExpQ2
    TuneURDF -->|loads URDF| URDF
    TuneURDF -->|writes logs| ExpURDF
    TunePID -->|runs PID tasks| Wrapper
    TunePID -->|writes Excel| ExpPID

    %% Utilities connections
    Train -->|uses| UtilsMain
    Play -->|uses| UtilsMain
    PPO -->|uses| RLUtils
    Wrapper -->|uses| Helpers

    %% Click Events
    click URDF "https://github.com/vsislab/robotamer4qmini/blob/main/assets/q1/urdf/q1.urdf"
    click Meshes "https://github.com/vsislab/robotamer4qmini/tree/main/assets/q1/meshes"
    click Wrapper "https://github.com/vsislab/robotamer4qmini/blob/main/env/gym_env_wrapper.py"
    click Robot "https://github.com/vsislab/robotamer4qmini/blob/main/env/legged_robot.py"
    click BaseTask "https://github.com/vsislab/robotamer4qmini/blob/main/env/tasks/base_task.py"
    click BIRLTask "https://github.com/vsislab/robotamer4qmini/blob/main/env/tasks/birl_task.py"
    click NullTask "https://github.com/vsislab/robotamer4qmini/blob/main/env/tasks/null_task.py"
    click DelayDeque "https://github.com/vsislab/robotamer4qmini/blob/main/env/utils/delay_torch_deque.py"
    click Helpers "https://github.com/vsislab/robotamer4qmini/blob/main/env/utils/helpers.py"
    click Math "https://github.com/vsislab/robotamer4qmini/blob/main/env/utils/math.py"
    click PhaseMod "https://github.com/vsislab/robotamer4qmini/blob/main/env/utils/phase_modulator.py"
    click Terrain "https://github.com/vsislab/robotamer4qmini/blob/main/env/utils/terrain.py"
    click CFG_Base "https://github.com/vsislab/robotamer4qmini/blob/main/config/Base.py"
    click CFG_BIRL "https://github.com/vsislab/robotamer4qmini/blob/main/config/BIRL.py"
    click PPO "https://github.com/vsislab/robotamer4qmini/blob/main/rl/alg/ppo.py"
    click Storage "https://github.com/vsislab/robotamer4qmini/blob/main/rl/storage/rollout_storage.py"
    click Common "https://github.com/vsislab/robotamer4qmini/blob/main/rl/module/common.py"
    click Cont "https://github.com/vsislab/robotamer4qmini/blob/main/rl/module/continuous.py"
    click Disc "https://github.com/vsislab/robotamer4qmini/blob/main/rl/module/discrete.py"
    click SimplePolicy "https://github.com/vsislab/robotamer4qmini/blob/main/model/simple_policy.py"
    click RNN "https://github.com/vsislab/robotamer4qmini/blob/main/model/rnn.py"
    click TCN "https://github.com/vsislab/robotamer4qmini/blob/main/model/tcn.py"
    click ModelUtils "https://github.com/vsislab/robotamer4qmini/blob/main/model/utils.py"
    click Train "https://github.com/vsislab/robotamer4qmini/blob/main/train.py"
    click Play "https://github.com/vsislab/robotamer4qmini/blob/main/play.py"
    click Export "https://github.com/vsislab/robotamer4qmini/blob/main/export_pt2onnx.py"
    click TuneURDF "https://github.com/vsislab/robotamer4qmini/blob/main/tune_urdf.py"
    click TunePID "https://github.com/vsislab/robotamer4qmini/blob/main/tune_pid.py"
    click ExpQ2 "https://github.com/vsislab/robotamer4qmini/tree/main/experiments/q2"
    click ExpPID "https://github.com/vsislab/robotamer4qmini/tree/main/experiments/tune_pid"
    click ExpURDF "https://github.com/vsislab/robotamer4qmini/tree/main/experiments/tune_urdf"
    click UtilsMain "https://github.com/vsislab/robotamer4qmini/tree/main/utils/"
    click RLUtils "https://github.com/vsislab/robotamer4qmini/blob/main/rl/utils/utils.py"

    %% Styles
    classDef external fill:#ffcccc,stroke:#cc0000;
    classDef config fill:#ccffcc,stroke:#009900;
    classDef script fill:#cce5ff,stroke:#004085;
    classDef core fill:#fff3cd,stroke:#856404;
    classDef store fill:#e2e3e5,stroke:#6c757d;
    classDef util fill:#d1ecf1,stroke:#0c5460;
```
