How vsislab/robotamer4qmini works
Project Type and Purpose
• This is a full-stack reinforcement learning framework for bipedal/quadruped robot locomotion using NVIDIA Isaac Gym.
• It supports training (train.py), evaluation/playback (play.py), model export (export_pt2onnx.py), URDF tuning (tune_urdf.py), PID tuning (tune_pid.py), and houses experiments, assets, configs, models, and utilities.
Main Components (Blocks)
A. Environment & Simulation Layer
• Isaac Gym simulator (external service)
• env/ (gym_env_wrapper.py, legged_robot.py, tasks, utils/terrain, etc.)
• assets/urdf & meshes
B. Configuration Layer
• config/ (Base.py, BIRL.py)
• command-line overrides (train.py, play.py flags)
C. RL Algorithm Layer
• rl/alg/ppo.py
• rl/storage/rollout_storage.py
• rl/module/… (policy networks, common interfaces)
D. Model Architectures
• model/ (simple_policy.py, rnn.py, tcn.py)
• model/utils.py
E. Training & Execution Scripts
• train.py (training loop)
• play.py (evaluation loop)
• export_pt2onnx.py
• tune_urdf.py
• tune_pid.py
F. Experiment Management & Logging
• experiments/ (pretrained checkpoints, logs, onnx deploy)
• tensorboard, Excel reports
G. Utility Modules
• utils/ (common helpers, yaml, attrdict)
• env/utils (phase_modulator, math, helpers)
• rl/utils
1. train.py reads config → instantiates env wrapper → spawns Isaac Gym environments
2. At each iteration:
– env simulates robot (using URDF & meshes) → returns observations
– rl/alg/ppo policy (model/) consumes observations → outputs actions
– actions → env step → new observations + rewards
– rollout_storage logs transitions → ppo.update() → new policy weights
3. Checkpoints (policy.pt) & training metrics (tensorboard logs) → experiments/
4. play.py uses same env & policy (loads .pt) → renders/writes data/video
5. export_pt2onnx.py loads .pt → onnxruntime export → experiments/<name>/deploy
6. tune_urdf.py loads assets/urdf → viewer; tune_pid.py runs specialized tasks to optimize PID → writes Excel & logs
Architectural Patterns & Principles
• Modular separation: environment, algorithm, model architectures, configs, utilities
• Config-driven behavior: swap tasks, tuning modes, hyperparameters
• Plugin-like tasks in env/tasks for extensibility
• Clear I/O boundaries: env ↔ policy, scripts ↔ experiments folder