How kartikbud/quadruped-locomotion-rl-env works
1. This repository is a reinforcement-learning locomotion project for a quadruped robot. The core loop is: simulate a quadruped environment, train a PPO policy on it, then run the trained policy to produce gait motion. The saved videos and model artifacts show that the project has already been used for both training and playback.
2. The main entry points are train_model.py and run_model.py. train_model.py is the training orchestrator: it likely constructs the environment, configures PPO hyperparameters, and writes checkpoints/logs. run_model.py is the inference/playback entry point: it loads a saved model from saved_models/final_models/ and runs it in the robot environment for visualization.
3. quadruped/env/ is the simulation-facing environment layer. env.py is the central boundary between the learning code and the robot dynamics. env_test.py suggests there is a dedicated sanity-check path for validating the environment independently of long training runs. This layer is where observations, rewards, action application, and episode resets are probably defined.
4. quadruped/motion/ contains the low-level gait generation and kinematics logic. The presence of trot.py, bezier_trajectory_generator.py, and inverse_kinematics.py indicates a structured motion stack: gait phase/foot trajectories are generated with Bezier curves, then converted into joint targets through inverse kinematics. This is the non-learning motion subsystem that either serves as a baseline controller or provides motion shaping for the RL environment.
5. The robot itself is described separately from the learning code in robot/. The spot.urdf, alt_spot.urdf, and matching XML files define the quadruped model, while the STL meshes provide the visual/geometry assets for the body, legs, feet, and chassis parts. These files are the physical model boundary consumed by the simulator and/or viewer.
6. quadruped/training/callbacks.py holds training-time hooks, likely for checkpointing, evaluation, or custom TensorBoard metrics. This is the place where training behavior is extended without changing the main training script, which keeps the model loop and logging concerns separated.
7. The project is built around Stable-Baselines3 PPO, based on the saved artifact layout (PPO_*.zip, policy.pth, policy.onnx, system_info.txt) and TensorBoard event files under logs/ and old_stuff/tensorboard_logs/. Architecture-wise, PPO is the learning engine, the environment is the simulator interface, and TensorBoard is the primary observability channel during training.
8. saved_models/final_models/ is the production-style model store. It contains multiple PPO checkpoints and at least one expanded directory export with both PyTorch and ONNX policy files, which suggests the project supports both native SB3 loading and model export for inference or deployment. old_stuff/ appears to be archived experiments and earlier checkpoints, kept out of the main path.
9. utils/pth_to_onnx.py is a conversion tool for exporting trained policy weights to ONNX. That creates a secondary artifact flow: training produces PyTorch/SB3 checkpoints, and the utility converts a policy into a format that can be consumed by other runtimes or deployment targets. utils/viewer.py is the visualization boundary for watching the robot execute a policy.
10. assets/ contains short videos of baseline and learned behaviors (baseline_trot.mov, no_policy.mov, with_policy.mov). These are not part of runtime execution, but they document the output of the system and help compare controller behavior qualitatively.
11. The repo structure separates concerns cleanly: motion primitives and kinematics, environment/simulation, RL training, inference, and exported artifacts. For an architecture diagram, the key flow is: robot description and motion primitives feed the environment; the environment feeds PPO training; training writes logs/checkpoints; inference loads the saved policy and drives the robot viewer/simulator.
12. In terms of runtime/tooling, the most important external dependency is Python with a reinforcement-learning stack centered on Stable-Baselines3 and TensorBoard. The repository is organized as an experimentation-to-deployment pipeline rather than as a general application: model training artifacts, logs, exported policies, and video evidence are all first-class outputs.
Main entry points: env.py Env core · train_model.py Train model · run_model.py Run model · final_models Saved models
Components 19
Runtime
Entry points for training, playback, and visualization
- Train model · Orchestrates PPO training, environment setup, and checkpointing
train_model.py - Run model · Loads a saved policy and runs playback/inference
run_model.py - Viewer · Visualization boundary for executing a policy in simulation
utils/viewer.py - ONNX export · Converts trained PyTorch/SB3 policies into ONNX
utils/pth_to_onnx.py
Learning
PPO training and its hooks
- Callbacks · Custom training-time hooks for evaluation, logging, and checkpointing
quadruped/training/callbacks.py - PPO · Stable-Baselines3 PPO policy optimization and inference runtime
Environment
Simulation boundary and environment validation
Motion
Gait generation and kinematics stack
- Bezier traj · Generates foot trajectories for gait timing and swing motion
quadruped/motion/bezier_trajectory_generator.py - Inverse kinematics · Converts foot targets into joint commands
quadruped/motion/inverse_kinematics.py - Trot gait · Defines trot phase logic and gait coordination
quadruped/motion/trot.py
Robot model
URDF/XML description and mesh assets
- Robot URDF · Primary quadruped model consumed by the simulator
robot/spot.urdf - Alt URDF · Alternate quadruped model variant
robot/alt_spot.urdf - Robot XML · XML-side robot model/configuration files
robot/spot.xml - Meshes · STL geometry for body, legs, feet, and chassis
robot/meshes
Artifacts
Saved models, logs, and outputs
- Saved models · Production-style policy checkpoints and exported policies
saved_models/final_models - TensorBoard logs · Training metrics and event files for PPO runs
logs - Archive · Older checkpoints and logs kept as historical experiments
old_stuff - Videos · Recorded behavior videos for baseline and learned policies
assets
Connections 21
- Train model → Env core: builds · Training uses the simulation environment as the learning interface
- Train model → PPO: optimizes · Training orchestrates PPO updates and rollout collection
- Train model → Callbacks: hooks · Training extends behavior through callbacks
- Env core → Robot URDF: loads · Environment consumes the robot description
- Env core → Robot XML: loads · Environment may use XML-side robot configuration
- Env core → Meshes: renders · Visualization/simulation uses robot geometry assets
- Env core → Trot gait: commands · Environment can shape actions or rewards around gait logic
- Trot gait → Bezier traj: generates · Gait logic uses trajectory generation for foot motion
- Bezier traj → Inverse kinematics: targets · Foot trajectories are converted into joint-space commands
- Inverse kinematics → Env core: applies · Joint targets are applied back into the simulated robot
- Env test → Env core: validates · Dedicated test path checks environment behavior independently
- Run model → Saved models: loads · Playback loads a previously trained policy artifact
- Run model → PPO: executes · Inference runs the PPO policy in the environment
- Run model → Viewer: displays · Playback is visualized through the viewer boundary
- Viewer → Env core: steps · Viewer drives the environment for live execution
- Saved models → ONNX export: exports · Trained models can be converted to ONNX for other runtimes
- ONNX export → Saved models: writes · Conversion produces exported policy files alongside checkpoints
- Train model → TensorBoard logs: writes · Training writes TensorBoard event files and run logs
- Train model → Saved models: persists · Training checkpoints and final policies are stored here
- Run model → Videos: produces · Policy runs are documented as playback videos
- Train model → Archive: preceded by · Legacy experiments and checkpoints exist outside the main path