# kartikbud/quadruped-locomotion-rl-env architecture

> Architecture diagram of the GitHub repository kartikbud/quadruped-locomotion-rl-env, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kartikbud/quadruped-locomotion-rl-env
- Repository: https://github.com/kartikbud/quadruped-locomotion-rl-env

## Overview

1. This repository is a reinforcement-learning locomotion project for a quadruped robot. The core loop is: simulate a quadruped environment, train a PPO policy on it, then run the trained policy to produce gait motion. The saved videos and model artifacts show that the project has already been used for both training and playback.

2. The main entry points are `train_model.py` and `run_model.py`. `train_model.py` is the training orchestrator: it likely constructs the environment, configures PPO hyperparameters, and writes checkpoints/logs. `run_model.py` is the inference/playback entry point: it loads a saved model from `saved_models/final_models/` and runs it in the robot environment for visualization.

3. `quadruped/env/` is the simulation-facing environment layer. `env.py` is the central boundary between the learning code and the robot dynamics. `env_test.py` suggests there is a dedicated sanity-check path for validating the environment independently of long training runs. This layer is where observations, rewards, action application, and episode resets are probably defined.

4. `quadruped/motion/` contains the low-level gait generation and kinematics logic. The presence of `trot.py`, `bezier_trajectory_generator.py`, and `inverse_kinematics.py` indicates a structured motion stack: gait phase/foot trajectories are generated with Bezier curves, then converted into joint targets through inverse kinematics. This is the non-learning motion subsystem that either serves as a baseline controller or provides motion shaping for the RL environment.

5. The robot itself is described separately from the learning code in `robot/`. The `spot.urdf`, `alt_spot.urdf`, and matching XML files define the quadruped model, while the STL meshes provide the visual/geometry assets for the body, legs, feet, and chassis parts. These files are the physical model boundary consumed by the simulator and/or viewer.

6. `quadruped/training/callbacks.py` holds training-time hooks, likely for checkpointing, evaluation, or custom TensorBoard metrics. This is the place where training behavior is extended without changing the main training script, which keeps the model loop and logging concerns separated.

7. The project is built around Stable-Baselines3 PPO, based on the saved artifact layout (`PPO_*.zip`, `policy.pth`, `policy.onnx`, `system_info.txt`) and TensorBoard event files under `logs/` and `old_stuff/tensorboard_logs/`. Architecture-wise, PPO is the learning engine, the environment is the simulator interface, and TensorBoard is the primary observability channel during training.

8. `saved_models/final_models/` is the production-style model store. It contains multiple PPO checkpoints and at least one expanded directory export with both PyTorch and ONNX policy files, which suggests the project supports both native SB3 loading and model export for inference or deployment. `old_stuff/` appears to be archived experiments and earlier checkpoints, kept out of the main path.

9. `utils/pth_to_onnx.py` is a conversion tool for exporting trained policy weights to ONNX. That creates a secondary artifact flow: training produces PyTorch/SB3 checkpoints, and the utility converts a policy into a format that can be consumed by other runtimes or deployment targets. `utils/viewer.py` is the visualization boundary for watching the robot execute a policy.

10. `assets/` contains short videos of baseline and learned behaviors (`baseline_trot.mov`, `no_policy.mov`, `with_policy.mov`). These are not part of runtime execution, but they document the output of the system and help compare controller behavior qualitatively.

11. The repo structure separates concerns cleanly: motion primitives and kinematics, environment/simulation, RL training, inference, and exported artifacts. For an architecture diagram, the key flow is: robot description and motion primitives feed the environment; the environment feeds PPO training; training writes logs/checkpoints; inference loads the saved policy and drives the robot viewer/simulator.

12. In terms of runtime/tooling, the most important external dependency is Python with a reinforcement-learning stack centered on Stable-Baselines3 and TensorBoard. The repository is organized as an experimentation-to-deployment pipeline rather than as a general application: model training artifacts, logs, exported policies, and video evidence are all first-class outputs.

Main entry points: [`quadruped/env/env.py`](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/env/env.py) (Env core), [`train_model.py`](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/train_model.py) (Train model), [`run_model.py`](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/run_model.py) (Run model), [`saved_models/final_models`](https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/saved_models/final_models) (Saved models)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_group_runtime["Runtime"]
  node_train_model_py["Train model<br/>entrypoint<br/>[train_model.py]"]
  node_run_model_py["Run model<br/>entrypoint<br/>[run_model.py]"]
  node_viewer_py["Viewer<br/>visualizer<br/>[viewer.py]"]
  node_onnx_tool["ONNX export<br/>conversion tool<br/>[pth_to_onnx.py]"]
end

subgraph group_group_learning["Learning"]
  node_callbacks_py["Callbacks<br/>training hooks<br/>[callbacks.py]"]
  node_ppo_engine(("PPO<br/>rl algorithm"))
end

subgraph group_group_env["Environment"]
  node_env_py["Env core<br/>simulation env<br/>[env.py]"]
  node_env_test_py["Env test<br/>sanity check<br/>[env_test.py]"]
end

subgraph group_group_motion["Motion"]
  node_bezier_py["Bezier traj<br/>trajectory generator"]
  node_ik_py["Inverse kinematics"]
  node_trot_py["Trot gait<br/>gait logic<br/>[trot.py]"]
end

subgraph group_group_robot["Robot model"]
  node_robot_urdf["Robot URDF<br/>robot description<br/>[spot.urdf]"]
  node_robot_alt_urdf["Alt URDF<br/>robot description<br/>[alt_spot.urdf]"]
  node_robot_xml["Robot XML<br/>robot config<br/>[spot.xml]"]
  node_robot_meshes[("Meshes<br/>assets")]
end

subgraph group_group_artifacts["Artifacts"]
  node_saved_models[("Saved models<br/>model store")]
  node_logs_dir[("TensorBoard logs<br/>training logs")]
  node_old_stuff_dir[("Archive<br/>legacy artifacts")]
  node_assets_dir[("Videos<br/>outputs")]
end

node_train_model_py -->|"builds"| node_env_py
node_train_model_py -->|"optimizes"| node_ppo_engine
node_train_model_py -->|"hooks"| node_callbacks_py
node_env_py -->|"loads"| node_robot_urdf
node_env_py -->|"loads"| node_robot_xml
node_env_py -->|"renders"| node_robot_meshes
node_env_py -.->|"commands"| node_trot_py
node_trot_py -->|"generates"| node_bezier_py
node_bezier_py -->|"targets"| node_ik_py
node_ik_py -->|"applies"| node_env_py
node_env_test_py -.->|"validates"| node_env_py
node_run_model_py -->|"loads"| node_saved_models
node_run_model_py -->|"executes"| node_ppo_engine
node_run_model_py -->|"displays"| node_viewer_py
node_viewer_py -->|"steps"| node_env_py
node_saved_models -.->|"exports"| node_onnx_tool
node_onnx_tool -->|"writes"| node_saved_models
node_train_model_py -->|"writes"| node_logs_dir
node_train_model_py -->|"persists"| node_saved_models
node_run_model_py -.->|"produces"| node_assets_dir
node_train_model_py -.->|"preceded by"| node_old_stuff_dir

click node_train_model_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/train_model.py"
click node_run_model_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/run_model.py"
click node_viewer_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/utils/viewer.py"
click node_callbacks_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/training/callbacks.py"
click node_env_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/env/env.py"
click node_env_test_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/env/env_test.py"
click node_bezier_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/motion/bezier_trajectory_generator.py"
click node_ik_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/motion/inverse_kinematics.py"
click node_trot_py "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/motion/trot.py"
click node_robot_urdf "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/robot/spot.urdf"
click node_robot_alt_urdf "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/robot/alt_spot.urdf"
click node_robot_xml "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/robot/spot.xml"
click node_robot_meshes "https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/robot/meshes"
click node_saved_models "https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/saved_models/final_models"
click node_logs_dir "https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/logs"
click node_old_stuff_dir "https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/old_stuff"
click node_assets_dir "https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/assets"
click node_onnx_tool "https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/utils/pth_to_onnx.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_model_py,node_run_model_py,node_viewer_py,node_onnx_tool toneBlue
class node_callbacks_py,node_ppo_engine toneAmber
class node_env_py,node_env_test_py toneMint
class node_bezier_py,node_ik_py,node_trot_py toneRose
class node_robot_urdf,node_robot_alt_urdf,node_robot_xml,node_robot_meshes toneIndigo
class node_saved_models,node_logs_dir,node_old_stuff_dir,node_assets_dir toneTeal
```

## Components

### Runtime

Entry points for training, playback, and visualization

- [Train model](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/train_model.py): entrypoint · Orchestrates PPO training, environment setup, and checkpointing · `train_model.py`
- [Run model](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/run_model.py): entrypoint · Loads a saved policy and runs playback/inference · `run_model.py`
- [Viewer](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/utils/viewer.py): visualizer · Visualization boundary for executing a policy in simulation · `utils/viewer.py`
- [ONNX export](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/utils/pth_to_onnx.py): conversion tool · Converts trained PyTorch/SB3 policies into ONNX · `utils/pth_to_onnx.py`

### Learning

PPO training and its hooks

- [Callbacks](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/training/callbacks.py): training hooks · Custom training-time hooks for evaluation, logging, and checkpointing · `quadruped/training/callbacks.py`
- **PPO**: rl algorithm · Stable-Baselines3 PPO policy optimization and inference runtime

### Environment

Simulation boundary and environment validation

- [Env core](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/env/env.py): simulation env · Main environment boundary between RL code and robot dynamics · `quadruped/env/env.py`
- [Env test](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/env/env_test.py): sanity check · Standalone validation path for the environment layer · `quadruped/env/env_test.py`

### Motion

Gait generation and kinematics stack

- [Bezier traj](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/motion/bezier_trajectory_generator.py): trajectory generator · Generates foot trajectories for gait timing and swing motion · `quadruped/motion/bezier_trajectory_generator.py`
- [Inverse kinematics](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/motion/inverse_kinematics.py): kinematics · Converts foot targets into joint commands · `quadruped/motion/inverse_kinematics.py`
- [Trot gait](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/quadruped/motion/trot.py): gait logic · Defines trot phase logic and gait coordination · `quadruped/motion/trot.py`

### Robot model

URDF/XML description and mesh assets

- [Robot URDF](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/robot/spot.urdf): robot description · Primary quadruped model consumed by the simulator · `robot/spot.urdf`
- [Alt URDF](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/robot/alt_spot.urdf): robot description · Alternate quadruped model variant · `robot/alt_spot.urdf`
- [Robot XML](https://github.com/kartikbud/quadruped-locomotion-rl-env/blob/main/robot/spot.xml): robot config · XML-side robot model/configuration files · `robot/spot.xml`
- [Meshes](https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/robot/meshes): assets · STL geometry for body, legs, feet, and chassis · `robot/meshes`

### Artifacts

Saved models, logs, and outputs

- [Saved models](https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/saved_models/final_models): model store · Production-style policy checkpoints and exported policies · `saved_models/final_models`
- [TensorBoard logs](https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/logs): training logs · Training metrics and event files for PPO runs · `logs`
- [Archive](https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/old_stuff): legacy artifacts · Older checkpoints and logs kept as historical experiments · `old_stuff`
- [Videos](https://github.com/kartikbud/quadruped-locomotion-rl-env/tree/main/assets): outputs · Recorded behavior videos for baseline and learned policies · `assets`

## Connections

- Train model → Env core: builds · Training uses the simulation environment as the learning interface
- Train model → PPO: optimizes · Training orchestrates PPO updates and rollout collection
- Train model → Callbacks: hooks · Training extends behavior through callbacks
- Env core → Robot URDF: loads · Environment consumes the robot description
- Env core → Robot XML: loads · Environment may use XML-side robot configuration
- Env core → Meshes: renders · Visualization/simulation uses robot geometry assets
- Env core → Trot gait: commands · Environment can shape actions or rewards around gait logic
- Trot gait → Bezier traj: generates · Gait logic uses trajectory generation for foot motion
- Bezier traj → Inverse kinematics: targets · Foot trajectories are converted into joint-space commands
- Inverse kinematics → Env core: applies · Joint targets are applied back into the simulated robot
- Env test → Env core: validates · Dedicated test path checks environment behavior independently
- Run model → Saved models: loads · Playback loads a previously trained policy artifact
- Run model → PPO: executes · Inference runs the PPO policy in the environment
- Run model → Viewer: displays · Playback is visualized through the viewer boundary
- Viewer → Env core: steps · Viewer drives the environment for live execution
- Saved models → ONNX export: exports · Trained models can be converted to ONNX for other runtimes
- ONNX export → Saved models: writes · Conversion produces exported policy files alongside checkpoints
- Train model → TensorBoard logs: writes · Training writes TensorBoard event files and run logs
- Train model → Saved models: persists · Training checkpoints and final policies are stored here
- Run model → Videos: produces · Policy runs are documented as playback videos
- Train model → Archive: preceded by · Legacy experiments and checkpoints exist outside the main path
