# antonilo/rl_locomotion architecture

> Architecture diagram of the GitHub repository antonilo/rl_locomotion, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/antonilo/rl_locomotion
- Repository: https://github.com/antonilo/rl_locomotion

## Overview

This repository trains and evaluates simulated A1 locomotion policies using Raisim, with reinforcement learning guided by a supplied privileged-policy expert and a separate DAgger workflow to distill a blind policy. Training runners create vectorized environments, collect observations and rewards, and optimize actor/critic models; evaluation and visualization scripts exercise saved policies. The graph includes the native simulation layer, learning components, and documented model/configuration support. Evidence is limited to the README and indexed excerpts; some subsystem links are therefore documented rather than inferred from unsampled code.

Main entry points: [`raisimGymTorch/env/envs/rsg_a1_task/runner.py`](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/runner.py) (Privileged runner), [`raisimGymTorch/env/RaisimGymVecEnv.py`](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/RaisimGymVecEnv.py) (Python env wrapper), [`raisimGymTorch/env/envs/dagger_a1/dagger.py`](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/dagger_a1/dagger.py) (DAgger runner), [`raisimGymTorch/env/VectorizedEnvironment.hpp`](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/VectorizedEnvironment.hpp) (Vectorized env)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_simulation["Simulation"]
  node_vec_wrapper["Python env wrapper<br/>[RaisimGymVecEnv.py]"]
  node_native_vec["Vectorized env"]
  node_env_api["Environment API<br/>[RaisimGymEnv.hpp]"]
  node_priv_env["Privileged A1 task<br/>[Environment.hpp]"]
  node_blind_env["Blind A1 task<br/>[Environment.hpp]"]
  node_yaml["YAML parser<br/>[Yaml.cpp]"]
end

subgraph group_learning["Learning"]
  node_ppo["PPO optimizer<br/>[ppo.py]"]
  node_dagger_algo["DAgger learner<br/>[dagger.py]"]
  node_networks["Policy networks<br/>[module.py]"]
  node_rollout_store[("Rollout storage<br/>[storage.py]")]
end

subgraph group_workflows["Policy workflows"]
  node_trainer["Privileged runner<br/>[runner.py]"]
  node_dagger_runner["DAgger runner<br/>[dagger.py]"]
  node_evaluation["Policy evaluation<br/>[evaluate_policy.py]"]
  node_visualization["Policy visualization<br/>[viz_policy.py]"]
  node_blind_evaluation["Blind policy tools<br/>[evaluate_policy.py]"]
end

subgraph group_artifacts["Models and results"]
  node_config_saver["Run configuration saver"]
  node_expert_model["Base expert policy<br/>[policy_22000.pt]"]
  node_task_config["Task configuration<br/>[cfg.yaml]"]
  node_result_analysis["Evaluation result analysis<br/>[compute_results.py]"]
end

node_operator(("Policy developer"))
node_raisim["Raisim simulator"]

node_operator -->|"starts training"| node_trainer
node_trainer -->|"creates environment"| node_vec_wrapper
node_trainer -->|"builds policy"| node_networks
node_trainer -->|"loads expert"| node_expert_model
node_trainer -->|"optimizes policy"| node_ppo
node_trainer -->|"saves run config"| node_config_saver
node_trainer -->|"loads settings"| node_task_config
node_ppo -->|"stores rollouts"| node_rollout_store
node_dagger_runner -->|"creates environment"| node_vec_wrapper
node_dagger_runner -->|"trains blind policy"| node_dagger_algo
node_dagger_runner -->|"builds policy"| node_networks
node_dagger_algo -->|"stores demonstrations"| node_rollout_store
node_vec_wrapper -->|"calls native environment"| node_native_vec
node_native_vec -.->|"runs task instances"| node_priv_env
node_native_vec -.->|"runs task instances"| node_blind_env
node_priv_env -->|"implements interface"| node_env_api
node_blind_env -->|"implements interface"| node_env_api
node_env_api -->|"uses configuration"| node_yaml
node_priv_env -.->|"simulates locomotion"| node_raisim
node_blind_env -.->|"simulates locomotion"| node_raisim
node_operator -->|"starts distillation"| node_dagger_runner
node_operator -->|"benchmarks policy"| node_evaluation
node_operator -->|"views policy"| node_visualization
node_operator -->|"evaluates blind policy"| node_blind_evaluation
node_evaluation -->|"uses environment"| node_vec_wrapper
node_visualization -->|"uses environment"| node_vec_wrapper
node_blind_evaluation -->|"uses environment"| node_vec_wrapper
node_operator -->|"summarizes results"| node_result_analysis

click node_trainer "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/runner.py"
click node_dagger_runner "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/dagger_a1/dagger.py"
click node_evaluation "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/evaluate_policy.py"
click node_visualization "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/viz_policy.py"
click node_blind_evaluation "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/dagger_a1/evaluate_policy.py"
click node_vec_wrapper "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/RaisimGymVecEnv.py"
click node_native_vec "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/VectorizedEnvironment.hpp"
click node_env_api "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/RaisimGymEnv.hpp"
click node_priv_env "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/Environment.hpp"
click node_blind_env "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/dagger_a1/Environment.hpp"
click node_yaml "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/Yaml.cpp"
click node_ppo "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/ppo.py"
click node_dagger_algo "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/dagger.py"
click node_networks "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/module.py"
click node_rollout_store "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/storage.py"
click node_config_saver "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/helper/raisim_gym_helper.py"
click node_expert_model "https://github.com/antonilo/rl_locomotion/blob/master/data/base_policy/policy_22000.pt"
click node_task_config "https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/cfg.yaml"
click node_result_analysis "https://github.com/antonilo/rl_locomotion/blob/master/eval_scripts/compute_results.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_vec_wrapper,node_native_vec,node_env_api,node_priv_env,node_blind_env,node_yaml toneBlue
class node_ppo,node_dagger_algo,node_networks,node_rollout_store toneAmber
class node_trainer,node_dagger_runner,node_evaluation,node_visualization,node_blind_evaluation toneMint
class node_config_saver,node_expert_model,node_task_config,node_result_analysis toneRose
class node_operator,node_raisim toneIndigo
```

## Components

### Simulation

- [Python env wrapper](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/RaisimGymVecEnv.py): `raisimGymTorch/env/RaisimGymVecEnv.py`
- [Vectorized env](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/VectorizedEnvironment.hpp): `raisimGymTorch/env/VectorizedEnvironment.hpp`
- [Environment API](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/RaisimGymEnv.hpp): `raisimGymTorch/env/RaisimGymEnv.hpp`
- [Privileged A1 task](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/Environment.hpp): `raisimGymTorch/env/envs/rsg_a1_task/Environment.hpp`
- [Blind A1 task](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/dagger_a1/Environment.hpp): `raisimGymTorch/env/envs/dagger_a1/Environment.hpp`
- [YAML parser](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/Yaml.cpp): `raisimGymTorch/env/Yaml.cpp`

### Learning

- [PPO optimizer](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/ppo.py): `raisimGymTorch/algo/ppo/ppo.py`
- [DAgger learner](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/dagger.py): `raisimGymTorch/algo/ppo/dagger.py`
- [Policy networks](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/module.py): `raisimGymTorch/algo/ppo/module.py`
- [Rollout storage](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/algo/ppo/storage.py): `raisimGymTorch/algo/ppo/storage.py`

### Policy workflows

- [Privileged runner](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/runner.py): `raisimGymTorch/env/envs/rsg_a1_task/runner.py`
- [DAgger runner](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/dagger_a1/dagger.py): `raisimGymTorch/env/envs/dagger_a1/dagger.py`
- [Policy evaluation](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/evaluate_policy.py): `raisimGymTorch/env/envs/rsg_a1_task/evaluate_policy.py`
- [Policy visualization](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/viz_policy.py): `raisimGymTorch/env/envs/rsg_a1_task/viz_policy.py`
- [Blind policy tools](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/dagger_a1/evaluate_policy.py): `raisimGymTorch/env/envs/dagger_a1/evaluate_policy.py`

### Models and results

- [Run configuration saver](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/helper/raisim_gym_helper.py): `raisimGymTorch/helper/raisim_gym_helper.py`
- [Base expert policy](https://github.com/antonilo/rl_locomotion/blob/master/data/base_policy/policy_22000.pt): `data/base_policy/policy_22000.pt`
- [Task configuration](https://github.com/antonilo/rl_locomotion/blob/master/raisimGymTorch/env/envs/rsg_a1_task/cfg.yaml): `raisimGymTorch/env/envs/rsg_a1_task/cfg.yaml`
- [Evaluation result analysis](https://github.com/antonilo/rl_locomotion/blob/master/eval_scripts/compute_results.py): `eval_scripts/compute_results.py`

### Other components

- **Policy developer**
- **Raisim simulator**

## Connections

- Policy developer → Privileged runner: starts training (evidence: `README.md`)
- Privileged runner → Python env wrapper: creates environment (evidence: `raisimGymTorch/env/envs/rsg_a1_task/runner.py`)
- Privileged runner → Policy networks: builds policy (evidence: `raisimGymTorch/env/envs/rsg_a1_task/runner.py`)
- Privileged runner → Base expert policy: loads expert (evidence: `raisimGymTorch/env/envs/rsg_a1_task/runner.py`)
- Privileged runner → PPO optimizer: optimizes policy (evidence: `raisimGymTorch/env/envs/rsg_a1_task/runner.py`)
- Privileged runner → Run configuration saver: saves run config (evidence: `raisimGymTorch/env/envs/rsg_a1_task/runner.py`)
- Privileged runner → Task configuration: loads settings (evidence: `raisimGymTorch/env/envs/rsg_a1_task/runner.py`)
- PPO optimizer → Rollout storage: stores rollouts (evidence: `raisimGymTorch/algo/ppo/ppo.py`)
- DAgger runner → Python env wrapper: creates environment (evidence: `raisimGymTorch/env/envs/dagger_a1/dagger.py`)
- DAgger runner → DAgger learner: trains blind policy (evidence: `raisimGymTorch/env/envs/dagger_a1/dagger.py`)
- DAgger runner → Policy networks: builds policy (evidence: `raisimGymTorch/env/envs/dagger_a1/dagger.py`)
- DAgger learner → Rollout storage: stores demonstrations (evidence: `raisimGymTorch/algo/ppo/dagger.py`)
- Python env wrapper → Vectorized env: calls native environment (evidence: `raisimGymTorch/env/RaisimGymVecEnv.py`)
- Vectorized env → Privileged A1 task: runs task instances
- Vectorized env → Blind A1 task: runs task instances
- Privileged A1 task → Environment API: implements interface (evidence: `raisimGymTorch/env/envs/rsg_a1_task/Environment.hpp`)
- Blind A1 task → Environment API: implements interface (evidence: `raisimGymTorch/env/envs/dagger_a1/Environment.hpp`)
- Environment API → YAML parser: uses configuration (evidence: `raisimGymTorch/env/RaisimGymEnv.hpp`)
- Privileged A1 task → Raisim simulator: simulates locomotion
- Blind A1 task → Raisim simulator: simulates locomotion
- Policy developer → DAgger runner: starts distillation (evidence: `README.md`)
- Policy developer → Policy evaluation: benchmarks policy (evidence: `README.md`)
- Policy developer → Policy visualization: views policy (evidence: `README.md`)
- Policy developer → Blind policy tools: evaluates blind policy (evidence: `README.md`)
- Policy evaluation → Python env wrapper: uses environment (evidence: `raisimGymTorch/env/envs/rsg_a1_task/evaluate_policy.py`)
- Policy visualization → Python env wrapper: uses environment (evidence: `raisimGymTorch/env/envs/rsg_a1_task/viz_policy.py`)
- Blind policy tools → Python env wrapper: uses environment (evidence: `raisimGymTorch/env/envs/dagger_a1/evaluate_policy.py`)
- Policy developer → Evaluation result analysis: summarizes results (evidence: `README.md`)
