How antonilo/rl_locomotion works
This repository trains and evaluates simulated A1 locomotion policies using Raisim, with reinforcement learning guided by a supplied privileged-policy expert and a separate DAgger workflow to distill a blind policy. Training runners create vectorized environments, collect observations and rewards, and optimize actor/critic models; evaluation and visualization scripts exercise saved policies. The graph includes the native simulation layer, learning components, and documented model/configuration support. Evidence is limited to the README and indexed excerpts; some subsystem links are therefore documented rather than inferred from unsampled code.
Main entry points: runner.py Privileged runner · RaisimGymVecEnv.py Python env wrapper · dagger.py DAgger runner · VectorizedEnvironment.hpp Vectorized env
Components 21
Simulation
- Python env wrapper
raisimGymTorch/env/RaisimGymVecEnv.py - Vectorized env
raisimGymTorch/env/VectorizedEnvironment.hpp - Environment API
raisimGymTorch/env/RaisimGymEnv.hpp - Privileged A1 task
raisimGymTorch/env/envs/rsg_a1_task/Environment.hpp - Blind A1 task
raisimGymTorch/env/envs/dagger_a1/Environment.hpp - YAML parser
raisimGymTorch/env/Yaml.cpp
Learning
- PPO optimizer
raisimGymTorch/algo/ppo/ppo.py - DAgger learner
raisimGymTorch/algo/ppo/dagger.py - Policy networks
raisimGymTorch/algo/ppo/module.py - Rollout storage
raisimGymTorch/algo/ppo/storage.py
Policy workflows
- Privileged runner
raisimGymTorch/env/envs/rsg_a1_task/runner.py - DAgger runner
raisimGymTorch/env/envs/dagger_a1/dagger.py - Policy evaluation
raisimGymTorch/env/envs/rsg_a1_task/evaluate_policy.py - Policy visualization
raisimGymTorch/env/envs/rsg_a1_task/viz_policy.py - Blind policy tools
raisimGymTorch/env/envs/dagger_a1/evaluate_policy.py
Models and results
- Run configuration saver
raisimGymTorch/helper/raisim_gym_helper.py - Base expert policy
data/base_policy/policy_22000.pt - Task configuration
raisimGymTorch/env/envs/rsg_a1_task/cfg.yaml - Evaluation result analysis
eval_scripts/compute_results.py
Other components
- Policy developer
- Raisim simulator
Connections 28
- Policy developer → Privileged runner: starts training
README.md - Privileged runner → Python env wrapper: creates environment
raisimGymTorch/env/envs/rsg_a1_task/runner.py - Privileged runner → Policy networks: builds policy
raisimGymTorch/env/envs/rsg_a1_task/runner.py - Privileged runner → Base expert policy: loads expert
raisimGymTorch/env/envs/rsg_a1_task/runner.py - Privileged runner → PPO optimizer: optimizes policy
raisimGymTorch/env/envs/rsg_a1_task/runner.py - Privileged runner → Run configuration saver: saves run config
raisimGymTorch/env/envs/rsg_a1_task/runner.py - Privileged runner → Task configuration: loads settings
raisimGymTorch/env/envs/rsg_a1_task/runner.py - PPO optimizer → Rollout storage: stores rollouts
raisimGymTorch/algo/ppo/ppo.py - DAgger runner → Python env wrapper: creates environment
raisimGymTorch/env/envs/dagger_a1/dagger.py - DAgger runner → DAgger learner: trains blind policy
raisimGymTorch/env/envs/dagger_a1/dagger.py - DAgger runner → Policy networks: builds policy
raisimGymTorch/env/envs/dagger_a1/dagger.py - DAgger learner → Rollout storage: stores demonstrations
raisimGymTorch/algo/ppo/dagger.py - Python env wrapper → Vectorized env: calls native environment
raisimGymTorch/env/RaisimGymVecEnv.py - Vectorized env → Privileged A1 task: runs task instances
- Vectorized env → Blind A1 task: runs task instances
- Privileged A1 task → Environment API: implements interface
raisimGymTorch/env/envs/rsg_a1_task/Environment.hpp - Blind A1 task → Environment API: implements interface
raisimGymTorch/env/envs/dagger_a1/Environment.hpp - Environment API → YAML parser: uses configuration
raisimGymTorch/env/RaisimGymEnv.hpp - Privileged A1 task → Raisim simulator: simulates locomotion
- Blind A1 task → Raisim simulator: simulates locomotion
- Policy developer → DAgger runner: starts distillation
README.md - Policy developer → Policy evaluation: benchmarks policy
README.md - Policy developer → Policy visualization: views policy
README.md - Policy developer → Blind policy tools: evaluates blind policy
README.md - Policy evaluation → Python env wrapper: uses environment
raisimGymTorch/env/envs/rsg_a1_task/evaluate_policy.py - Policy visualization → Python env wrapper: uses environment
raisimGymTorch/env/envs/rsg_a1_task/viz_policy.py - Blind policy tools → Python env wrapper: uses environment
raisimGymTorch/env/envs/dagger_a1/evaluate_policy.py - Policy developer → Evaluation result analysis: summarizes results
README.md