How aungkaung1928/microduck-rl works
microduck-rl trains a CPU MuJoCo balance-and-push-recovery policy, evaluates it on nominal and held-out physics, and exports a verified ONNX policy. The workflow runs from the environment contract and PD baseline through parallel rollouts, PPO, domain-randomisation experiments, common episode metrics, evaluation, and export. The README documents these project stages; sampled source shows environment construction, worker behavior, randomisation, metrics, policy loading, and ONNX conversion. Some orchestration links rely on README documentation because the relevant source excerpts are partial.
Main entry points: ppo.py PPO Trainer · export_onnx.py ONNX Export · env.py MuJoCo Environment · eval_policy.py Policy Evaluation
Components 15
Simulation Contract
- Model Configuration
common.py - MuJoCo Environment
env.py - PD Baseline
baseline.py - Process Vector Env
vec_env.py - CPU Throughput
bench.py
Policy Learning
- Training Guard
boxcheck.py - PPO Trainer
ppo.py - Domain Randomization
dr.py
Evaluation
- Episode Metrics
metrics.py - Policy Evaluation
eval_policy.py - Held-out Gap Analysis
eval_gap.py
Policy Export
- ONNX Export
export_onnx.py - ONNX Runtime
- ONNX Policy
Other components
- Experimenter
Connections 20
- Experimenter → CPU Throughput: measures
README.md - Experimenter → PD Baseline: measures
README.md - Model Configuration → MuJoCo Environment: configures
env.py - MuJoCo Environment → PD Baseline: simulates
baseline.py - MuJoCo Environment → Process Vector Env: steps in workers
vec_env.py - Experimenter → Training Guard: starts training
README.md - Training Guard → PPO Trainer: gates
ppo.py - PPO Trainer → Process Vector Env: collects rollouts
README.md - PPO Trainer → Domain Randomization: trains with
README.md - Domain Randomization → MuJoCo Environment: changes reset physics
env.py - PPO Trainer → Episode Metrics: measures episodes
README.md - Experimenter → Policy Evaluation: evaluates policy
README.md - Policy Evaluation → Episode Metrics: uses metrics
eval_policy.py - Policy Evaluation → PPO Trainer: loads policy
eval_policy.py - Held-out Gap Analysis → Policy Evaluation: compares variants
eval_gap.py - Experimenter → ONNX Export: exports policy
README.md - ONNX Export → PPO Trainer: loads checkpoint
export_onnx.py - ONNX Export → MuJoCo Environment: samples observations
export_onnx.py - ONNX Export → ONNX Runtime: verifies and times
export_onnx.py - ONNX Export → ONNX Policy: writes
README.md