# pollen-robotics/microduck_rl architecture

> Architecture diagram of the GitHub repository pollen-robotics/microduck_rl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pollen-robotics/microduck_rl
- Repository: https://github.com/pollen-robotics/microduck_rl

## Overview

Microduck RL trains MuJoCo policies for a small biped, exports them for deployment, and supports publishing plus sim-to-real tooling. The main workflow is task configuration and robot/actuator simulation → PPO training → ONNX export → policy packaging. README-described task families are grouped by behavior rather than enumerated individually. The source index supports several implementation links; unsampled wiring is omitted or marked unverified.

Main entry points: [`src/mjlab_microduck/train_cli.py`](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/train_cli.py) (PPO training), [`src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py`](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py) (Locomotion tasks), [`src/mjlab_microduck/tasks/mdp.py`](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/mdp.py) (Rewards and MDP), [`src/mjlab_microduck/sim/body_server.py`](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/sim/body_server.py) (MuJoCo body server)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Training and tasks"]
  node_train["PPO training<br/>[train_cli.py]"]
  node_registry["Task registry<br/>[__init__.py]"]
  node_locomotion["Locomotion tasks"]
  node_recovery["Recovery tasks"]
  node_sitstand["Sit-stand task"]
  node_special["Tricks and rollers"]
  node_mdp["Rewards and MDP<br/>[mdp.py]"]
  node_symmetry["Symmetry augmentation<br/>[symmetry.py]"]
  node_distill["Policy distillation<br/>[distill.py]"]
end

subgraph group_simulation["Robot simulation"]
  node_robotcfg["Robot and actuator config"]
  node_actuator["BAM friction model<br/>[friction_dr_bam.py]"]
  node_backlash["Backlash models<br/>[backlash.py]"]
end

subgraph group_delivery["Policy delivery"]
  node_trainhook["HF job dispatch<br/>[train_hook.py]"]
  node_hfjobs["HF job submission<br/>[hf_jobs.py]"]
  node_export["ONNX export<br/>[export.py]"]
  node_publish["Policy publishing<br/>[cli.py]"]
  node_manifest["Policy manifest<br/>[manifest.py]"]
end

subgraph group_simtools["Simulation tools"]
  node_bodyserver["MuJoCo body server<br/>[body_server.py]"]
  node_camera["Camera simulation<br/>[camera.py]"]
  node_tof["ToF simulation<br/>[tof.py]"]
  node_testbench["Testbench environment"]
end

node_engineer(("Engineer"))
node_runtime(("Robot runtime"))

node_engineer -->|"starts training"| node_train
node_train -.->|"selects task"| node_registry
node_registry -.->|"registers tasks"| node_locomotion
node_train -->|"trains policy"| node_locomotion
node_train -->|"trains policy"| node_recovery
node_train -->|"trains policy"| node_sitstand
node_train -->|"trains policy"| node_special
node_locomotion -->|"uses terms"| node_mdp
node_locomotion -->|"configures"| node_symmetry
node_recovery -->|"uses terms"| node_mdp
node_sitstand -->|"uses terms"| node_mdp
node_locomotion -->|"configures robot"| node_robotcfg
node_robotcfg -->|"configures actuator"| node_actuator
node_mdp -.->|"randomizes friction"| node_actuator
node_train -.->|"can distill"| node_distill
node_locomotion -->|"has variants"| node_backlash
node_engineer -->|"requests HF run"| node_trainhook
node_trainhook -->|"dispatches"| node_hfjobs
node_train -->|"exports checkpoint"| node_export
node_export -->|"provides ONNX"| node_publish
node_publish -->|"writes manifest"| node_manifest
node_runtime -->|"validates policy"| node_manifest
node_runtime -->|"runs ONNX"| node_export
node_bodyserver -->|"streams frames"| node_camera
node_bodyserver -->|"uses sensor"| node_tof
node_bodyserver -->|"serves over TCP"| node_runtime
node_testbench -->|"uses constants"| node_robotcfg

click node_train "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/train_cli.py"
click node_registry "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/__init__.py"
click node_locomotion "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py"
click node_recovery "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/microduck_standup_env_cfg.py"
click node_sitstand "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/microduck_sitstand_env_cfg.py"
click node_special "https://github.com/pollen-robotics/microduck_rl/tree/develop/src/mjlab_microduck/tasks"
click node_mdp "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/mdp.py"
click node_symmetry "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/symmetry.py"
click node_distill "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/distill.py"
click node_robotcfg "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/robot/microduck_constants.py"
click node_actuator "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/actuator/friction_dr_bam.py"
click node_backlash "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/backlash.py"
click node_trainhook "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/train_hook.py"
click node_hfjobs "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/hf_jobs.py"
click node_export "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/export.py"
click node_publish "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/publish/cli.py"
click node_manifest "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/publish/manifest.py"
click node_bodyserver "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/sim/body_server.py"
click node_camera "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/sim/camera.py"
click node_tof "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/sim/tof.py"
click node_testbench "https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/testbench_env_cfg.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train,node_registry,node_locomotion,node_recovery,node_sitstand,node_special,node_mdp,node_symmetry,node_distill toneBlue
class node_robotcfg,node_actuator,node_backlash toneAmber
class node_trainhook,node_hfjobs,node_export,node_publish,node_manifest toneMint
class node_bodyserver,node_camera,node_tof,node_testbench toneRose
class node_engineer,node_runtime toneIndigo
```

## Components

### Training and tasks

- [PPO training](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/train_cli.py): `src/mjlab_microduck/train_cli.py`
- [Task registry](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/__init__.py): `src/mjlab_microduck/tasks/__init__.py`
- [Locomotion tasks](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py): `src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py`
- [Recovery tasks](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/microduck_standup_env_cfg.py): `src/mjlab_microduck/tasks/microduck_standup_env_cfg.py`
- [Sit-stand task](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/microduck_sitstand_env_cfg.py): `src/mjlab_microduck/tasks/microduck_sitstand_env_cfg.py`
- [Tricks and rollers](https://github.com/pollen-robotics/microduck_rl/tree/develop/src/mjlab_microduck/tasks): `src/mjlab_microduck/tasks`
- [Rewards and MDP](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/mdp.py): `src/mjlab_microduck/tasks/mdp.py`
- [Symmetry augmentation](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/symmetry.py): `src/mjlab_microduck/tasks/symmetry.py`
- [Policy distillation](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/distill.py): `src/mjlab_microduck/tasks/distill.py`

### Robot simulation

- [Robot and actuator config](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/robot/microduck_constants.py): `src/mjlab_microduck/robot/microduck_constants.py`
- [BAM friction model](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/actuator/friction_dr_bam.py): `src/mjlab_microduck/actuator/friction_dr_bam.py`
- [Backlash models](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/backlash.py): `src/mjlab_microduck/tasks/backlash.py`

### Policy delivery

- [HF job dispatch](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/train_hook.py): `src/mjlab_microduck/train_hook.py`
- [HF job submission](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/hf_jobs.py): `src/mjlab_microduck/hf_jobs.py`
- [ONNX export](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/export.py): `src/mjlab_microduck/export.py`
- [Policy publishing](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/publish/cli.py): `src/mjlab_microduck/publish/cli.py`
- [Policy manifest](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/publish/manifest.py): `src/mjlab_microduck/publish/manifest.py`

### Simulation tools

- [MuJoCo body server](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/sim/body_server.py): `src/mjlab_microduck/sim/body_server.py`
- [Camera simulation](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/sim/camera.py): `src/mjlab_microduck/sim/camera.py`
- [ToF simulation](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/sim/tof.py): `src/mjlab_microduck/sim/tof.py`
- [Testbench environment](https://github.com/pollen-robotics/microduck_rl/blob/develop/src/mjlab_microduck/tasks/testbench_env_cfg.py): `src/mjlab_microduck/tasks/testbench_env_cfg.py`

### Other components

- **Engineer**
- **Robot runtime**

## Connections

- Engineer → PPO training: starts training (evidence: `README.md`)
- PPO training → Task registry: selects task
- Task registry → Locomotion tasks: registers tasks
- PPO training → Locomotion tasks: trains policy (evidence: `README.md`)
- PPO training → Recovery tasks: trains policy (evidence: `README.md`)
- PPO training → Sit-stand task: trains policy (evidence: `README.md`)
- PPO training → Tricks and rollers: trains policy (evidence: `README.md`)
- Locomotion tasks → Rewards and MDP: uses terms (evidence: `src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py`)
- Locomotion tasks → Symmetry augmentation: configures (evidence: `src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py`)
- Recovery tasks → Rewards and MDP: uses terms (evidence: `src/mjlab_microduck/tasks/microduck_standup_env_cfg.py`)
- Sit-stand task → Rewards and MDP: uses terms (evidence: `src/mjlab_microduck/tasks/microduck_sitstand_env_cfg.py`)
- Locomotion tasks → Robot and actuator config: configures robot (evidence: `src/mjlab_microduck/tasks/microduck_velocity_env_cfg.py`)
- Robot and actuator config → BAM friction model: configures actuator (evidence: `src/mjlab_microduck/robot/microduck_constants.py`)
- Rewards and MDP → BAM friction model: randomizes friction (evidence: `src/mjlab_microduck/tasks/mdp.py`)
- PPO training → Policy distillation: can distill
- Locomotion tasks → Backlash models: has variants (evidence: `README.md`)
- Engineer → HF job dispatch: requests HF run (evidence: `README.md`)
- HF job dispatch → HF job submission: dispatches (evidence: `src/mjlab_microduck/train_hook.py`)
- PPO training → ONNX export: exports checkpoint (evidence: `README.md`)
- ONNX export → Policy publishing: provides ONNX (evidence: `README.md`)
- Policy publishing → Policy manifest: writes manifest (evidence: `src/mjlab_microduck/publish/cli.py`)
- Robot runtime → Policy manifest: validates policy (evidence: `src/mjlab_microduck/publish/manifest.py`)
- Robot runtime → ONNX export: runs ONNX (evidence: `README.md`)
- MuJoCo body server → Camera simulation: streams frames (evidence: `src/mjlab_microduck/sim/body_server.py`)
- MuJoCo body server → ToF simulation: uses sensor (evidence: `src/mjlab_microduck/sim/body_server.py`)
- MuJoCo body server → Robot runtime: serves over TCP (evidence: `src/mjlab_microduck/sim/body_server.py`)
- Testbench environment → Robot and actuator config: uses constants (evidence: `src/mjlab_microduck/tasks/testbench_env_cfg.py`)