# jmelzer244/self-righting-rl-arm architecture

> Architecture diagram of the GitHub repository jmelzer244/self-righting-rl-arm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jmelzer244/self-righting-rl-arm
- Repository: https://github.com/jmelzer244/self-righting-rl-arm

## Overview

This repository builds a sim-to-real self-righting controller: a MuJoCo arm model feeds a Gymnasium task, PPO trains a policy (optionally with randomized physics and noisy observations), and an export step packages network weights for an ESP32-S3 closed-loop controller. The README also documents the Onshape-to-MuJoCo model workflow and hardware sensing and actuation. The firmware and model-generation internals were not sampled, so those links reflect README documentation rather than inspected implementation.

Main entry points: [`selfright_env.py`](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/selfright_env.py) (Self-right task), [`scene.xml`](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/scene.xml) (MuJoCo scene), [`domain_rand.py`](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/domain_rand.py) (Randomized task), [`selfright_esp32.ino`](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/selfright_esp32.ino) (ESP32 controller)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_model["Arm model"]
  node_robot_model["Robot model<br/>[robot.xml]"]
  node_scene["MuJoCo scene<br/>[scene.xml]"]
  node_meshes["Arm meshes"]
end

subgraph group_simulation["Simulation task"]
  node_env["Self-right task<br/>[selfright_env.py]"]
  node_randomized_env["Randomized task<br/>[domain_rand.py]"]
  node_viewer["Scene viewer<br/>[view.py]"]
end

subgraph group_learning["Policy learning"]
  node_sim_trainer["PPO training<br/>[train.py]"]
  node_dr_trainer["Randomized PPO<br/>[train_dr.py]"]
  node_rollout["Rollout recording<br/>[watch.py]"]
end

subgraph group_deployment["Embedded deployment"]
  node_exporter["Policy exporter<br/>[export_policy.py]"]
  node_weights["MLP weights<br/>[policy_weights.h]"]
  node_firmware["ESP32 controller"]
end

node_designer(("Model author"))
node_onshape["Onshape export"]
node_imu["MPU6050 IMU"]
node_servos["Arm servos"]
node_trainer(("Policy trainer"))

node_designer -->|"models arm"| node_onshape
node_onshape -->|"exports model"| node_robot_model
node_robot_model -.->|"builds scene"| node_scene
node_meshes -.->|"provides geometry"| node_scene
node_scene -->|"loads model"| node_env
node_env -->|"extends task"| node_randomized_env
node_trainer -->|"starts training"| node_sim_trainer
node_sim_trainer -->|"trains with"| node_env
node_trainer -->|"starts training"| node_dr_trainer
node_dr_trainer -->|"trains with"| node_randomized_env
node_randomized_env -->|"inherits task"| node_env
node_trainer -->|"records rollout"| node_rollout
node_rollout -->|"steps environment"| node_env
node_scene -->|"loads scene"| node_viewer
node_trainer -->|"exports policy"| node_exporter
node_exporter -->|"writes weights"| node_weights
node_weights -->|"supplies policy"| node_firmware
node_imu -->|"provides readings"| node_firmware
node_firmware -->|"drives servos"| node_servos

click node_robot_model "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/robot.xml"
click node_scene "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/scene.xml"
click node_meshes "https://github.com/jmelzer244/self-righting-rl-arm/tree/main/assets"
click node_env "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/selfright_env.py"
click node_randomized_env "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/domain_rand.py"
click node_sim_trainer "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/train.py"
click node_dr_trainer "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/train_dr.py"
click node_rollout "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/watch.py"
click node_viewer "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/view.py"
click node_exporter "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/export_policy.py"
click node_weights "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/policy_weights.h"
click node_firmware "https://github.com/jmelzer244/self-righting-rl-arm/blob/main/selfright_esp32.ino"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_robot_model,node_scene,node_meshes toneBlue
class node_env,node_randomized_env,node_viewer toneAmber
class node_sim_trainer,node_dr_trainer,node_rollout toneMint
class node_exporter,node_weights,node_firmware toneRose
class node_designer,node_onshape,node_imu,node_servos,node_trainer toneIndigo
```

## Components

### Arm model

- [Robot model](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/robot.xml): `robot.xml`
- [MuJoCo scene](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/scene.xml): `scene.xml`
- [Arm meshes](https://github.com/jmelzer244/self-righting-rl-arm/tree/main/assets): `assets`

### Simulation task

- [Self-right task](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/selfright_env.py): `selfright_env.py`
- [Randomized task](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/domain_rand.py): `domain_rand.py`
- [Scene viewer](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/view.py): `view.py`

### Policy learning

- [PPO training](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/train.py): `train.py`
- [Randomized PPO](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/train_dr.py): `train_dr.py`
- [Rollout recording](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/watch.py): `watch.py`

### Embedded deployment

- [Policy exporter](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/export_policy.py): `export_policy.py`
- [MLP weights](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/policy_weights.h): `policy_weights.h`
- [ESP32 controller](https://github.com/jmelzer244/self-righting-rl-arm/blob/main/selfright_esp32.ino): `selfright_esp32.ino`

### Other components

- **Model author**
- **Onshape export**
- **MPU6050 IMU**
- **Arm servos**
- **Policy trainer**

## Connections

- Model author → Onshape export: models arm (evidence: `README.md`)
- Onshape export → Robot model: exports model (evidence: `README.md`)
- Robot model → MuJoCo scene: builds scene
- Arm meshes → MuJoCo scene: provides geometry
- MuJoCo scene → Self-right task: loads model (evidence: `selfright_env.py`)
- Self-right task → Randomized task: extends task (evidence: `domain_rand.py`)
- Policy trainer → PPO training: starts training (evidence: `README.md`)
- PPO training → Self-right task: trains with (evidence: `train.py`)
- Policy trainer → Randomized PPO: starts training (evidence: `README.md`)
- Randomized PPO → Randomized task: trains with (evidence: `train_dr.py`)
- Randomized task → Self-right task: inherits task (evidence: `domain_rand.py`)
- Policy trainer → Rollout recording: records rollout (evidence: `README.md`)
- Rollout recording → Self-right task: steps environment (evidence: `watch.py`)
- MuJoCo scene → Scene viewer: loads scene (evidence: `view.py`)
- Policy trainer → Policy exporter: exports policy (evidence: `README.md`)
- Policy exporter → MLP weights: writes weights (evidence: `export_policy.py`)
- MLP weights → ESP32 controller: supplies policy (evidence: `README.md`)
- MPU6050 IMU → ESP32 controller: provides readings (evidence: `README.md`)
- ESP32 controller → Arm servos: drives servos (evidence: `README.md`)
