# zecice/so101-pingpong-balance architecture

> Architecture diagram of the GitHub repository zecice/so101-pingpong-balance, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zecice/so101-pingpong-balance
- Repository: https://github.com/zecice/so101-pingpong-balance

## Overview

This project balances a ping-pong ball on an SO-101 paddle, training a PPO controller in MuJoCo and running that controller on a real arm. The main runtime path is wrist-camera image → orange-blob detection → ball-position estimation → filtered velocity → controller → joint commands; simulation supplies training and evaluation, while hardware setup and safety utilities support deployment. The README documents the end-to-end workflow; source excerpts and the SOURCE INDEX support the internal links shown. Some unsampled implementation details are omitted rather than inferred.

Main entry points: [`hardware/run_real.py`](https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/run_real.py) (Real-arm loop), [`perception/estimator.py`](https://github.com/zecice/so101-pingpong-balance/blob/main/perception/estimator.py) (Ball estimator), [`sim/policies.py`](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/policies.py) (Controller policy), [`sim/env.py`](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/env.py) (Balance environment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_simulation["Simulation and control"]
  node_velocity["Filtered velocity<br/>[env.py]"]
  node_controller["Controller policy<br/>[policies.py]"]
  node_ppo["PPO actor-critic<br/>[ppo.py]"]
  node_balance_env["Balance environment<br/>[env.py]"]
  node_vector_env["Parallel environments<br/>[vec_env.py]"]
  node_ram_check["RAM guard<br/>[ram.py]"]
  node_training["Controller training"]
  node_evaluation["Policy evaluation<br/>[eval_policy.py]"]
  node_viewer["Simulation viewer<br/>[view_controller.py]"]
end

subgraph group_perception["Ball perception"]
  node_detector["Orange detector<br/>[detector.py]"]
  node_estimator["Ball estimator<br/>[estimator.py]"]
end

subgraph group_hardware["Real-arm operation"]
  node_camera["Wrist camera<br/>[camera.py]"]
  node_real_loop["Real-arm loop<br/>[run_real.py]"]
  node_pose_guide["Home pose setup<br/>[pose_guide.py]"]
  node_latency["Latency measurement<br/>[measure_latency.py]"]
  node_torque["Torque release<br/>[torque_off.py]"]
end

subgraph group_resources["Models and weights"]
  node_sim_assets["Arm and scene model<br/>[scene_balance.xml]"]
  node_controller_weights["Controller weights<br/>[controller.pt]"]
  node_estimator_weights["Estimator weights<br/>[estimator.pt]"]
end

node_arm(("SO-101 arm"))
node_ball(("Ping-pong ball"))

node_camera -->|"provides images"| node_detector
node_detector -->|"provides detections"| node_estimator
node_estimator -->|"provides positions"| node_velocity
node_velocity -->|"provides state"| node_controller
node_controller -->|"supplies actions"| node_real_loop
node_real_loop -->|"commands joints"| node_arm
node_ball -->|"appears in image"| node_detector
node_training -->|"collects rollouts"| node_vector_env
node_training -->|"updates policy"| node_ppo
node_vector_env -->|"steps environments"| node_balance_env
node_vector_env -->|"checks memory"| node_ram_check
node_balance_env -.->|"uses scene model"| node_sim_assets
node_evaluation -->|"evaluates in"| node_balance_env
node_evaluation -->|"queries policy"| node_controller
node_viewer -->|"steps environment"| node_balance_env
node_viewer -->|"queries policy"| node_controller
node_real_loop -->|"opens camera"| node_camera
node_real_loop -->|"detects ball"| node_detector
node_real_loop -->|"estimates position"| node_estimator
node_real_loop -->|"filters velocity"| node_velocity
node_real_loop -->|"loads weights"| node_controller_weights
node_real_loop -->|"loads weights"| node_estimator_weights
node_training -->|"produces checkpoint"| node_controller_weights
node_estimator -->|"trains weights"| node_estimator_weights
node_estimator -->|"collects simulated data"| node_balance_env
node_estimator -->|"detects training frames"| node_detector
node_estimator -->|"uses balancing policy"| node_controller
node_pose_guide -->|"guides pose setup"| node_arm
node_real_loop -->|"uses saved home pose"| node_pose_guide
node_latency -->|"measures camera delay"| node_camera
node_torque -->|"releases torque"| node_arm

click node_camera "https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/camera.py"
click node_detector "https://github.com/zecice/so101-pingpong-balance/blob/main/perception/detector.py"
click node_estimator "https://github.com/zecice/so101-pingpong-balance/blob/main/perception/estimator.py"
click node_velocity "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/env.py"
click node_controller "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/policies.py"
click node_ppo "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/ppo.py"
click node_balance_env "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/env.py"
click node_vector_env "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/vec_env.py"
click node_ram_check "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/ram.py"
click node_training "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/train_controller.py"
click node_evaluation "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/eval_policy.py"
click node_viewer "https://github.com/zecice/so101-pingpong-balance/blob/main/sim/view_controller.py"
click node_real_loop "https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/run_real.py"
click node_pose_guide "https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/pose_guide.py"
click node_latency "https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/measure_latency.py"
click node_torque "https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/torque_off.py"
click node_sim_assets "https://github.com/zecice/so101-pingpong-balance/blob/main/assets/so101/scene_balance.xml"
click node_controller_weights "https://github.com/zecice/so101-pingpong-balance/blob/main/weights/controller.pt"
click node_estimator_weights "https://github.com/zecice/so101-pingpong-balance/blob/main/weights/estimator.pt"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_velocity,node_controller,node_ppo,node_balance_env,node_vector_env,node_ram_check,node_training,node_evaluation,node_viewer toneBlue
class node_detector,node_estimator toneAmber
class node_camera,node_real_loop,node_pose_guide,node_latency,node_torque toneMint
class node_sim_assets,node_controller_weights,node_estimator_weights toneRose
class node_arm,node_ball toneIndigo
```

## Components

### Simulation and control

- [Filtered velocity](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/env.py): `sim/env.py`
- [Controller policy](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/policies.py): `sim/policies.py`
- [PPO actor-critic](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/ppo.py): `sim/ppo.py`
- [Balance environment](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/env.py): `sim/env.py`
- [Parallel environments](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/vec_env.py): `sim/vec_env.py`
- [RAM guard](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/ram.py): `sim/ram.py`
- [Controller training](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/train_controller.py): `sim/train_controller.py`
- [Policy evaluation](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/eval_policy.py): `sim/eval_policy.py`
- [Simulation viewer](https://github.com/zecice/so101-pingpong-balance/blob/main/sim/view_controller.py): `sim/view_controller.py`

### Ball perception

- [Orange detector](https://github.com/zecice/so101-pingpong-balance/blob/main/perception/detector.py): `perception/detector.py`
- [Ball estimator](https://github.com/zecice/so101-pingpong-balance/blob/main/perception/estimator.py): `perception/estimator.py`

### Real-arm operation

- [Wrist camera](https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/camera.py): `hardware/camera.py`
- [Real-arm loop](https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/run_real.py): `hardware/run_real.py`
- [Home pose setup](https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/pose_guide.py): `hardware/pose_guide.py`
- [Latency measurement](https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/measure_latency.py): `hardware/measure_latency.py`
- [Torque release](https://github.com/zecice/so101-pingpong-balance/blob/main/hardware/torque_off.py): `hardware/torque_off.py`

### Models and weights

- [Arm and scene model](https://github.com/zecice/so101-pingpong-balance/blob/main/assets/so101/scene_balance.xml): `assets/so101/scene_balance.xml`
- [Controller weights](https://github.com/zecice/so101-pingpong-balance/blob/main/weights/controller.pt): `weights/controller.pt`
- [Estimator weights](https://github.com/zecice/so101-pingpong-balance/blob/main/weights/estimator.pt): `weights/estimator.pt`

### Other components

- **SO-101 arm**
- **Ping-pong ball**

## Connections

- Wrist camera → Orange detector: provides images (evidence: `README.md`)
- Orange detector → Ball estimator: provides detections (evidence: `README.md`)
- Ball estimator → Filtered velocity: provides positions (evidence: `README.md`)
- Filtered velocity → Controller policy: provides state (evidence: `README.md`)
- Controller policy → Real-arm loop: supplies actions (evidence: `README.md`)
- Real-arm loop → SO-101 arm: commands joints (evidence: `hardware/run_real.py`)
- Ping-pong ball → Orange detector: appears in image (evidence: `README.md`)
- Controller training → Parallel environments: collects rollouts (evidence: `sim/train_controller.py`)
- Controller training → PPO actor-critic: updates policy (evidence: `sim/train_controller.py`)
- Parallel environments → Balance environment: steps environments (evidence: `sim/vec_env.py`)
- Parallel environments → RAM guard: checks memory (evidence: `sim/vec_env.py`)
- Balance environment → Arm and scene model: uses scene model
- Policy evaluation → Balance environment: evaluates in (evidence: `sim/eval_policy.py`)
- Policy evaluation → Controller policy: queries policy (evidence: `sim/eval_policy.py`)
- Simulation viewer → Balance environment: steps environment (evidence: `sim/view_controller.py`)
- Simulation viewer → Controller policy: queries policy (evidence: `sim/view_controller.py`)
- Real-arm loop → Wrist camera: opens camera (evidence: `hardware/run_real.py`)
- Real-arm loop → Orange detector: detects ball (evidence: `hardware/run_real.py`)
- Real-arm loop → Ball estimator: estimates position (evidence: `hardware/run_real.py`)
- Real-arm loop → Filtered velocity: filters velocity (evidence: `README.md`)
- Real-arm loop → Controller weights: loads weights (evidence: `README.md`)
- Real-arm loop → Estimator weights: loads weights (evidence: `README.md`)
- Controller training → Controller weights: produces checkpoint (evidence: `README.md`)
- Ball estimator → Estimator weights: trains weights (evidence: `README.md`)
- Ball estimator → Balance environment: collects simulated data (evidence: `perception/estimator.py`)
- Ball estimator → Orange detector: detects training frames (evidence: `perception/estimator.py`)
- Ball estimator → Controller policy: uses balancing policy (evidence: `perception/estimator.py`)
- Home pose setup → SO-101 arm: guides pose setup (evidence: `hardware/pose_guide.py`)
- Real-arm loop → Home pose setup: uses saved home pose (evidence: `README.md`)
- Latency measurement → Wrist camera: measures camera delay (evidence: `README.md`)
- Torque release → SO-101 arm: releases torque (evidence: `README.md`)
