# shuyi-song-07/mujoco_simulate architecture

> Architecture diagram of the GitHub repository shuyi-song-07/mujoco_simulate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shuyi-song-07/mujoco_simulate
- Repository: https://github.com/shuyi-song-07/mujoco_simulate

## Overview

This repository supports a robot-learning workflow: operators control simulated Panda pick-and-place or Push-T tasks through webcam gestures, record demonstrations, validate datasets, and train an ACT baseline. The recorder coordinates simulation, task state, and asynchronous rendering/data capture; separate tools inspect trajectories and dataset integrity. The graph reflects README-described capabilities where source excerpts were unavailable; unsampled evaluation or training internals are represented only at their documented entry points.

Main entry points: [`simulation/mujoco/record_mujoco_panda.py`](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/record_mujoco_panda.py) (Panda Recorder), [`teleoperation/mediapipe/app.js`](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/app.js) (Gesture UI), [`dataset/demonstrations`](https://github.com/shuyi-song-07/mujoco_simulate/tree/main/dataset/demonstrations) (Demonstration Datasets), [`dataset/demonstrations/analyze_action_trajectory.py`](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/dataset/demonstrations/analyze_action_trajectory.py) (Trajectory Analysis)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_teleop["Gesture Teleoperation"]
  node_web_ui["Gesture UI<br/>[app.js]"]
  node_gesture_logic["Motion Mapping"]
  node_web_server["Web Proxy<br/>[server.js]"]
  node_gesture_model["Gesture Model"]
end

subgraph group_simulation["Simulation Recording"]
  node_panda_recorder["Panda Recorder"]
  node_push_task["Push-T Launcher"]
  node_panda_scene["Panda Scenes<br/>[push_t_scene.xml]"]
  node_render_workers["Render Workers<br/>[render_workers.py]"]
end

subgraph group_data["Demonstration Data"]
  node_demo_data[("Demonstration Datasets")]
  node_video_data[("Episode Videos<br/>[render_workers.py]")]
end

subgraph group_learning["Training and Evaluation"]
  node_act_training["ACT Training"]
  node_dataset_validation["Dataset Validation"]
  node_trajectory_analysis["Trajectory Analysis"]
  node_evaluation_output["Analysis Reports"]
end

node_operator(("Operator"))
node_lerobot["LeRobot"]

node_operator -->|"controls"| node_web_ui
node_web_ui -->|"recognizes gestures"| node_gesture_model
node_web_ui -->|"maps motion"| node_gesture_logic
node_web_ui -->|"sends commands"| node_web_server
node_web_server -->|"proxies API"| node_panda_recorder
node_panda_recorder -->|"serves previews"| node_web_server
node_push_task -->|"launches task"| node_panda_recorder
node_panda_recorder -->|"loads scene"| node_panda_scene
node_panda_recorder -->|"dispatches frames"| node_render_workers
node_render_workers -->|"writes episodes"| node_demo_data
node_render_workers -->|"encodes views"| node_video_data
node_act_training -->|"reads dataset"| node_demo_data
node_act_training -->|"trains with"| node_lerobot
node_dataset_validation -->|"checks dataset"| node_demo_data
node_dataset_validation -->|"writes reports"| node_evaluation_output
node_trajectory_analysis -->|"reads trajectories"| node_demo_data
node_trajectory_analysis -->|"reconstructs pose"| node_panda_scene
node_trajectory_analysis -->|"writes plots"| node_evaluation_output

click node_web_ui "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/app.js"
click node_gesture_logic "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/interaction-state.js"
click node_web_server "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/server.js"
click node_gesture_model "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/models/gesture_recognizer.task"
click node_panda_recorder "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/record_mujoco_panda.py"
click node_push_task "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/record_mujoco_push_t.py"
click node_panda_scene "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/assets/franka_emika_panda/push_t_scene.xml"
click node_render_workers "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/render_workers.py"
click node_demo_data "https://github.com/shuyi-song-07/mujoco_simulate/tree/main/dataset/demonstrations"
click node_video_data "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/render_workers.py"
click node_act_training "https://github.com/shuyi-song-07/mujoco_simulate/tree/main/training/ACT"
click node_dataset_validation "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/evaluation/validate_multicam_dataset.py"
click node_trajectory_analysis "https://github.com/shuyi-song-07/mujoco_simulate/blob/main/dataset/demonstrations/analyze_action_trajectory.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web_ui,node_gesture_logic,node_web_server,node_gesture_model toneBlue
class node_panda_recorder,node_push_task,node_panda_scene,node_render_workers toneAmber
class node_demo_data,node_video_data toneMint
class node_act_training,node_dataset_validation,node_trajectory_analysis,node_evaluation_output toneRose
class node_operator,node_lerobot toneIndigo
```

## Components

### Gesture Teleoperation

- [Gesture UI](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/app.js): `teleoperation/mediapipe/app.js`
- [Motion Mapping](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/interaction-state.js): `teleoperation/mediapipe/interaction-state.js`
- [Web Proxy](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/server.js): `teleoperation/mediapipe/server.js`
- [Gesture Model](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/teleoperation/mediapipe/models/gesture_recognizer.task): `teleoperation/mediapipe/models/gesture_recognizer.task`

### Simulation Recording

- [Panda Recorder](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/record_mujoco_panda.py): `simulation/mujoco/record_mujoco_panda.py`
- [Push-T Launcher](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/record_mujoco_push_t.py): `simulation/mujoco/record_mujoco_push_t.py`
- [Panda Scenes](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/assets/franka_emika_panda/push_t_scene.xml): `simulation/mujoco/assets/franka_emika_panda/push_t_scene.xml`
- [Render Workers](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/render_workers.py): `simulation/mujoco/render_workers.py`

### Demonstration Data

- [Demonstration Datasets](https://github.com/shuyi-song-07/mujoco_simulate/tree/main/dataset/demonstrations): `dataset/demonstrations`
- [Episode Videos](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/simulation/mujoco/render_workers.py): `simulation/mujoco/render_workers.py`

### Training and Evaluation

- [ACT Training](https://github.com/shuyi-song-07/mujoco_simulate/tree/main/training/ACT): `training/ACT`
- [Dataset Validation](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/evaluation/validate_multicam_dataset.py): `evaluation/validate_multicam_dataset.py`
- [Trajectory Analysis](https://github.com/shuyi-song-07/mujoco_simulate/blob/main/dataset/demonstrations/analyze_action_trajectory.py): `dataset/demonstrations/analyze_action_trajectory.py`
- **Analysis Reports**

### Other components

- **Operator**
- **LeRobot**

## Connections

- Operator → Gesture UI: controls
- Gesture UI → Gesture Model: recognizes gestures
- Gesture UI → Motion Mapping: maps motion
- Gesture UI → Web Proxy: sends commands
- Web Proxy → Panda Recorder: proxies API
- Panda Recorder → Web Proxy: serves previews
- Push-T Launcher → Panda Recorder: launches task
- Panda Recorder → Panda Scenes: loads scene
- Panda Recorder → Render Workers: dispatches frames
- Render Workers → Demonstration Datasets: writes episodes
- Render Workers → Episode Videos: encodes views
- ACT Training → Demonstration Datasets: reads dataset
- ACT Training → LeRobot: trains with
- Dataset Validation → Demonstration Datasets: checks dataset
- Dataset Validation → Analysis Reports: writes reports
- Trajectory Analysis → Demonstration Datasets: reads trajectories
- Trajectory Analysis → Panda Scenes: reconstructs pose
- Trajectory Analysis → Analysis Reports: writes plots