How shuyi-song-07/mujoco_simulate works
This repository supports a robot-learning workflow: operators control simulated Panda pick-and-place or Push-T tasks through webcam gestures, record demonstrations, validate datasets, and train an ACT baseline. The recorder coordinates simulation, task state, and asynchronous rendering/data capture; separate tools inspect trajectories and dataset integrity. The graph reflects README-described capabilities where source excerpts were unavailable; unsampled evaluation or training internals are represented only at their documented entry points.
Main entry points: record_mujoco_panda.py Panda Recorder · app.js Gesture UI · demonstrations Demonstration Datasets · analyze_action_trajectory.py Trajectory Analysis
Components 16
Gesture Teleoperation
- Gesture UI
teleoperation/mediapipe/app.js - Motion Mapping
teleoperation/mediapipe/interaction-state.js - Web Proxy
teleoperation/mediapipe/server.js - Gesture Model
teleoperation/mediapipe/models/gesture_recognizer.task
Simulation Recording
- Panda Recorder
simulation/mujoco/record_mujoco_panda.py - Push-T Launcher
simulation/mujoco/record_mujoco_push_t.py - Panda Scenes
simulation/mujoco/assets/franka_emika_panda/push_t_scene.xml - Render Workers
simulation/mujoco/render_workers.py
Demonstration Data
- Demonstration Datasets
dataset/demonstrations - Episode Videos
simulation/mujoco/render_workers.py
Training and Evaluation
- ACT Training
training/ACT - Dataset Validation
evaluation/validate_multicam_dataset.py - Trajectory Analysis
dataset/demonstrations/analyze_action_trajectory.py - Analysis Reports
Other components
- Operator
- LeRobot
Connections 18
- Operator → Gesture UI: controls
- Gesture UI → Gesture Model: recognizes gestures
- Gesture UI → Motion Mapping: maps motion
- Gesture UI → Web Proxy: sends commands
- Web Proxy → Panda Recorder: proxies API
- Panda Recorder → Web Proxy: serves previews
- Push-T Launcher → Panda Recorder: launches task
- Panda Recorder → Panda Scenes: loads scene
- Panda Recorder → Render Workers: dispatches frames
- Render Workers → Demonstration Datasets: writes episodes
- Render Workers → Episode Videos: encodes views
- ACT Training → Demonstration Datasets: reads dataset
- ACT Training → LeRobot: trains with
- Dataset Validation → Demonstration Datasets: checks dataset
- Dataset Validation → Analysis Reports: writes reports
- Trajectory Analysis → Demonstration Datasets: reads trajectories
- Trajectory Analysis → Panda Scenes: reconstructs pose
- Trajectory Analysis → Analysis Reports: writes plots