How wiscohumanoids/manipulation_sim works
manipulation_sim is a simulated robotics-learning sandbox. Its main walkthrough collects scripted Fetch demonstrations, converts them to a LeRobot dataset, trains an ACT policy, then evaluates it in the environment. Separate branches provide a Jacobian-IK pick-and-place baseline, Diffusion Policy, PPO reach/pick-place training, and an interactive policy launcher. Edges reflect README workflow statements or shown imports/calls; unsampled approach internals are represented only where the README documents their role.
Main entry points: collect_demos.py Demo collection · play.py Interactive launcher · arm_pick_place.py Fetch tasks · pick_place_scripted.py IK pick-place
Components 21
Simulation
- Fetch tasks
envs/arm_pick_place.py - Reach task
envs/arm_reach.py
Scripted control
- IK pick-place
approaches/scripted_ik/pick_place_scripted.py - Jacobian IK
approaches/scripted_ik/ik_solver.py
Imitation learning
- Scripted expert
approaches/imitation_act/scripted_expert.py - Demo collection
approaches/imitation_act/collect_demos.py - Raw episodes
utils/recording.py - Dataset formatting
approaches/imitation_act/format_dataset.py - LeRobot dataset
approaches/imitation_act/format_dataset.py - ACT training
approaches/imitation_act/train_act.py - Diffusion training
approaches/imitation_diffusion/train_diffusion.py - ACT evaluation
approaches/imitation_act/evaluate.py - Diffusion evaluation
approaches/imitation_diffusion/evaluate.py - Policy checkpoint
approaches/imitation_act/train_act.py
Reinforcement learning
- PPO training
approaches/rl_grasp/train_rl.py
Interactive runtime
- Video output
utils/recording.py - Interactive launcher
play.py - Output paths
utils/repo_paths.py - MuJoCo rendering
utils/viz.py
Other components
- Learner
- Hugging Face Hub
Connections 34
- Learner → IK pick-place: run baseline
README.md - IK pick-place → Jacobian IK: solves with
approaches/scripted_ik/pick_place_scripted.py - Learner → Demo collection: collect demos
README.md - Demo collection → Fetch tasks: steps task
approaches/imitation_act/collect_demos.py - Demo collection → Scripted expert: requests actions
approaches/imitation_act/collect_demos.py - Demo collection → Raw episodes: saves episodes
approaches/imitation_act/collect_demos.py - Dataset formatting → Raw episodes: loads episodes
approaches/imitation_act/format_dataset.py - Dataset formatting → LeRobot dataset: creates dataset
approaches/imitation_act/format_dataset.py - Learner → Dataset formatting: format demos
README.md - Dataset formatting → Hugging Face Hub: optionally pushes
README.md - Learner → ACT training: train policy
README.md - ACT training → LeRobot dataset: trains from
README.md - ACT training → Policy checkpoint: saves checkpoint
README.md - Learner → Diffusion training: choose diffusion
- Diffusion training → LeRobot dataset: trains from
approaches/imitation_act/format_dataset.py - Diffusion training → Policy checkpoint: saves policy
approaches/imitation_diffusion/train_diffusion.py - Learner → ACT evaluation: evaluate policy
README.md - ACT evaluation → Fetch tasks: runs policy in
approaches/imitation_act/evaluate.py - ACT evaluation → Policy checkpoint: loads checkpoint
README.md - ACT evaluation → Video output: records video
approaches/imitation_act/evaluate.py - Diffusion evaluation → Fetch tasks: runs policy in
approaches/imitation_diffusion/evaluate.py - Diffusion evaluation → Video output: records video
approaches/imitation_diffusion/evaluate.py - Learner → PPO training: train PPO
approaches/rl_grasp/train_rl.py - PPO training → Fetch tasks: selects pick-place
approaches/rl_grasp/train_rl.py - PPO training → Reach task: selects reach
approaches/rl_grasp/train_rl.py - Learner → Interactive launcher: launch interactively
play.py - Interactive launcher → Fetch tasks: steps environment
play.py - Interactive launcher → Scripted expert: records oracle demos
play.py - Interactive launcher → Policy checkpoint: discovers policy
play.py - IK pick-place → MuJoCo rendering: renders with
approaches/scripted_ik/pick_place_scripted.py - IK pick-place → Video output: writes recording
approaches/scripted_ik/pick_place_scripted.py - Interactive launcher → MuJoCo rendering: renders frames
play.py - Demo collection → Output paths: uses output paths
approaches/imitation_act/collect_demos.py - ACT training → Output paths: uses output paths
approaches/imitation_act/train_act.py