How dev2703/bimanual-manipulation works
Bimanual is a MuJoCo-based simulated dinner-table manipulation system for ALOHA 2, with a legacy SO-101 comparison profile. The documented workflow registers a task skill, runs its scripted contact expert in simulation, checks success through held-out gates, and records expert demonstrations for learned policies such as ACT and SmolVLA. Policy runners and evaluation support rollouts; task memory tracks long-horizon progress, while verifier components provide a separate learned perception capability. Edges use README statements and source-index references; unsampled implementations are represented only where the tree supports their paths.
Main entry points: registry.py Skill registry · experts Contact experts · aloha_env.py ALOHA environment · record.py Dataset recording
Components 22
Tasks and experts
- Skill registry
bimanual/skills/registry.py - Contact experts
bimanual/experts
Simulation and control
- ALOHA environment
bimanual/sim/aloha_env.py - Differential IK
bimanual/control/aloha_ik.py - Motion primitives
bimanual/experts/aloha_motion.py - Inter-arm state
bimanual/control/aloha_context.py
Policies and state
- Task memory
bimanual/memory/task_memory.py - Policy contracts
bimanual/policy/types.py - ACT runner
bimanual/policy/aloha_act_runner.py - SmolVLA runner
bimanual/policy/smolvla/runner.py - Compact VLA
bimanual/policy/compact_vla.py - Scene verifier
bimanual/perception/verifier.py
Data and evaluation
- Dataset recording
bimanual/data/record.py - LeRobot datasets
bimanual/data - Dataset audit
bimanual/data/audit.py - Skill gates
bimanual/evaluation/skill_gate.py - Success predicates
bimanual/evaluation/aloha_predicates.py - Pour geometry
bimanual/evaluation/pour_geometry.py
SO-101 legacy
- SO-101 simulation
bimanual/legacy_so101/sim/env.py - SO-101 experts
bimanual/legacy_so101/experts/generate.py - SO-101 IK
bimanual/legacy_so101/control/ik.py
Other components
- Task user
Connections 26
- Task user → Skill registry: selects skill
README.md - Skill registry → Contact experts: dispatches
bimanual/skills/registry.py - Contact experts → ALOHA environment: acts in
bimanual/experts/aloha_motion.py - Contact experts → Differential IK: solves motion
bimanual/experts/aloha_motion.py - Motion primitives → ALOHA environment: steps
bimanual/experts/aloha_motion.py - Contact experts → Motion primitives: uses
bimanual/experts/aloha_handoff.py - Inter-arm state → ALOHA environment: reads state
bimanual/control/aloha_context.py - Inter-arm state → Policy contracts: defines embodiment
bimanual/control/aloha_context.py - Skill registry → Task memory: builds task state
bimanual/skills/registry.py - Skill registry → Skill gates: supplies skills
bimanual/evaluation/skill_gate.py - Skill gates → Success predicates: checks success
- Contact experts → Pour geometry: aligns pour
bimanual/experts/aloha_pour.py - Skill registry → Pour geometry: scores pour
bimanual/skills/registry.py - Skill registry → Dataset recording: selects recording skill
bimanual/data/record.py - Dataset recording → ALOHA environment: records rollouts
bimanual/data/record.py - Dataset recording → LeRobot datasets: writes episodes
README.md - Contact experts → Dataset recording: produce demonstrations
README.md - LeRobot datasets → Dataset audit: audits
README.md - LeRobot datasets → ACT runner: trains policy
README.md - ACT runner → ALOHA environment: runs policy
bimanual/policy/aloha_act_runner.py - SmolVLA runner → ALOHA environment: runs policy
bimanual/policy/smolvla/runner.py - Compact VLA → Policy contracts: returns policy output
bimanual/policy/compact_vla.py - Scene verifier → Policy contracts: uses observation contracts
- SO-101 experts → SO-101 simulation: generates rollouts
bimanual/legacy_so101/experts/generate.py - SO-101 experts → SO-101 IK: solves motion
bimanual/legacy_so101/experts/generate.py - SO-101 simulation → SO-101 IK: uses kinematics