# dev2703/bimanual-manipulation architecture

> Architecture diagram of the GitHub repository dev2703/bimanual-manipulation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dev2703/bimanual-manipulation
- Repository: https://github.com/dev2703/bimanual-manipulation

## Overview

Bimanual is a MuJoCo-based simulated dinner-table manipulation system for ALOHA 2, with a legacy SO-101 comparison profile. The documented workflow registers a task skill, runs its scripted contact expert in simulation, checks success through held-out gates, and records expert demonstrations for learned policies such as ACT and SmolVLA. Policy runners and evaluation support rollouts; task memory tracks long-horizon progress, while verifier components provide a separate learned perception capability. Edges use README statements and source-index references; unsampled implementations are represented only where the tree supports their paths.

Main entry points: [`bimanual/skills/registry.py`](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/skills/registry.py) (Skill registry), [`bimanual/experts`](https://github.com/dev2703/bimanual-manipulation/tree/main/bimanual/experts) (Contact experts), [`bimanual/sim/aloha_env.py`](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/sim/aloha_env.py) (ALOHA environment), [`bimanual/data/record.py`](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/data/record.py) (Dataset recording)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_tasks["Tasks and experts"]
  node_registry["Skill registry<br/>[registry.py]"]
  node_experts["Contact experts"]
end

subgraph group_sim["Simulation and control"]
  node_env["ALOHA environment<br/>[aloha_env.py]"]
  node_ik["Differential IK<br/>[aloha_ik.py]"]
  node_motion["Motion primitives<br/>[aloha_motion.py]"]
  node_context["Inter-arm state<br/>[aloha_context.py]"]
end

subgraph group_learning["Policies and state"]
  node_memory["Task memory<br/>[task_memory.py]"]
  node_policy_contract["Policy contracts<br/>[types.py]"]
  node_act["ACT runner"]
  node_smolvla["SmolVLA runner<br/>[runner.py]"]
  node_compact["Compact VLA<br/>[compact_vla.py]"]
  node_verifier["Scene verifier<br/>[verifier.py]"]
end

subgraph group_dataeval["Data and evaluation"]
  node_record["Dataset recording<br/>[record.py]"]
  node_dataset[("LeRobot datasets")]
  node_audit["Dataset audit<br/>[audit.py]"]
  node_gate["Skill gates<br/>[skill_gate.py]"]
  node_predicates["Success predicates"]
  node_pour_geometry["Pour geometry<br/>[pour_geometry.py]"]
end

subgraph group_legacy["SO-101 legacy"]
  node_legacy_sim["SO-101 simulation<br/>[env.py]"]
  node_legacy_experts["SO-101 experts<br/>[generate.py]"]
  node_legacy_ik["SO-101 IK<br/>[ik.py]"]
end

node_user(("Task user"))

node_user -->|"selects skill"| node_registry
node_registry -->|"dispatches"| node_experts
node_experts -->|"acts in"| node_env
node_experts -->|"solves motion"| node_ik
node_motion -->|"steps"| node_env
node_experts -->|"uses"| node_motion
node_context -->|"reads state"| node_env
node_context -->|"defines embodiment"| node_policy_contract
node_registry -->|"builds task state"| node_memory
node_registry -->|"supplies skills"| node_gate
node_gate -->|"checks success"| node_predicates
node_experts -->|"aligns pour"| node_pour_geometry
node_registry -->|"scores pour"| node_pour_geometry
node_registry -->|"selects recording skill"| node_record
node_record -->|"records rollouts"| node_env
node_record -->|"writes episodes"| node_dataset
node_experts -->|"produce demonstrations"| node_record
node_dataset -->|"audits"| node_audit
node_dataset -.->|"trains policy"| node_act
node_act -->|"runs policy"| node_env
node_smolvla -->|"runs policy"| node_env
node_compact -->|"returns policy output"| node_policy_contract
node_verifier -.->|"uses observation contracts"| node_policy_contract
node_legacy_experts -->|"generates rollouts"| node_legacy_sim
node_legacy_experts -->|"solves motion"| node_legacy_ik
node_legacy_sim -.->|"uses kinematics"| node_legacy_ik

click node_registry "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/skills/registry.py"
click node_experts "https://github.com/dev2703/bimanual-manipulation/tree/main/bimanual/experts"
click node_env "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/sim/aloha_env.py"
click node_ik "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/control/aloha_ik.py"
click node_motion "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/experts/aloha_motion.py"
click node_context "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/control/aloha_context.py"
click node_memory "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/memory/task_memory.py"
click node_policy_contract "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/types.py"
click node_act "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/aloha_act_runner.py"
click node_smolvla "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/smolvla/runner.py"
click node_compact "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/compact_vla.py"
click node_verifier "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/perception/verifier.py"
click node_record "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/data/record.py"
click node_dataset "https://github.com/dev2703/bimanual-manipulation/tree/main/bimanual/data"
click node_audit "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/data/audit.py"
click node_gate "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/evaluation/skill_gate.py"
click node_predicates "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/evaluation/aloha_predicates.py"
click node_pour_geometry "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/evaluation/pour_geometry.py"
click node_legacy_sim "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/legacy_so101/sim/env.py"
click node_legacy_experts "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/legacy_so101/experts/generate.py"
click node_legacy_ik "https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/legacy_so101/control/ik.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_registry,node_experts toneBlue
class node_env,node_ik,node_motion,node_context toneAmber
class node_memory,node_policy_contract,node_act,node_smolvla,node_compact,node_verifier toneMint
class node_record,node_dataset,node_audit,node_gate,node_predicates,node_pour_geometry,node_user toneRose
class node_legacy_sim,node_legacy_experts,node_legacy_ik toneIndigo
```

## Components

### Tasks and experts

- [Skill registry](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/skills/registry.py): `bimanual/skills/registry.py`
- [Contact experts](https://github.com/dev2703/bimanual-manipulation/tree/main/bimanual/experts): `bimanual/experts`

### Simulation and control

- [ALOHA environment](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/sim/aloha_env.py): `bimanual/sim/aloha_env.py`
- [Differential IK](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/control/aloha_ik.py): `bimanual/control/aloha_ik.py`
- [Motion primitives](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/experts/aloha_motion.py): `bimanual/experts/aloha_motion.py`
- [Inter-arm state](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/control/aloha_context.py): `bimanual/control/aloha_context.py`

### Policies and state

- [Task memory](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/memory/task_memory.py): `bimanual/memory/task_memory.py`
- [Policy contracts](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/types.py): `bimanual/policy/types.py`
- [ACT runner](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/aloha_act_runner.py): `bimanual/policy/aloha_act_runner.py`
- [SmolVLA runner](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/smolvla/runner.py): `bimanual/policy/smolvla/runner.py`
- [Compact VLA](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/policy/compact_vla.py): `bimanual/policy/compact_vla.py`
- [Scene verifier](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/perception/verifier.py): `bimanual/perception/verifier.py`

### Data and evaluation

- [Dataset recording](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/data/record.py): `bimanual/data/record.py`
- [LeRobot datasets](https://github.com/dev2703/bimanual-manipulation/tree/main/bimanual/data): `bimanual/data`
- [Dataset audit](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/data/audit.py): `bimanual/data/audit.py`
- [Skill gates](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/evaluation/skill_gate.py): `bimanual/evaluation/skill_gate.py`
- [Success predicates](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/evaluation/aloha_predicates.py): `bimanual/evaluation/aloha_predicates.py`
- [Pour geometry](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/evaluation/pour_geometry.py): `bimanual/evaluation/pour_geometry.py`

### SO-101 legacy

- [SO-101 simulation](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/legacy_so101/sim/env.py): `bimanual/legacy_so101/sim/env.py`
- [SO-101 experts](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/legacy_so101/experts/generate.py): `bimanual/legacy_so101/experts/generate.py`
- [SO-101 IK](https://github.com/dev2703/bimanual-manipulation/blob/main/bimanual/legacy_so101/control/ik.py): `bimanual/legacy_so101/control/ik.py`

### Other components

- **Task user**

## Connections

- Task user → Skill registry: selects skill (evidence: `README.md`)
- Skill registry → Contact experts: dispatches (evidence: `bimanual/skills/registry.py`)
- Contact experts → ALOHA environment: acts in (evidence: `bimanual/experts/aloha_motion.py`)
- Contact experts → Differential IK: solves motion (evidence: `bimanual/experts/aloha_motion.py`)
- Motion primitives → ALOHA environment: steps (evidence: `bimanual/experts/aloha_motion.py`)
- Contact experts → Motion primitives: uses (evidence: `bimanual/experts/aloha_handoff.py`)
- Inter-arm state → ALOHA environment: reads state (evidence: `bimanual/control/aloha_context.py`)
- Inter-arm state → Policy contracts: defines embodiment (evidence: `bimanual/control/aloha_context.py`)
- Skill registry → Task memory: builds task state (evidence: `bimanual/skills/registry.py`)
- Skill registry → Skill gates: supplies skills (evidence: `bimanual/evaluation/skill_gate.py`)
- Skill gates → Success predicates: checks success
- Contact experts → Pour geometry: aligns pour (evidence: `bimanual/experts/aloha_pour.py`)
- Skill registry → Pour geometry: scores pour (evidence: `bimanual/skills/registry.py`)
- Skill registry → Dataset recording: selects recording skill (evidence: `bimanual/data/record.py`)
- Dataset recording → ALOHA environment: records rollouts (evidence: `bimanual/data/record.py`)
- Dataset recording → LeRobot datasets: writes episodes (evidence: `README.md`)
- Contact experts → Dataset recording: produce demonstrations (evidence: `README.md`)
- LeRobot datasets → Dataset audit: audits (evidence: `README.md`)
- LeRobot datasets → ACT runner: trains policy (evidence: `README.md`)
- ACT runner → ALOHA environment: runs policy (evidence: `bimanual/policy/aloha_act_runner.py`)
- SmolVLA runner → ALOHA environment: runs policy (evidence: `bimanual/policy/smolvla/runner.py`)
- Compact VLA → Policy contracts: returns policy output (evidence: `bimanual/policy/compact_vla.py`)
- Scene verifier → Policy contracts: uses observation contracts
- SO-101 experts → SO-101 simulation: generates rollouts (evidence: `bimanual/legacy_so101/experts/generate.py`)
- SO-101 experts → SO-101 IK: solves motion (evidence: `bimanual/legacy_so101/experts/generate.py`)
- SO-101 simulation → SO-101 IK: uses kinematics
