# aks-lab-univertsity-of-tartu/bimanual_manipulation architecture

> Architecture diagram of the GitHub repository aks-lab-univertsity-of-tartu/bimanual_manipulation, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aks-lab-univertsity-of-tartu/bimanual_manipulation
- Repository: https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation

## Overview

This is a ROS 2 dual-arm manipulation demo whose principal workflow is online planning: the demo coordinates an MPC/CEM planner, which uses MJX-based trajectory optimization and can use an IK solver to refine end-effector targets. The repository also includes a MuJoCo trajectory visualizer and a Mink differential-IK library with task and limit abstractions. The README documents the launch and visualization capabilities, but the robot model/configuration files it describes are absent from this tree. Relationships for unsampled implementations are included only where the README or source index documents them.

Main entry points: [`real_demo/collision_free_ik/mink/configuration.py`](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/configuration.py) (Robot configuration), [`real_demo/sampling_based_planner/mpc_planner.py`](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/mpc_planner.py) (MPC / CEM), [`real_demo/ik_based_planner/ik_solver.py`](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/ik_based_planner/ik_solver.py) (IK solver), [`real_demo/collision_free_ik/mink/solve_ik.py`](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/solve_ik.py) (QP IK solve)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_apps["ROS applications"]
  node_demo_launch["Demo launch"]
  node_demo_app["Dual-arm demo<br/>[dual_arm_demo.py]"]
  node_viz_launch["Visualizer launch"]
  node_visualizer["Trajectory visualizer<br/>[visualizer.py]"]
end

subgraph group_planning["Planning"]
  node_mpc["MPC / CEM<br/>[mpc_planner.py]"]
  node_mjx["MJX planner<br/>[mjx_planner.py]"]
  node_mlp["MLP projection<br/>[mlp_singledof.py]"]
  node_quat["Quaternion math<br/>[quat_math.py]"]
end

subgraph group_ik["Inverse kinematics"]
  node_ik_solver["IK solver<br/>[ik_solver.py]"]
  node_solve_ik["QP IK solve<br/>[solve_ik.py]"]
  node_frame_task["Frame task<br/>[frame_task.py]"]
  node_task_base["Task objective<br/>[task.py]"]
end

subgraph group_mink["Mink foundations"]
  node_configuration["Robot configuration<br/>[configuration.py]"]
  node_config_limit["Configuration limit"]
  node_collision_limit["Collision limit"]
  node_limit_base["Limit interface<br/>[limit.py]"]
  node_lie["Lie geometry"]
  node_exceptions["IK exceptions<br/>[exceptions.py]"]
end

subgraph group_data["Demo data"]
  node_demo_data["Setup and trajectories"]
end

node_operator(("Operator"))

node_operator -->|"launches"| node_demo_launch
node_demo_launch -.->|"starts"| node_demo_app
node_demo_app -->|"runs planning"| node_mpc
node_demo_app -->|"uses rotations"| node_quat
node_operator -->|"launches"| node_viz_launch
node_viz_launch -.->|"starts"| node_visualizer
node_visualizer -->|"uses rotations"| node_quat
node_visualizer -.->|"reads trajectories"| node_demo_data
node_mpc -->|"optimizes trajectory"| node_mjx
node_mpc -->|"loads projection"| node_mlp
node_mpc -->|"uses rotations"| node_quat
node_mpc -->|"solves targets"| node_ik_solver
node_ik_solver -->|"initializes state"| node_configuration
node_ik_solver -->|"sets target"| node_frame_task
node_ik_solver -->|"sets limit"| node_config_limit
node_ik_solver -->|"solves velocity"| node_solve_ik
node_solve_ik -->|"builds objective"| node_task_base
node_solve_ik -->|"builds inequalities"| node_limit_base
node_solve_ik -->|"uses state"| node_configuration
node_frame_task -->|"extends"| node_task_base
node_frame_task -->|"reads transforms"| node_configuration
node_configuration -->|"builds transforms"| node_lie
node_configuration -->|"raises errors"| node_exceptions
node_config_limit -->|"implements"| node_limit_base
node_collision_limit -->|"implements"| node_limit_base
node_collision_limit -->|"checks contacts"| node_configuration

click node_demo_launch "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/launch/dual_arm_demo.launch.py"
click node_demo_app "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/real_demo/dual_arm_demo.py"
click node_viz_launch "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/launch/visualizer.launch.py"
click node_visualizer "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/real_demo/visualizer.py"
click node_mpc "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/mpc_planner.py"
click node_mjx "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/mjx_planner.py"
click node_mlp "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/Simple_MLP/mlp_singledof.py"
click node_quat "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/quat_math.py"
click node_ik_solver "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/ik_based_planner/ik_solver.py"
click node_solve_ik "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/solve_ik.py"
click node_frame_task "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/tasks/frame_task.py"
click node_task_base "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/tasks/task.py"
click node_configuration "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/configuration.py"
click node_config_limit "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/limits/configuration_limit.py"
click node_collision_limit "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/limits/collision_avoidance_limit.py"
click node_limit_base "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/limits/limit.py"
click node_lie "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/tree/main/real_demo/collision_free_ik/mink/lie"
click node_exceptions "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/exceptions.py"
click node_demo_data "https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/tree/main/real_demo/data"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_demo_launch,node_demo_app,node_viz_launch,node_visualizer toneBlue
class node_mpc,node_mjx,node_mlp,node_quat toneAmber
class node_ik_solver,node_solve_ik,node_frame_task,node_task_base toneMint
class node_configuration,node_config_limit,node_collision_limit,node_limit_base,node_lie,node_exceptions toneRose
class node_demo_data,node_operator toneIndigo
```

## Components

### ROS applications

- [Demo launch](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/launch/dual_arm_demo.launch.py): `real_demo/launch/dual_arm_demo.launch.py`
- [Dual-arm demo](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/real_demo/dual_arm_demo.py): `real_demo/real_demo/dual_arm_demo.py`
- [Visualizer launch](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/launch/visualizer.launch.py): `real_demo/launch/visualizer.launch.py`
- [Trajectory visualizer](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/real_demo/visualizer.py): `real_demo/real_demo/visualizer.py`

### Planning

- [MPC / CEM](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/mpc_planner.py): `real_demo/sampling_based_planner/mpc_planner.py`
- [MJX planner](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/mjx_planner.py): `real_demo/sampling_based_planner/mjx_planner.py`
- [MLP projection](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/Simple_MLP/mlp_singledof.py): `real_demo/sampling_based_planner/Simple_MLP/mlp_singledof.py`
- [Quaternion math](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/sampling_based_planner/quat_math.py): `real_demo/sampling_based_planner/quat_math.py`

### Inverse kinematics

- [IK solver](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/ik_based_planner/ik_solver.py): `real_demo/ik_based_planner/ik_solver.py`
- [QP IK solve](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/solve_ik.py): `real_demo/collision_free_ik/mink/solve_ik.py`
- [Frame task](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/tasks/frame_task.py): `real_demo/collision_free_ik/mink/tasks/frame_task.py`
- [Task objective](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/tasks/task.py): `real_demo/collision_free_ik/mink/tasks/task.py`

### Mink foundations

- [Robot configuration](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/configuration.py): `real_demo/collision_free_ik/mink/configuration.py`
- [Configuration limit](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/limits/configuration_limit.py): `real_demo/collision_free_ik/mink/limits/configuration_limit.py`
- [Collision limit](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/limits/collision_avoidance_limit.py): `real_demo/collision_free_ik/mink/limits/collision_avoidance_limit.py`
- [Limit interface](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/limits/limit.py): `real_demo/collision_free_ik/mink/limits/limit.py`
- [Lie geometry](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/tree/main/real_demo/collision_free_ik/mink/lie): `real_demo/collision_free_ik/mink/lie`
- [IK exceptions](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/blob/main/real_demo/collision_free_ik/mink/exceptions.py): `real_demo/collision_free_ik/mink/exceptions.py`

### Demo data

- [Setup and trajectories](https://github.com/aks-lab-univertsity-of-tartu/bimanual_manipulation/tree/main/real_demo/data): `real_demo/data`

### Other components

- **Operator**

## Connections

- Operator → Demo launch: launches
- Demo launch → Dual-arm demo: starts
- Dual-arm demo → MPC / CEM: runs planning (evidence: `real_demo/real_demo/dual_arm_demo.py`)
- Dual-arm demo → Quaternion math: uses rotations (evidence: `real_demo/real_demo/dual_arm_demo.py`)
- Operator → Visualizer launch: launches
- Visualizer launch → Trajectory visualizer: starts
- Trajectory visualizer → Quaternion math: uses rotations (evidence: `real_demo/real_demo/visualizer.py`)
- Trajectory visualizer → Setup and trajectories: reads trajectories
- MPC / CEM → MJX planner: optimizes trajectory (evidence: `real_demo/sampling_based_planner/mpc_planner.py`)
- MPC / CEM → MLP projection: loads projection (evidence: `real_demo/sampling_based_planner/mpc_planner.py`)
- MPC / CEM → Quaternion math: uses rotations (evidence: `real_demo/sampling_based_planner/mpc_planner.py`)
- MPC / CEM → IK solver: solves targets (evidence: `real_demo/sampling_based_planner/mpc_planner.py`)
- IK solver → Robot configuration: initializes state (evidence: `real_demo/ik_based_planner/ik_solver.py`)
- IK solver → Frame task: sets target (evidence: `real_demo/ik_based_planner/ik_solver.py`)
- IK solver → Configuration limit: sets limit (evidence: `real_demo/ik_based_planner/ik_solver.py`)
- IK solver → QP IK solve: solves velocity (evidence: `real_demo/ik_based_planner/ik_solver.py`)
- QP IK solve → Task objective: builds objective (evidence: `real_demo/collision_free_ik/mink/solve_ik.py`)
- QP IK solve → Limit interface: builds inequalities (evidence: `real_demo/collision_free_ik/mink/solve_ik.py`)
- QP IK solve → Robot configuration: uses state (evidence: `real_demo/collision_free_ik/mink/solve_ik.py`)
- Frame task → Task objective: extends (evidence: `real_demo/collision_free_ik/mink/tasks/frame_task.py`)
- Frame task → Robot configuration: reads transforms (evidence: `real_demo/collision_free_ik/mink/tasks/frame_task.py`)
- Robot configuration → Lie geometry: builds transforms (evidence: `real_demo/collision_free_ik/mink/configuration.py`)
- Robot configuration → IK exceptions: raises errors (evidence: `real_demo/collision_free_ik/mink/configuration.py`)
- Configuration limit → Limit interface: implements (evidence: `real_demo/collision_free_ik/mink/limits/configuration_limit.py`)
- Collision limit → Limit interface: implements (evidence: `real_demo/collision_free_ik/mink/limits/collision_avoidance_limit.py`)
- Collision limit → Robot configuration: checks contacts (evidence: `real_demo/collision_free_ik/mink/limits/collision_avoidance_limit.py`)
