# bai-zihan/robotwin architecture

> Architecture diagram of the GitHub repository bai-zihan/robotwin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bai-zihan/robotwin
- Repository: https://github.com/bai-zihan/robotwin

## Overview

RoboTwin is a bimanual manipulation simulation benchmark and configurable trajectory generator, with a broad set of task environments and a collection of robot-learning policies. The documented user workflow is to select a task and configuration, run the task to collect trajectories, then process data for policy training and evaluation/deployment. The task base source supports simulation, camera observations, and robot motion planning. Policy families and processing capabilities are represented from the repository tree; their internal wiring was not established by the sampled excerpts.

Main entry points: [`policy/RDT/train/train.py`](https://github.com/bai-zihan/robotwin/blob/main/policy/RDT/train/train.py) (Policy Training), [`envs/_base_task.py`](https://github.com/bai-zihan/robotwin/blob/main/envs/_base_task.py) (Task Environment), [`policy/ACT/record_sim_episodes.py`](https://github.com/bai-zihan/robotwin/blob/main/policy/ACT/record_sim_episodes.py) (Episode Collection), [`envs`](https://github.com/bai-zihan/robotwin/tree/main/envs) (Manipulation Tasks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_sim["Task Simulation"]
  node_tasks["Manipulation Tasks"]
  node_base["Task Environment<br/>[_base_task.py]"]
  node_robot["Robot Control"]
  node_cameras["Camera Sensing<br/>[camera.py]"]
  node_randomization["Scene Randomization"]
end

subgraph group_data["Trajectory Data"]
  node_collector["Episode Collection"]
  node_episode_data[("Recorded Trajectories")]
  node_conversion["Data Processing<br/>[process_data.py]"]
  node_task_metadata["Task Descriptions<br/>[task_info.py]"]
end

subgraph group_policies["Policy Models"]
  node_act["ACT Policy<br/>[act_policy.py]"]
  node_diffusion["Diffusion Policy"]
  node_pointcloud["DP3 Policy<br/>[dp3.py]"]
  node_vla["Vision-Language Policies"]
  node_other_policies["Additional Policies"]
  node_training["Policy Training<br/>[train.py]"]
end

subgraph group_execution["Robot Execution"]
  node_deployment["Policy Deployment<br/>[deploy_policy.py]"]
  node_robot_task["Robot Task Runtime<br/>[_base_task.py]"]
  node_pi_client["OpenPI WebSocket Client"]
end

node_user(("Benchmark User"))
node_inference_server{{"Inference Server"}}

node_user -->|"selects task"| node_tasks
node_tasks -->|"defines environment"| node_base
node_base -->|"creates scene actors"| node_randomization
node_base -->|"plans motion"| node_robot
node_base -->|"captures observations"| node_cameras
node_user -->|"starts collection"| node_collector
node_collector -->|"runs episodes"| node_base
node_collector -->|"writes trajectories"| node_episode_data
node_episode_data -->|"loads trajectories"| node_conversion
node_conversion -->|"prepares training data"| node_training
node_training -->|"trains"| node_act
node_training -->|"trains"| node_diffusion
node_training -->|"trains"| node_pointcloud
node_training -->|"trains"| node_vla
node_training -->|"trains"| node_other_policies
node_user -->|"starts evaluation"| node_deployment
node_deployment -->|"runs policy"| node_robot_task
node_pi_client -->|"sends observations"| node_inference_server
node_inference_server -->|"returns actions"| node_pi_client
node_task_metadata -.->|"describes tasks"| node_tasks

click node_tasks "https://github.com/bai-zihan/robotwin/tree/main/envs"
click node_base "https://github.com/bai-zihan/robotwin/blob/main/envs/_base_task.py"
click node_robot "https://github.com/bai-zihan/robotwin/tree/main/envs/robot"
click node_cameras "https://github.com/bai-zihan/robotwin/blob/main/envs/camera/camera.py"
click node_randomization "https://github.com/bai-zihan/robotwin/blob/main/envs/utils/rand_create_actor.py"
click node_collector "https://github.com/bai-zihan/robotwin/blob/main/policy/ACT/record_sim_episodes.py"
click node_conversion "https://github.com/bai-zihan/robotwin/blob/main/policy/DP/process_data.py"
click node_task_metadata "https://github.com/bai-zihan/robotwin/blob/main/code_gen/task_info.py"
click node_act "https://github.com/bai-zihan/robotwin/blob/main/policy/ACT/act_policy.py"
click node_diffusion "https://github.com/bai-zihan/robotwin/tree/main/policy/DP/diffusion_policy"
click node_pointcloud "https://github.com/bai-zihan/robotwin/blob/main/policy/DP3/3D-Diffusion-Policy/diffusion_policy_3d/policy/dp3.py"
click node_vla "https://github.com/bai-zihan/robotwin/blob/main/policy/DexVLA/dex_vla/models/modeling_dex_vla.py"
click node_other_policies "https://github.com/bai-zihan/robotwin/tree/main/policy"
click node_training "https://github.com/bai-zihan/robotwin/blob/main/policy/RDT/train/train.py"
click node_deployment "https://github.com/bai-zihan/robotwin/blob/main/policy/ACT/deploy_policy.py"
click node_robot_task "https://github.com/bai-zihan/robotwin/blob/main/envs/_base_task.py"
click node_pi_client "https://github.com/bai-zihan/robotwin/blob/main/policy/pi0/packages/openpi-client/src/openpi_client/websocket_client_policy.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_tasks,node_base,node_robot,node_cameras,node_randomization,node_user toneBlue
class node_collector,node_episode_data,node_conversion,node_task_metadata toneAmber
class node_act,node_diffusion,node_pointcloud,node_vla,node_other_policies,node_training,node_inference_server toneMint
class node_deployment,node_robot_task,node_pi_client toneRose
```

## Components

### Task Simulation

- [Manipulation Tasks](https://github.com/bai-zihan/robotwin/tree/main/envs): `envs`
- [Task Environment](https://github.com/bai-zihan/robotwin/blob/main/envs/_base_task.py): `envs/_base_task.py`
- [Robot Control](https://github.com/bai-zihan/robotwin/tree/main/envs/robot): `envs/robot`
- [Camera Sensing](https://github.com/bai-zihan/robotwin/blob/main/envs/camera/camera.py): `envs/camera/camera.py`
- [Scene Randomization](https://github.com/bai-zihan/robotwin/blob/main/envs/utils/rand_create_actor.py): `envs/utils/rand_create_actor.py`

### Trajectory Data

- [Episode Collection](https://github.com/bai-zihan/robotwin/blob/main/policy/ACT/record_sim_episodes.py): `policy/ACT/record_sim_episodes.py`
- **Recorded Trajectories**
- [Data Processing](https://github.com/bai-zihan/robotwin/blob/main/policy/DP/process_data.py): `policy/DP/process_data.py`
- [Task Descriptions](https://github.com/bai-zihan/robotwin/blob/main/code_gen/task_info.py): `code_gen/task_info.py`

### Policy Models

- [ACT Policy](https://github.com/bai-zihan/robotwin/blob/main/policy/ACT/act_policy.py): `policy/ACT/act_policy.py`
- [Diffusion Policy](https://github.com/bai-zihan/robotwin/tree/main/policy/DP/diffusion_policy): `policy/DP/diffusion_policy`
- [DP3 Policy](https://github.com/bai-zihan/robotwin/blob/main/policy/DP3/3D-Diffusion-Policy/diffusion_policy_3d/policy/dp3.py): `policy/DP3/3D-Diffusion-Policy/diffusion_policy_3d/policy/dp3.py`
- [Vision-Language Policies](https://github.com/bai-zihan/robotwin/blob/main/policy/DexVLA/dex_vla/models/modeling_dex_vla.py): `policy/DexVLA/dex_vla/models/modeling_dex_vla.py`
- [Additional Policies](https://github.com/bai-zihan/robotwin/tree/main/policy): `policy`
- [Policy Training](https://github.com/bai-zihan/robotwin/blob/main/policy/RDT/train/train.py): `policy/RDT/train/train.py`

### Robot Execution

- [Policy Deployment](https://github.com/bai-zihan/robotwin/blob/main/policy/ACT/deploy_policy.py): `policy/ACT/deploy_policy.py`
- [Robot Task Runtime](https://github.com/bai-zihan/robotwin/blob/main/envs/_base_task.py): `envs/_base_task.py`
- [OpenPI WebSocket Client](https://github.com/bai-zihan/robotwin/blob/main/policy/pi0/packages/openpi-client/src/openpi_client/websocket_client_policy.py): `policy/pi0/packages/openpi-client/src/openpi_client/websocket_client_policy.py`

### Other components

- **Benchmark User**
- **Inference Server**

## Connections

- Benchmark User → Manipulation Tasks: selects task
- Manipulation Tasks → Task Environment: defines environment
- Task Environment → Scene Randomization: creates scene actors
- Task Environment → Robot Control: plans motion
- Task Environment → Camera Sensing: captures observations
- Benchmark User → Episode Collection: starts collection
- Episode Collection → Task Environment: runs episodes
- Episode Collection → Recorded Trajectories: writes trajectories
- Recorded Trajectories → Data Processing: loads trajectories
- Data Processing → Policy Training: prepares training data
- Policy Training → ACT Policy: trains
- Policy Training → Diffusion Policy: trains
- Policy Training → DP3 Policy: trains
- Policy Training → Vision-Language Policies: trains
- Policy Training → Additional Policies: trains
- Benchmark User → Policy Deployment: starts evaluation
- Policy Deployment → Robot Task Runtime: runs policy
- OpenPI WebSocket Client → Inference Server: sends observations
- Inference Server → OpenPI WebSocket Client: returns actions
- Task Descriptions → Manipulation Tasks: describes tasks