# robotwin-platform/robotwin architecture

> Architecture diagram of the GitHub repository robotwin-platform/robotwin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/robotwin-platform/robotwin
- Repository: https://github.com/robotwin-platform/robotwin

## Overview

RoboTwin is a bimanual robotic-manipulation benchmark and simulation platform. A user selects a task and policy; a task environment builds a SAPIEN scene, actors, cameras, and robot, emits observations, receives policy actions, plans and executes motion, then checks success and optionally records an episode. Supporting branches generate task programs and natural-language instructions, render or describe assets, and encode camera data for datasets. The excerpts show core runtime wiring, while policy-server internals and some persistence details remain unsampled.

Main entry points: [`envs/_base_task.py`](https://github.com/robotwin-platform/robotwin/blob/main/envs/_base_task.py) (Task Runtime), [`scripts/eval_policy_multitask.py`](https://github.com/robotwin-platform/robotwin/blob/main/scripts/eval_policy_multitask.py) (Policy Evaluator), [`code_gen/task_info.py`](https://github.com/robotwin-platform/robotwin/blob/main/code_gen/task_info.py) (Task Metadata), [`envs/robot/robot.py`](https://github.com/robotwin-platform/robotwin/blob/main/envs/robot/robot.py) (Bimanual Robot)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_tasks["Task Environments"]
  node_task_env["Task Runtime<br/>[_base_task.py]"]
  node_task_scenarios["Task Scenarios"]
  node_task_metadata["Task Metadata<br/>[task_info.py]"]
end

subgraph group_control["Simulation Control"]
  node_scene_actors["Scene Actors<br/>[create_actor.py]"]
  node_pose_utils["Pose Utilities<br/>[transforms.py]"]
  node_action_api["Action API<br/>[action.py]"]
  node_robot["Bimanual Robot<br/>[robot.py]"]
  node_planner["Motion Planner<br/>[planner.py]"]
end

subgraph group_data["Observation Data"]
  node_cameras["Camera Sensors<br/>[camera.py]"]
  node_image_codec["Image Codec"]
  node_episode_store[("Episode Files<br/>[save_file.py]")]
end

subgraph group_generation["Task Generation"]
  node_gpt_agent["GPT Agent<br/>[gpt_agent.py]"]
  node_observation_agent["Observation Agent"]
  node_code_runner["Code Runner<br/>[run_code.py]"]
  node_instruction_generator["Instruction Generator"]
  node_asset_renderer["Asset Renderer"]
end

subgraph group_evaluation["Collection Evaluation"]
  node_data_collector["Data Collector<br/>[collect_data.py]"]
  node_policy_evaluator["Policy Evaluator"]
end

node_user(("User"))
node_policy(("Robot Policy"))
node_xpolicylab["XPolicyLab"]

node_user -->|"selects task"| node_task_env
node_user -->|"starts evaluation"| node_policy_evaluator
node_task_metadata -->|"guides generation"| node_gpt_agent
node_gpt_agent -->|"generates code"| node_code_runner
node_observation_agent -->|"returns observations"| node_gpt_agent
node_code_runner -->|"runs task"| node_task_env
node_task_env -->|"creates actors"| node_scene_actors
node_task_env -->|"captures observations"| node_cameras
node_task_env -->|"executes controls"| node_robot
node_task_scenarios -->|"requests actions"| node_action_api
node_action_api -->|"dispatches actions"| node_robot
node_robot -->|"plans motion"| node_planner
node_cameras -->|"encodes images"| node_image_codec
node_image_codec -->|"writes frames"| node_episode_store
node_task_env -->|"records episodes"| node_episode_store
node_data_collector -->|"collects episodes"| node_task_env
node_policy_evaluator -->|"runs policies"| node_task_env
node_policy_evaluator -->|"requests actions"| node_policy
node_policy -->|"returns actions"| node_task_env
node_policy_evaluator -.->|"uses deployment"| node_xpolicylab
node_instruction_generator -->|"renders instructions"| node_task_metadata
node_asset_renderer -->|"visualizes assets"| node_task_metadata

click node_task_env "https://github.com/robotwin-platform/robotwin/blob/main/envs/_base_task.py"
click node_task_scenarios "https://github.com/robotwin-platform/robotwin/blob/main/envs/place_bread_basket.py"
click node_task_metadata "https://github.com/robotwin-platform/robotwin/blob/main/code_gen/task_info.py"
click node_scene_actors "https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/create_actor.py"
click node_pose_utils "https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/transforms.py"
click node_action_api "https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/action.py"
click node_robot "https://github.com/robotwin-platform/robotwin/blob/main/envs/robot/robot.py"
click node_planner "https://github.com/robotwin-platform/robotwin/blob/main/envs/robot/planner.py"
click node_cameras "https://github.com/robotwin-platform/robotwin/blob/main/envs/camera/camera.py"
click node_image_codec "https://github.com/robotwin-platform/robotwin/blob/main/data/decode_image_bit.py"
click node_episode_store "https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/save_file.py"
click node_gpt_agent "https://github.com/robotwin-platform/robotwin/blob/main/code_gen/gpt_agent.py"
click node_observation_agent "https://github.com/robotwin-platform/robotwin/blob/main/code_gen/observation_agent.py"
click node_code_runner "https://github.com/robotwin-platform/robotwin/blob/main/code_gen/run_code.py"
click node_instruction_generator "https://github.com/robotwin-platform/robotwin/blob/main/description/utils/generate_episode_instructions.py"
click node_asset_renderer "https://github.com/robotwin-platform/robotwin/blob/main/description/utils/get_image_from_glb.py"
click node_data_collector "https://github.com/robotwin-platform/robotwin/blob/main/scripts/collect_data.py"
click node_policy_evaluator "https://github.com/robotwin-platform/robotwin/blob/main/scripts/eval_policy_multitask.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_task_env,node_task_scenarios,node_task_metadata,node_user toneBlue
class node_scene_actors,node_pose_utils,node_action_api,node_robot,node_planner toneAmber
class node_cameras,node_image_codec,node_episode_store toneMint
class node_gpt_agent,node_observation_agent,node_code_runner,node_instruction_generator,node_asset_renderer toneRose
class node_data_collector,node_policy_evaluator,node_policy,node_xpolicylab toneIndigo
```

## Components

### Task Environments

- [Task Runtime](https://github.com/robotwin-platform/robotwin/blob/main/envs/_base_task.py): `envs/_base_task.py`
- [Task Scenarios](https://github.com/robotwin-platform/robotwin/blob/main/envs/place_bread_basket.py): `envs/place_bread_basket.py`
- [Task Metadata](https://github.com/robotwin-platform/robotwin/blob/main/code_gen/task_info.py): `code_gen/task_info.py`

### Simulation Control

- [Scene Actors](https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/create_actor.py): `envs/utils/create_actor.py`
- [Pose Utilities](https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/transforms.py): `envs/utils/transforms.py`
- [Action API](https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/action.py): `envs/utils/action.py`
- [Bimanual Robot](https://github.com/robotwin-platform/robotwin/blob/main/envs/robot/robot.py): `envs/robot/robot.py`
- [Motion Planner](https://github.com/robotwin-platform/robotwin/blob/main/envs/robot/planner.py): `envs/robot/planner.py`

### Observation Data

- [Camera Sensors](https://github.com/robotwin-platform/robotwin/blob/main/envs/camera/camera.py): `envs/camera/camera.py`
- [Image Codec](https://github.com/robotwin-platform/robotwin/blob/main/data/decode_image_bit.py): `data/decode_image_bit.py`
- [Episode Files](https://github.com/robotwin-platform/robotwin/blob/main/envs/utils/save_file.py): `envs/utils/save_file.py`

### Task Generation

- [GPT Agent](https://github.com/robotwin-platform/robotwin/blob/main/code_gen/gpt_agent.py): `code_gen/gpt_agent.py`
- [Observation Agent](https://github.com/robotwin-platform/robotwin/blob/main/code_gen/observation_agent.py): `code_gen/observation_agent.py`
- [Code Runner](https://github.com/robotwin-platform/robotwin/blob/main/code_gen/run_code.py): `code_gen/run_code.py`
- [Instruction Generator](https://github.com/robotwin-platform/robotwin/blob/main/description/utils/generate_episode_instructions.py): `description/utils/generate_episode_instructions.py`
- [Asset Renderer](https://github.com/robotwin-platform/robotwin/blob/main/description/utils/get_image_from_glb.py): `description/utils/get_image_from_glb.py`

### Collection Evaluation

- [Data Collector](https://github.com/robotwin-platform/robotwin/blob/main/scripts/collect_data.py): `scripts/collect_data.py`
- [Policy Evaluator](https://github.com/robotwin-platform/robotwin/blob/main/scripts/eval_policy_multitask.py): `scripts/eval_policy_multitask.py`

### Other components

- **User**
- **Robot Policy**
- **XPolicyLab**

## Connections

- User → Task Runtime: selects task
- User → Policy Evaluator: starts evaluation
- Task Metadata → GPT Agent: guides generation
- GPT Agent → Code Runner: generates code
- Observation Agent → GPT Agent: returns observations
- Code Runner → Task Runtime: runs task
- Task Runtime → Scene Actors: creates actors
- Task Runtime → Camera Sensors: captures observations
- Task Runtime → Bimanual Robot: executes controls
- Task Scenarios → Action API: requests actions
- Action API → Bimanual Robot: dispatches actions
- Bimanual Robot → Motion Planner: plans motion
- Camera Sensors → Image Codec: encodes images
- Image Codec → Episode Files: writes frames
- Task Runtime → Episode Files: records episodes
- Data Collector → Task Runtime: collects episodes
- Policy Evaluator → Task Runtime: runs policies
- Policy Evaluator → Robot Policy: requests actions
- Robot Policy → Task Runtime: returns actions
- Policy Evaluator → XPolicyLab: uses deployment
- Instruction Generator → Task Metadata: renders instructions
- Asset Renderer → Task Metadata: visualizes assets
