# physical-intelligence/openpi architecture

> Architecture diagram of the GitHub repository physical-intelligence/openpi, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/physical-intelligence/openpi
- Repository: https://github.com/physical-intelligence/openpi

## Overview

openpi is a robotics vision-language-action model project: it supports π₀, π₀-FAST, and π₀.₅, with base checkpoints, fine-tuning, and robot-specific inference. The principal runtime path is a robot-side client sending observations over WebSocket to a policy server, which adapts inputs, runs a model, and returns actions. Training is a separate data-to-model workflow. Some subsystem links rely on README claims and tree paths because their source was not sampled; uncertain call wiring is omitted.

Main entry points: [`src/openpi/policies`](https://github.com/physical-intelligence/openpi/tree/main/src/openpi/policies) (Robot policy adapters), [`src/openpi/training/data_loader.py`](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/training/data_loader.py) (Data loading), [`packages/openpi-client/src/openpi_client/runtime/runtime.py`](https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/runtime.py) (Agent runtime), [`src/openpi/models`](https://github.com/physical-intelligence/openpi/tree/main/src/openpi/models) (JAX model family)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_training["Training and data"]
  node_train_entry["Training entrypoints<br/>[train.py]"]
  node_torch_train["PyTorch training<br/>[train_pytorch.py]"]
  node_data["Data loading<br/>[data_loader.py]"]
  node_droid_data["DROID RLDS data"]
  node_transforms["Data transforms<br/>[transforms.py]"]
  node_stats["Normalization stats"]
  node_training_support["Optimization and checkpoints"]
end

subgraph group_models["VLA models"]
  node_jax_models["JAX model family"]
  node_pytorch_models["PyTorch π₀ models<br/>[pi0_pytorch.py]"]
  node_tokenizers["Action tokenizers<br/>[tokenizer.py]"]
  node_downloads["Checkpoint downloads<br/>[download.py]"]
end

subgraph group_inference["Policy inference"]
  node_serve_entry["Policy serving entrypoint<br/>[serve_policy.py]"]
  node_policy_config["Policy construction<br/>[policy_config.py]"]
  node_policy_adapters["Robot policy adapters"]
  node_aloha["ALOHA adapter<br/>[aloha_policy.py]"]
  node_droid["DROID adapter<br/>[droid_policy.py]"]
  node_libero["LIBERO adapter<br/>[libero_policy.py]"]
  node_ws_server["WebSocket policy server"]
end

subgraph group_client["Robot client runtime"]
  node_client_policy["WebSocket client policy"]
  node_runtime["Agent runtime<br/>[runtime.py]"]
  node_agent["Policy agent<br/>[policy_agent.py]"]
  node_environment["Environment interface<br/>[environment.py]"]
  node_subscriber["Action subscriber<br/>[subscriber.py]"]
end

node_robot(("Robot or application"))
node_dataset[("Robot datasets")]
node_gcs[("Checkpoint storage")]

node_dataset -->|"provides samples"| node_data
node_droid_data -->|"supplies DROID data"| node_data
node_data -->|"applies transforms"| node_transforms
node_train_entry -->|"loads batches"| node_data
node_torch_train -->|"loads batches"| node_data
node_train_entry -->|"trains and checkpoints"| node_training_support
node_torch_train -->|"trains and checkpoints"| node_training_support
node_stats -->|"computes statistics"| node_dataset
node_training_support -.->|"loads or saves checkpoints"| node_gcs
node_train_entry -->|"trains model"| node_jax_models
node_torch_train -->|"trains model"| node_pytorch_models
node_jax_models -->|"uses tokenizers"| node_tokenizers
node_pytorch_models -->|"uses tokenizers"| node_tokenizers
node_tokenizers -->|"fetches tokenizer assets"| node_downloads
node_serve_entry -->|"constructs policy"| node_policy_config
node_policy_config -->|"configures adapter"| node_policy_adapters
node_policy_adapters -->|"selects platform adapter"| node_aloha
node_policy_adapters -->|"selects platform adapter"| node_droid
node_policy_adapters -->|"selects platform adapter"| node_libero
node_serve_entry -->|"serves policy"| node_ws_server
node_ws_server -->|"invokes inference"| node_policy_adapters
node_policy_adapters -->|"runs model"| node_jax_models
node_policy_adapters -.->|"runs model"| node_pytorch_models
node_jax_models -->|"loads model assets"| node_downloads
node_robot -->|"provides observations"| node_runtime
node_runtime -->|"reads environment"| node_environment
node_runtime -->|"dispatches agent"| node_agent
node_agent -->|"requests inference"| node_client_policy
node_client_policy -->|"sends observations"| node_ws_server
node_ws_server -->|"returns actions"| node_client_policy
node_runtime -->|"dispatches actions"| node_subscriber
node_subscriber -->|"delivers actions"| node_robot

click node_train_entry "https://github.com/physical-intelligence/openpi/blob/main/scripts/train.py"
click node_torch_train "https://github.com/physical-intelligence/openpi/blob/main/scripts/train_pytorch.py"
click node_data "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/training/data_loader.py"
click node_droid_data "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/training/droid_rlds_dataset.py"
click node_transforms "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/transforms.py"
click node_stats "https://github.com/physical-intelligence/openpi/blob/main/scripts/compute_norm_stats.py"
click node_training_support "https://github.com/physical-intelligence/openpi/tree/main/src/openpi/training"
click node_jax_models "https://github.com/physical-intelligence/openpi/tree/main/src/openpi/models"
click node_pytorch_models "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/models_pytorch/pi0_pytorch.py"
click node_tokenizers "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/models/tokenizer.py"
click node_downloads "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/shared/download.py"
click node_serve_entry "https://github.com/physical-intelligence/openpi/blob/main/scripts/serve_policy.py"
click node_policy_config "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/policy_config.py"
click node_policy_adapters "https://github.com/physical-intelligence/openpi/tree/main/src/openpi/policies"
click node_aloha "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/aloha_policy.py"
click node_droid "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/droid_policy.py"
click node_libero "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/libero_policy.py"
click node_ws_server "https://github.com/physical-intelligence/openpi/blob/main/src/openpi/serving/websocket_policy_server.py"
click node_client_policy "https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/websocket_client_policy.py"
click node_runtime "https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/runtime.py"
click node_agent "https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/agents/policy_agent.py"
click node_environment "https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/environment.py"
click node_subscriber "https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/subscriber.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_entry,node_torch_train,node_data,node_droid_data,node_transforms,node_stats,node_training_support toneBlue
class node_jax_models,node_pytorch_models,node_tokenizers,node_downloads,node_dataset,node_gcs toneAmber
class node_serve_entry,node_policy_config,node_policy_adapters,node_aloha,node_droid,node_libero,node_ws_server toneMint
class node_client_policy,node_runtime,node_agent,node_environment,node_subscriber toneRose
class node_robot toneIndigo
```

## Components

### Training and data

- [Training entrypoints](https://github.com/physical-intelligence/openpi/blob/main/scripts/train.py): `scripts/train.py`
- [PyTorch training](https://github.com/physical-intelligence/openpi/blob/main/scripts/train_pytorch.py): `scripts/train_pytorch.py`
- [Data loading](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/training/data_loader.py): `src/openpi/training/data_loader.py`
- [DROID RLDS data](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/training/droid_rlds_dataset.py): `src/openpi/training/droid_rlds_dataset.py`
- [Data transforms](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/transforms.py): `src/openpi/transforms.py`
- [Normalization stats](https://github.com/physical-intelligence/openpi/blob/main/scripts/compute_norm_stats.py): `scripts/compute_norm_stats.py`
- [Optimization and checkpoints](https://github.com/physical-intelligence/openpi/tree/main/src/openpi/training): `src/openpi/training`

### VLA models

- [JAX model family](https://github.com/physical-intelligence/openpi/tree/main/src/openpi/models): `src/openpi/models`
- [PyTorch π₀ models](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/models_pytorch/pi0_pytorch.py): `src/openpi/models_pytorch/pi0_pytorch.py`
- [Action tokenizers](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/models/tokenizer.py): `src/openpi/models/tokenizer.py`
- [Checkpoint downloads](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/shared/download.py): `src/openpi/shared/download.py`

### Policy inference

- [Policy serving entrypoint](https://github.com/physical-intelligence/openpi/blob/main/scripts/serve_policy.py): `scripts/serve_policy.py`
- [Policy construction](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/policy_config.py): `src/openpi/policies/policy_config.py`
- [Robot policy adapters](https://github.com/physical-intelligence/openpi/tree/main/src/openpi/policies): `src/openpi/policies`
- [ALOHA adapter](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/aloha_policy.py): `src/openpi/policies/aloha_policy.py`
- [DROID adapter](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/droid_policy.py): `src/openpi/policies/droid_policy.py`
- [LIBERO adapter](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/policies/libero_policy.py): `src/openpi/policies/libero_policy.py`
- [WebSocket policy server](https://github.com/physical-intelligence/openpi/blob/main/src/openpi/serving/websocket_policy_server.py): `src/openpi/serving/websocket_policy_server.py`

### Robot client runtime

- [WebSocket client policy](https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/websocket_client_policy.py): `packages/openpi-client/src/openpi_client/websocket_client_policy.py`
- [Agent runtime](https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/runtime.py): `packages/openpi-client/src/openpi_client/runtime/runtime.py`
- [Policy agent](https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/agents/policy_agent.py): `packages/openpi-client/src/openpi_client/runtime/agents/policy_agent.py`
- [Environment interface](https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/environment.py): `packages/openpi-client/src/openpi_client/runtime/environment.py`
- [Action subscriber](https://github.com/physical-intelligence/openpi/blob/main/packages/openpi-client/src/openpi_client/runtime/subscriber.py): `packages/openpi-client/src/openpi_client/runtime/subscriber.py`

### Other components

- **Robot or application**
- **Robot datasets**
- **Checkpoint storage**

## Connections

- Robot datasets → Data loading: provides samples
- DROID RLDS data → Data loading: supplies DROID data
- Data loading → Data transforms: applies transforms
- Training entrypoints → Data loading: loads batches
- PyTorch training → Data loading: loads batches
- Training entrypoints → Optimization and checkpoints: trains and checkpoints
- PyTorch training → Optimization and checkpoints: trains and checkpoints
- Normalization stats → Robot datasets: computes statistics
- Optimization and checkpoints → Checkpoint storage: loads or saves checkpoints
- Training entrypoints → JAX model family: trains model
- PyTorch training → PyTorch π₀ models: trains model
- JAX model family → Action tokenizers: uses tokenizers
- PyTorch π₀ models → Action tokenizers: uses tokenizers
- Action tokenizers → Checkpoint downloads: fetches tokenizer assets
- Policy serving entrypoint → Policy construction: constructs policy
- Policy construction → Robot policy adapters: configures adapter
- Robot policy adapters → ALOHA adapter: selects platform adapter
- Robot policy adapters → DROID adapter: selects platform adapter
- Robot policy adapters → LIBERO adapter: selects platform adapter
- Policy serving entrypoint → WebSocket policy server: serves policy
- WebSocket policy server → Robot policy adapters: invokes inference
- Robot policy adapters → JAX model family: runs model
- Robot policy adapters → PyTorch π₀ models: runs model
- JAX model family → Checkpoint downloads: loads model assets
- Robot or application → Agent runtime: provides observations
- Agent runtime → Environment interface: reads environment
- Agent runtime → Policy agent: dispatches agent
- Policy agent → WebSocket client policy: requests inference
- WebSocket client policy → WebSocket policy server: sends observations
- WebSocket policy server → WebSocket client policy: returns actions
- Agent runtime → Action subscriber: dispatches actions
- Action subscriber → Robot or application: delivers actions
