# robbyant/lingbot-va architecture

> Architecture diagram of the GitHub repository robbyant/lingbot-va, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/robbyant/lingbot-va
- Repository: https://github.com/robbyant/lingbot-va

## Overview

LingBot-VA combines autoregressive video-world modeling and robot-action inference, with post-training on LeRobot datasets and standalone or remote inference. The principal evaluation loop sends observations and prompts from a simulator client to a WebSocket policy server, receives action chunks, executes them in the environment, and records outcomes. The model and scheduler form the inference core; training and image-to-video-action generation are separate supported paths. Some server and training wiring is inferred only from README capabilities and sampled excerpts.

Main entry points: [`wan_va/wan_va_server.py`](https://github.com/robbyant/lingbot-va/blob/main/wan_va/wan_va_server.py) (VA inference runtime), [`wan_va/utils/Simple_Remote_Infer/deploy/websocket_client_policy.py`](https://github.com/robbyant/lingbot-va/blob/main/wan_va/utils/Simple_Remote_Infer/deploy/websocket_client_policy.py) (WebSocket client), [`wan_va/train.py`](https://github.com/robbyant/lingbot-va/blob/main/wan_va/train.py) (Post-training loop), [`evaluation/robotwin/eval_polict_client_openpi.py`](https://github.com/robbyant/lingbot-va/blob/main/evaluation/robotwin/eval_polict_client_openpi.py) (RoboTwin client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_evaluation["Robot Evaluation"]
  node_robotwin["RoboTwin client"]
  node_libero["LIBERO client<br/>[client.py]"]
  node_results["Episodes and videos<br/>[client.py]"]
end

subgraph group_serving["Inference Serving"]
  node_wsclient["WebSocket client"]
  node_wsserver["WebSocket server"]
  node_va_server["VA inference runtime<br/>[wan_va_server.py]"]
  node_i2va["Image-to-video-action<br/>[wan_va_server.py]"]
  node_configs["Model configurations<br/>[shared_config.py]"]
end

subgraph group_model["World-Action Model"]
  node_transformer["Video-action transformer<br/>[model.py]"]
  node_scheduler["Flow-match scheduler<br/>[scheduler.py]"]
end

subgraph group_training["Post-Training"]
  node_dataset["Latent LeRobot data"]
  node_trainer["Post-training loop<br/>[train.py]"]
  node_distributed["Distributed sharding<br/>[fsdp.py]"]
end

node_simulators(("Robot simulators"))
node_lerobot[("LeRobot datasets")]
node_policy(("Robot policy caller"))

node_robotwin -->|"steps tasks"| node_simulators
node_libero -->|"steps tasks"| node_simulators
node_robotwin -->|"sends observations"| node_wsclient
node_libero -->|"sends observations"| node_wsclient
node_wsclient -->|"transfers requests"| node_wsserver
node_wsserver -->|"dispatches inference"| node_va_server
node_va_server -->|"predicts chunks"| node_transformer
node_va_server -->|"denoises outputs"| node_scheduler
node_va_server -->|"loads settings"| node_configs
node_va_server -->|"returns actions"| node_wsserver
node_wsclient -->|"returns actions"| node_robotwin
node_wsclient -->|"returns actions"| node_libero
node_robotwin -->|"records episodes"| node_results
node_libero -->|"saves videos"| node_results
node_policy -->|"requests generation"| node_i2va
node_i2va -->|"generates video-action"| node_transformer
node_i2va -->|"denoises outputs"| node_scheduler
node_trainer -->|"loads batches"| node_dataset
node_dataset -->|"reads episodes"| node_lerobot
node_trainer -->|"trains model"| node_transformer
node_trainer -->|"adds training noise"| node_scheduler
node_trainer -->|"shards model"| node_distributed

click node_robotwin "https://github.com/robbyant/lingbot-va/blob/main/evaluation/robotwin/eval_polict_client_openpi.py"
click node_libero "https://github.com/robbyant/lingbot-va/blob/main/evaluation/libero/client.py"
click node_results "https://github.com/robbyant/lingbot-va/blob/main/evaluation/libero/client.py"
click node_wsclient "https://github.com/robbyant/lingbot-va/blob/main/wan_va/utils/Simple_Remote_Infer/deploy/websocket_client_policy.py"
click node_wsserver "https://github.com/robbyant/lingbot-va/blob/main/wan_va/utils/Simple_Remote_Infer/deploy/websocket_policy_server.py"
click node_va_server "https://github.com/robbyant/lingbot-va/blob/main/wan_va/wan_va_server.py"
click node_i2va "https://github.com/robbyant/lingbot-va/blob/main/wan_va/wan_va_server.py"
click node_configs "https://github.com/robbyant/lingbot-va/blob/main/wan_va/configs/shared_config.py"
click node_transformer "https://github.com/robbyant/lingbot-va/blob/main/wan_va/modules/model.py"
click node_scheduler "https://github.com/robbyant/lingbot-va/blob/main/wan_va/utils/scheduler.py"
click node_dataset "https://github.com/robbyant/lingbot-va/blob/main/wan_va/dataset/lerobot_latent_dataset.py"
click node_trainer "https://github.com/robbyant/lingbot-va/blob/main/wan_va/train.py"
click node_distributed "https://github.com/robbyant/lingbot-va/blob/main/wan_va/distributed/fsdp.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_robotwin,node_libero,node_results toneBlue
class node_wsclient,node_wsserver,node_va_server,node_i2va,node_configs,node_lerobot toneAmber
class node_transformer,node_scheduler toneMint
class node_dataset,node_trainer,node_distributed toneRose
class node_simulators,node_policy toneIndigo
```

## Components

### Robot Evaluation

- [RoboTwin client](https://github.com/robbyant/lingbot-va/blob/main/evaluation/robotwin/eval_polict_client_openpi.py): `evaluation/robotwin/eval_polict_client_openpi.py`
- [LIBERO client](https://github.com/robbyant/lingbot-va/blob/main/evaluation/libero/client.py): `evaluation/libero/client.py`
- [Episodes and videos](https://github.com/robbyant/lingbot-va/blob/main/evaluation/libero/client.py): `evaluation/libero/client.py`

### Inference Serving

- [WebSocket client](https://github.com/robbyant/lingbot-va/blob/main/wan_va/utils/Simple_Remote_Infer/deploy/websocket_client_policy.py): `wan_va/utils/Simple_Remote_Infer/deploy/websocket_client_policy.py`
- [WebSocket server](https://github.com/robbyant/lingbot-va/blob/main/wan_va/utils/Simple_Remote_Infer/deploy/websocket_policy_server.py): `wan_va/utils/Simple_Remote_Infer/deploy/websocket_policy_server.py`
- [VA inference runtime](https://github.com/robbyant/lingbot-va/blob/main/wan_va/wan_va_server.py): `wan_va/wan_va_server.py`
- [Image-to-video-action](https://github.com/robbyant/lingbot-va/blob/main/wan_va/wan_va_server.py): `wan_va/wan_va_server.py`
- [Model configurations](https://github.com/robbyant/lingbot-va/blob/main/wan_va/configs/shared_config.py): `wan_va/configs/shared_config.py`

### World-Action Model

- [Video-action transformer](https://github.com/robbyant/lingbot-va/blob/main/wan_va/modules/model.py): `wan_va/modules/model.py`
- [Flow-match scheduler](https://github.com/robbyant/lingbot-va/blob/main/wan_va/utils/scheduler.py): `wan_va/utils/scheduler.py`

### Post-Training

- [Latent LeRobot data](https://github.com/robbyant/lingbot-va/blob/main/wan_va/dataset/lerobot_latent_dataset.py): `wan_va/dataset/lerobot_latent_dataset.py`
- [Post-training loop](https://github.com/robbyant/lingbot-va/blob/main/wan_va/train.py): `wan_va/train.py`
- [Distributed sharding](https://github.com/robbyant/lingbot-va/blob/main/wan_va/distributed/fsdp.py): `wan_va/distributed/fsdp.py`

### Other components

- **Robot simulators**
- **LeRobot datasets**
- **Robot policy caller**

## Connections

- RoboTwin client → Robot simulators: steps tasks
- LIBERO client → Robot simulators: steps tasks
- RoboTwin client → WebSocket client: sends observations
- LIBERO client → WebSocket client: sends observations
- WebSocket client → WebSocket server: transfers requests
- WebSocket server → VA inference runtime: dispatches inference
- VA inference runtime → Video-action transformer: predicts chunks
- VA inference runtime → Flow-match scheduler: denoises outputs
- VA inference runtime → Model configurations: loads settings
- VA inference runtime → WebSocket server: returns actions
- WebSocket client → RoboTwin client: returns actions
- WebSocket client → LIBERO client: returns actions
- RoboTwin client → Episodes and videos: records episodes
- LIBERO client → Episodes and videos: saves videos
- Robot policy caller → Image-to-video-action: requests generation
- Image-to-video-action → Video-action transformer: generates video-action
- Image-to-video-action → Flow-match scheduler: denoises outputs
- Post-training loop → Latent LeRobot data: loads batches
- Latent LeRobot data → LeRobot datasets: reads episodes
- Post-training loop → Video-action transformer: trains model
- Post-training loop → Flow-match scheduler: adds training noise
- Post-training loop → Distributed sharding: shards model