# zju-omniai/vla-corrector architecture

> Architecture diagram of the GitHub repository zju-omniai/vla-corrector, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zju-omniai/vla-corrector
- Repository: https://github.com/zju-omniai/vla-corrector

## Overview

VLA-Corrector adds latent-dynamics monitoring and recovery guidance to action-chunked VLA inference. The documented workflow extracts frozen visual features and executed actions, trains a lightweight residual corrector, then monitors observations during robot execution; persistent mismatch truncates queued actions and triggers Online Gradient Guidance for the recovery replan. The tree includes corresponding corrector, inference, policy, robot, and async-inference modules. The README describes their intended integration; sampled excerpts do not show the corrector-to-policy wiring, so those documented links are distinguished from directly evidenced transport behavior.

Main entry points: [`src/lerobot/async_inference/robot_client.py`](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/async_inference/robot_client.py) (Robot client), [`src/lerobot/async_inference/policy_server.py`](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/async_inference/policy_server.py) (Policy server), [`src/lerobot/transport/services.proto`](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/transport/services.proto) (Inference transport), [`src/siglip_dynamics/inference/circuit_breaker.py`](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/inference/circuit_breaker.py) (Interrupt control)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data_training["Data and training"]
  node_dataset["Dynamics dataset<br/>[dataset.py]"]
  node_extract["Feature extraction<br/>[extract.py]"]
  node_corrector_model["Dynamics corrector<br/>[MLP.py]"]
  node_training["Corrector training<br/>[train.py]"]
end

subgraph group_corrector_inference["Corrector inference"]
  node_frozen_policy["VLA policy<br/>[factory.py]"]
  node_safety["Vision monitor<br/>[safety_module.py]"]
  node_guidance["Online guidance"]
  node_circuit["Interrupt control<br/>[circuit_breaker.py]"]
  node_policy_server["Policy server<br/>[policy_server.py]"]
end

subgraph group_robot_runtime["Robot runtime"]
  node_robot_client["Robot client<br/>[robot_client.py]"]
  node_robot["Robot hardware<br/>[robot.py]"]
  node_action_queue["Action queue<br/>[action_queue.py]"]
  node_camera["Camera input<br/>[camera.py]"]
end

subgraph group_lerobot_core["LeRobot core"]
  node_processors["Policy processors<br/>[pipeline.py]"]
  node_dataset_store[("LeRobot datasets<br/>[lerobot_dataset.py]")]
  node_transport["Inference transport<br/>[services.proto]"]
end

node_robot_user(("Robot operator"))

node_dataset -->|"provides samples"| node_extract
node_extract -->|"extracts features"| node_corrector_model
node_dataset -->|"loads samples"| node_training
node_training -->|"trains"| node_corrector_model
node_robot_user -->|"starts control"| node_robot_client
node_robot_client -->|"connects and acts"| node_robot
node_camera -->|"supplies observations"| node_robot_client
node_robot_client -->|"sends observations"| node_transport
node_transport -->|"delivers observations"| node_policy_server
node_policy_server -->|"requests actions"| node_frozen_policy
node_frozen_policy -->|"returns action chunks"| node_policy_server
node_policy_server -->|"sends actions"| node_transport
node_transport -->|"delivers actions"| node_robot_client
node_robot_client -->|"aggregates actions"| node_action_queue
node_robot_client -->|"executes queued actions"| node_robot
node_robot_client -.->|"monitors observations"| node_safety
node_corrector_model -.->|"predicts latent residuals"| node_safety
node_safety -.->|"signals persistent drift"| node_circuit
node_circuit -.->|"truncates stale actions"| node_action_queue
node_circuit -.->|"triggers recovery guidance"| node_guidance
node_guidance -.->|"guides recovery replan"| node_policy_server
node_processors -.->|"processes policy inputs"| node_frozen_policy
node_dataset_store -.->|"provides dataset data"| node_dataset

click node_dataset "https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/dataset.py"
click node_extract "https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/extract.py"
click node_corrector_model "https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/MLP.py"
click node_training "https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/train.py"
click node_frozen_policy "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/policies/factory.py"
click node_safety "https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/inference/safety_module.py"
click node_guidance "https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/inference/guidance_injector.py"
click node_circuit "https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/inference/circuit_breaker.py"
click node_policy_server "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/async_inference/policy_server.py"
click node_robot_client "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/async_inference/robot_client.py"
click node_robot "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/robots/robot.py"
click node_action_queue "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/policies/rtc/action_queue.py"
click node_camera "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/cameras/camera.py"
click node_processors "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/processor/pipeline.py"
click node_dataset_store "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/datasets/lerobot_dataset.py"
click node_transport "https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/transport/services.proto"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset,node_extract,node_corrector_model,node_training toneBlue
class node_frozen_policy,node_safety,node_guidance,node_circuit,node_policy_server toneAmber
class node_robot_client,node_robot,node_action_queue,node_camera toneMint
class node_processors,node_dataset_store,node_transport toneRose
class node_robot_user toneIndigo
```

## Components

### Data and training

- [Dynamics dataset](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/dataset.py): `src/siglip_dynamics/dataset.py`
- [Feature extraction](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/extract.py): `src/siglip_dynamics/extract.py`
- [Dynamics corrector](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/MLP.py): `src/siglip_dynamics/MLP.py`
- [Corrector training](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/train.py): `src/siglip_dynamics/train.py`

### Corrector inference

- [VLA policy](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/policies/factory.py): `src/lerobot/policies/factory.py`
- [Vision monitor](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/inference/safety_module.py): `src/siglip_dynamics/inference/safety_module.py`
- [Online guidance](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/inference/guidance_injector.py): `src/siglip_dynamics/inference/guidance_injector.py`
- [Interrupt control](https://github.com/zju-omniai/vla-corrector/blob/main/src/siglip_dynamics/inference/circuit_breaker.py): `src/siglip_dynamics/inference/circuit_breaker.py`
- [Policy server](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/async_inference/policy_server.py): `src/lerobot/async_inference/policy_server.py`

### Robot runtime

- [Robot client](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/async_inference/robot_client.py): `src/lerobot/async_inference/robot_client.py`
- [Robot hardware](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/robots/robot.py): `src/lerobot/robots/robot.py`
- [Action queue](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/policies/rtc/action_queue.py): `src/lerobot/policies/rtc/action_queue.py`
- [Camera input](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/cameras/camera.py): `src/lerobot/cameras/camera.py`

### LeRobot core

- [Policy processors](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/processor/pipeline.py): `src/lerobot/processor/pipeline.py`
- [LeRobot datasets](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/datasets/lerobot_dataset.py): `src/lerobot/datasets/lerobot_dataset.py`
- [Inference transport](https://github.com/zju-omniai/vla-corrector/blob/main/src/lerobot/transport/services.proto): `src/lerobot/transport/services.proto`

### Other components

- **Robot operator**

## Connections

- Dynamics dataset → Feature extraction: provides samples
- Feature extraction → Dynamics corrector: extracts features
- Dynamics dataset → Corrector training: loads samples
- Corrector training → Dynamics corrector: trains
- Robot operator → Robot client: starts control
- Robot client → Robot hardware: connects and acts
- Camera input → Robot client: supplies observations
- Robot client → Inference transport: sends observations
- Inference transport → Policy server: delivers observations
- Policy server → VLA policy: requests actions
- VLA policy → Policy server: returns action chunks
- Policy server → Inference transport: sends actions
- Inference transport → Robot client: delivers actions
- Robot client → Action queue: aggregates actions
- Robot client → Robot hardware: executes queued actions
- Robot client → Vision monitor: monitors observations
- Dynamics corrector → Vision monitor: predicts latent residuals
- Vision monitor → Interrupt control: signals persistent drift
- Interrupt control → Action queue: truncates stale actions
- Interrupt control → Online guidance: triggers recovery guidance
- Online guidance → Policy server: guides recovery replan
- Policy processors → VLA policy: processes policy inputs
- LeRobot datasets → Dynamics dataset: provides dataset data