How zju-omniai/vla-corrector works
VLA-Corrector adds latent-dynamics monitoring and recovery guidance to action-chunked VLA inference. The documented workflow extracts frozen visual features and executed actions, trains a lightweight residual corrector, then monitors observations during robot execution; persistent mismatch truncates queued actions and triggers Online Gradient Guidance for the recovery replan. The tree includes corresponding corrector, inference, policy, robot, and async-inference modules. The README describes their intended integration; sampled excerpts do not show the corrector-to-policy wiring, so those documented links are distinguished from directly evidenced transport behavior.
Main entry points: robot_client.py Robot client · policy_server.py Policy server · services.proto Inference transport · circuit_breaker.py Interrupt control
Components 17
Data and training
- Dynamics dataset
src/siglip_dynamics/dataset.py - Feature extraction
src/siglip_dynamics/extract.py - Dynamics corrector
src/siglip_dynamics/MLP.py - Corrector training
src/siglip_dynamics/train.py
Corrector inference
- VLA policy
src/lerobot/policies/factory.py - Vision monitor
src/siglip_dynamics/inference/safety_module.py - Online guidance
src/siglip_dynamics/inference/guidance_injector.py - Interrupt control
src/siglip_dynamics/inference/circuit_breaker.py - Policy server
src/lerobot/async_inference/policy_server.py
Robot runtime
- Robot client
src/lerobot/async_inference/robot_client.py - Robot hardware
src/lerobot/robots/robot.py - Action queue
src/lerobot/policies/rtc/action_queue.py - Camera input
src/lerobot/cameras/camera.py
LeRobot core
- Policy processors
src/lerobot/processor/pipeline.py - LeRobot datasets
src/lerobot/datasets/lerobot_dataset.py - Inference transport
src/lerobot/transport/services.proto
Other components
- Robot operator
Connections 23
- Dynamics dataset → Feature extraction: provides samples
- Feature extraction → Dynamics corrector: extracts features
- Dynamics dataset → Corrector training: loads samples
- Corrector training → Dynamics corrector: trains
- Robot operator → Robot client: starts control
- Robot client → Robot hardware: connects and acts
- Camera input → Robot client: supplies observations
- Robot client → Inference transport: sends observations
- Inference transport → Policy server: delivers observations
- Policy server → VLA policy: requests actions
- VLA policy → Policy server: returns action chunks
- Policy server → Inference transport: sends actions
- Inference transport → Robot client: delivers actions
- Robot client → Action queue: aggregates actions
- Robot client → Robot hardware: executes queued actions
- Robot client → Vision monitor: monitors observations
- Dynamics corrector → Vision monitor: predicts latent residuals
- Vision monitor → Interrupt control: signals persistent drift
- Interrupt control → Action queue: truncates stale actions
- Interrupt control → Online guidance: triggers recovery guidance
- Online guidance → Policy server: guides recovery replan
- Policy processors → VLA policy: processes policy inputs
- LeRobot datasets → Dynamics dataset: provides dataset data