How starvla/starvla-wbc works
StarVLA-WBC trains and serves vision-language-action policies that produce humanoid joint-reference chunks; controller adapters translate those into controller-native inputs, while the external WBC stack retains real-time control and safety. The README documents the larger model and controller capabilities; sampled sources confirm training setup, dataset transforms, checkpoint normalization, and portions of the client and BFM-Zero runtime. Unsampled subsystem wiring is represented only where the README states the interface or capability.
Main entry points: framework VLA frameworks · policy_wrapper.py Policy wrapper · server_policy.py Policy server · websocket_policy_client.py WebSocket client
Components 22
Data preparation
- LeRobot datasets
starVLA/dataloader/gr00t_lerobot/datasets.py - Modality transforms
starVLA/dataloader/gr00t_lerobot/transform/state_action.py - VLM datasets
starVLA/dataloader/vlm_datasets.py
Models and training
- VLA trainer
starVLA/training/train_starvla.py - VLA frameworks
starVLA/model/framework - Qwen VLMs
starVLA/model/modules/vlm - World models
starVLA/model/modules/world_model - Action heads
starVLA/model/modules/action_model
Policy serving
- Policy normalization
deployment/model_server/policy_norm_processor.py - Policy wrapper
deployment/model_server/policy_wrapper.py - Policy server
deployment/model_server/server_policy.py - WebSocket transport
deployment/model_server/tools/websocket_policy_server.py - ZMQ transport
deployment/model_server/tools/zmq_policy_server.py - WebSocket client
deployment/model_server/tools/websocket_policy_client.py
Humanoid control
- Humanoid command interface
starVLA/controller/humanoid.py - Controller adapters
starVLA/controller - BFM-Zero runtime
starVLA/controller/bfm_zero.py - Controller backend
starVLA/controller/simple_native.py
Other components
- Policy consumer
- Camera, language, state
- External WBC
- Robot or simulator
Connections 21
- LeRobot datasets → Modality transforms: transform samples
- VLM datasets → VLA trainer: provide batches
- VLA trainer → VLA frameworks: build and train
- VLA frameworks → Qwen VLMs: use backbone
- VLA frameworks → World models: use conditioning
- VLA frameworks → Action heads: predict actions
- Camera, language, state → WebSocket client: submit observations
- WebSocket client → WebSocket transport: request policy
- WebSocket transport → Policy server: dispatch request
- ZMQ transport → Policy server: dispatch request
- Policy server → Policy wrapper: invoke policy
- Policy wrapper → VLA frameworks: run inference
- Policy wrapper → Policy normalization: process actions
- Policy server → WebSocket client: return actions
- Policy consumer → WebSocket client: request action
- Policy wrapper → Humanoid command interface: decode action chunk
- Humanoid command interface → Controller adapters: translate command
- Controller adapters → Controller backend: provide native input
- BFM-Zero runtime → Controller backend: produce joint targets
- Controller backend → External WBC: return joint targets
- External WBC → Robot or simulator: control