How xiaomirobotics/xiaomi-robotics-u0 works
Xiaomi-Robotics-U0 is a multimodal world foundation model with inference and distributed training code. Inference composes task, runtime, and profile settings, then runs AR or FlashAR generation; documented tasks cover T2I, X2I, scene generation, transfer, and interleaved text/video. FlashAR also offers a vLLM batching path. Training uses FSDP, packed data, and Ulysses sequence parallelism. Several inference entry points and training subsystems are represented from the README and tree; their internal wiring is not fully sampled.
Main entry points: scripts Inference Entrypoints · train.py Training Loop · model.py FlashAR Model · modeling_unis.py Training Model
Components 23
Inference
- Config Composition
inference/configs/compose.py - Task Configs
inference/configs/tasks - Inference Entrypoints
inference/scripts
AR Model
- AR Generation
inference/xr_u0_ar/generation.py - Sequence Generation
inference/xr_u0_ar/sequence_generation.py - UNIS Model
inference/xr_u0_ar/modeling_unis.py - UNIS Config
inference/xr_u0_ar/configuration_unis.py - Vision Tokenizer
inference/xr_u0_ar/vision_tokenizer/ibq.py - Depth Preprocessing
inference/xr_u0_ar/depth_preprocess.py
FlashAR
- FlashAR Model
inference/xr_u0_flashar/model.py - FlashAR Outputs
inference/xr_u0_flashar/outputs.py - vLLM API
inference/xr_u0_flashar/vllm/api.py - vLLM Runtime
inference/xr_u0_flashar/vllm/runtime.py
Training
- Training Data
training/src/wm_fsdp/data - Training Loop
training/src/wm_fsdp/train/train.py - Training Model
training/src/wm_fsdp/models/modeling_unis.py - Training Config
training/src/wm_fsdp/models/configuration_unis.py - Training Tokenizers
training/src/wm_fsdp/tokenizers - FSDP Utilities
training/src/wm_fsdp/train/fsdp_utils.py - Ulysses Parallelism
training/src/wm_fsdp/train/ulysses.py
Other components
- User
- Model Weights
- Generated Result
Connections 23
- User → Inference Entrypoints: invokes
README.md - Config Composition → Task Configs: loads task
inference/configs/compose.py - Inference Entrypoints → Config Composition: uses settings
- Inference Entrypoints → AR Generation: runs AR
- Inference Entrypoints → FlashAR Model: runs FlashAR
inference/scripts/inference_flashar.py - AR Generation → UNIS Model: generates with
- UNIS Model → UNIS Config: uses config
inference/xr_u0_ar/modeling_unis.py - Vision Tokenizer → Model Weights: loads tokenizer weights
- Inference Entrypoints → Depth Preprocessing: preprocesses depth
inference/scripts/inference_flashar.py - FlashAR Model → UNIS Model: wraps backbone
inference/xr_u0_flashar/model.py - FlashAR Model → FlashAR Outputs: produces outputs
- vLLM API → vLLM Runtime: dispatches runtime
- User → vLLM API: requests batched generation
- vLLM Runtime → FlashAR Model: uses FlashAR model
inference/xr_u0_flashar/vllm/runtime.py - Inference Entrypoints → Generated Result: returns generation
- User → Training Loop: starts training
README.md - Training Loop → Training Data: loads batches
training/src/wm_fsdp/train/train.py - Training Loop → Training Model: trains model
training/src/wm_fsdp/train/train.py - Training Model → Training Config: uses config
training/src/wm_fsdp/models/modeling_unis.py - Training Loop → FSDP Utilities: uses utilities
training/src/wm_fsdp/train/train.py - Training Loop → Ulysses Parallelism: uses parallelism
training/src/wm_fsdp/train/train.py - Training Model → Training Tokenizers: uses tokenizer
- Training Loop → Model Weights: saves checkpoints