# huixi-ai/rhinovla architecture

> Architecture diagram of the GitHub repository huixi-ai/rhinovla, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/huixi-ai/rhinovla
- Repository: https://github.com/huixi-ai/rhinovla

## Overview

RhinoVLA is a cross-embodiment vision-language-action system with a fine-tuning stack and an RPU inference backend. The main workflow maps LeRobot demonstrations into the 72D contract, trains a Qwen3-VL-plus-action-expert policy, then uses prepared artifacts and normalization data for robot inference. The graph also includes documented evaluation and visualization capabilities. Sampled excerpts support the principal data and model relationships; inference policy internals and several training subsystems are represented only at their documented enclosing paths.

Main entry points: [`rhinovla/training/train.py`](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/train.py) (Training orchestration), [`rhinovla/inference/rpu_backend/executor.py`](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/inference/rpu_backend/executor.py) (RPU executor), [`rhinovla/dataloader/lerobot_native72.py`](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/dataloader/lerobot_native72.py) (LeRobot adapter), [`datasets/compute_norm_json.py`](https://github.com/huixi-ai/rhinovla/blob/main/datasets/compute_norm_json.py) (Norm computation)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_mapping["72D mapping"]
  node_norm_compute["Norm computation"]
  node_loader["LeRobot adapter"]
end

subgraph group_model["Policy model"]
  node_policy["RhinoVLA framework<br/>[RhinoVLA.py]"]
  node_qwen["Qwen3-VL encoder<br/>[qwen.py]"]
  node_processor_assets["Qwen processor assets"]
  node_action_expert["Action expert<br/>[action_expert.py]"]
end

subgraph group_training["Training and evaluation"]
  node_train_config["Fine-tune config"]
  node_trainer["Training orchestration<br/>[train.py]"]
  node_trainer_tools["Checkpoint and optimizer tools<br/>[trainer_tools.py]"]
  node_eval["Evaluation and metrics<br/>[eval.py]"]
  node_visualize["Training visualizations<br/>[visualize.py]"]
end

subgraph group_inference["RPU inference"]
  node_infer_cli["Inference CLI<br/>[cli.py]"]
  node_executor["RPU executor<br/>[executor.py]"]
  node_norm_runtime["Inference normalization<br/>[norm.py]"]
end

node_engineer(("Model engineer"))
node_dataset[("LeRobot demonstrations")]
node_norm_file["Normalization stats<br/>[norm.json]"]
node_checkpoint["Trained checkpoint"]
node_operator(("Robot operator"))
node_rpu_policy["Prepared RPU policy"]
node_robot_inputs[("Robot cameras and state")]
node_robot_actions["Robot action interface"]

node_engineer -->|"computes stats"| node_norm_compute
node_dataset -->|"reads demonstrations"| node_norm_compute
node_mapping -->|"defines slots"| node_norm_compute
node_norm_compute -->|"writes stats"| node_norm_file
node_engineer -->|"starts fine-tuning"| node_trainer
node_train_config -->|"configures run"| node_trainer
node_trainer -->|"builds batches"| node_loader
node_loader -->|"reads samples"| node_dataset
node_loader -->|"maps state and actions"| node_mapping
node_loader -->|"checks statistics"| node_norm_file
node_trainer -->|"trains policy"| node_policy
node_policy -->|"encodes vision and language"| node_qwen
node_qwen -->|"loads processor"| node_processor_assets
node_policy -->|"predicts actions"| node_action_expert
node_trainer -->|"manages optimization"| node_trainer_tools
node_trainer -->|"evaluates policy"| node_eval
node_trainer -->|"logs visualizations"| node_visualize
node_trainer_tools -->|"saves checkpoint"| node_checkpoint
node_operator -->|"requests inference"| node_infer_cli
node_infer_cli -->|"configures executor"| node_executor
node_executor -->|"loads prepared policy"| node_rpu_policy
node_executor -->|"loads normalization"| node_norm_runtime
node_norm_runtime -->|"expands native slots"| node_mapping
node_executor -->|"submits observations"| node_robot_inputs
node_executor -->|"returns action chunks"| node_robot_actions
node_checkpoint -.->|"optional checkpoint"| node_executor

click node_dataset "https://github.com/huixi-ai/rhinovla/tree/main/datasets/example_lerobot_v3"
click node_mapping "https://github.com/huixi-ai/rhinovla/blob/main/configs/data_mappings/demo_lerobot_native72.yaml"
click node_norm_compute "https://github.com/huixi-ai/rhinovla/blob/main/datasets/compute_norm_json.py"
click node_norm_file "https://github.com/huixi-ai/rhinovla/blob/main/datasets/example_lerobot_v3/meta/norm.json"
click node_loader "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/dataloader/lerobot_native72.py"
click node_train_config "https://github.com/huixi-ai/rhinovla/blob/main/configs/training/demo_full_finetune.yaml"
click node_trainer "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/train.py"
click node_policy "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/model/framework/RhinoVLA.py"
click node_qwen "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/model/modules/qwen.py"
click node_processor_assets "https://github.com/huixi-ai/rhinovla/tree/main/rhinovla/assets/qwen3_vl_processor"
click node_action_expert "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/model/modules/action_expert.py"
click node_trainer_tools "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/trainer_utils/trainer_tools.py"
click node_eval "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/eval.py"
click node_visualize "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/visualize.py"
click node_infer_cli "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/inference/rpu_backend/cli.py"
click node_executor "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/inference/rpu_backend/executor.py"
click node_norm_runtime "https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/inference/rpu_backend/norm.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_mapping,node_norm_compute,node_loader toneBlue
class node_policy,node_qwen,node_processor_assets,node_action_expert,node_dataset,node_robot_inputs toneAmber
class node_train_config,node_trainer,node_trainer_tools,node_eval,node_visualize toneMint
class node_infer_cli,node_executor,node_norm_runtime toneRose
class node_engineer,node_checkpoint,node_operator,node_rpu_policy,node_robot_actions toneIndigo
class node_norm_file toneTeal
```

## Components

### Data preparation

- [72D mapping](https://github.com/huixi-ai/rhinovla/blob/main/configs/data_mappings/demo_lerobot_native72.yaml): `configs/data_mappings/demo_lerobot_native72.yaml`
- [Norm computation](https://github.com/huixi-ai/rhinovla/blob/main/datasets/compute_norm_json.py): `datasets/compute_norm_json.py`
- [LeRobot adapter](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/dataloader/lerobot_native72.py): `rhinovla/dataloader/lerobot_native72.py`

### Policy model

- [RhinoVLA framework](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/model/framework/RhinoVLA.py): `rhinovla/model/framework/RhinoVLA.py`
- [Qwen3-VL encoder](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/model/modules/qwen.py): `rhinovla/model/modules/qwen.py`
- [Qwen processor assets](https://github.com/huixi-ai/rhinovla/tree/main/rhinovla/assets/qwen3_vl_processor): `rhinovla/assets/qwen3_vl_processor`
- [Action expert](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/model/modules/action_expert.py): `rhinovla/model/modules/action_expert.py`

### Training and evaluation

- [Fine-tune config](https://github.com/huixi-ai/rhinovla/blob/main/configs/training/demo_full_finetune.yaml): `configs/training/demo_full_finetune.yaml`
- [Training orchestration](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/train.py): `rhinovla/training/train.py`
- [Checkpoint and optimizer tools](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/trainer_utils/trainer_tools.py): `rhinovla/training/trainer_utils/trainer_tools.py`
- [Evaluation and metrics](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/eval.py): `rhinovla/training/eval.py`
- [Training visualizations](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/training/visualize.py): `rhinovla/training/visualize.py`

### RPU inference

- [Inference CLI](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/inference/rpu_backend/cli.py): `rhinovla/inference/rpu_backend/cli.py`
- [RPU executor](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/inference/rpu_backend/executor.py): `rhinovla/inference/rpu_backend/executor.py`
- [Inference normalization](https://github.com/huixi-ai/rhinovla/blob/main/rhinovla/inference/rpu_backend/norm.py): `rhinovla/inference/rpu_backend/norm.py`

### Other components

- **Model engineer**
- [LeRobot demonstrations](https://github.com/huixi-ai/rhinovla/tree/main/datasets/example_lerobot_v3): `datasets/example_lerobot_v3`
- [Normalization stats](https://github.com/huixi-ai/rhinovla/blob/main/datasets/example_lerobot_v3/meta/norm.json): `datasets/example_lerobot_v3/meta/norm.json`
- **Trained checkpoint**
- **Robot operator**
- **Prepared RPU policy**
- **Robot cameras and state**
- **Robot action interface**

## Connections

- Model engineer → Norm computation: computes stats
- LeRobot demonstrations → Norm computation: reads demonstrations
- 72D mapping → Norm computation: defines slots
- Norm computation → Normalization stats: writes stats
- Model engineer → Training orchestration: starts fine-tuning
- Fine-tune config → Training orchestration: configures run
- Training orchestration → LeRobot adapter: builds batches
- LeRobot adapter → LeRobot demonstrations: reads samples
- LeRobot adapter → 72D mapping: maps state and actions
- LeRobot adapter → Normalization stats: checks statistics
- Training orchestration → RhinoVLA framework: trains policy
- RhinoVLA framework → Qwen3-VL encoder: encodes vision and language
- Qwen3-VL encoder → Qwen processor assets: loads processor
- RhinoVLA framework → Action expert: predicts actions
- Training orchestration → Checkpoint and optimizer tools: manages optimization
- Training orchestration → Evaluation and metrics: evaluates policy
- Training orchestration → Training visualizations: logs visualizations
- Checkpoint and optimizer tools → Trained checkpoint: saves checkpoint
- Robot operator → Inference CLI: requests inference
- Inference CLI → RPU executor: configures executor
- RPU executor → Prepared RPU policy: loads prepared policy
- RPU executor → Inference normalization: loads normalization
- Inference normalization → 72D mapping: expands native slots
- RPU executor → Robot cameras and state: submits observations
- RPU executor → Robot action interface: returns action chunks
- Trained checkpoint → RPU executor: optional checkpoint