# openmoss/moss-vl architecture

> Architecture diagram of the GitHub repository openmoss/moss-vl, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/openmoss/moss-vl
- Repository: https://github.com/openmoss/moss-vl

## Overview

MOSS-VL is an open-weight video-language model project, with workflows for offline and real-time video inference, supervised fine-tuning, and checkpoint quantization. Users provide video, images, prompts, or training data; inference scripts prepare media and run the model, while training and quantization scripts produce adapted or smaller checkpoints. The graph includes the shared device abstraction and the documented FP8, NF4, and KV-cache quantization branches. Source excerpts are partial, so only relationships shown in excerpts, the source index, or README documentation are drawn.

Main entry points: [`inference/realtime/run_online_inference.py`](https://github.com/openmoss/moss-vl/blob/main/inference/realtime/run_online_inference.py) (Realtime inference), [`finetune/train.py`](https://github.com/openmoss/moss-vl/blob/main/finetune/train.py) (SFT trainer), [`inference/offline/run_inference.py`](https://github.com/openmoss/moss-vl/blob/main/inference/offline/run_inference.py) (Offline inference), [`quantization/quantize_fp8_dynamic.py`](https://github.com/openmoss/moss-vl/blob/main/quantization/quantize_fp8_dynamic.py) (FP8 quantizer)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_inference["Inference"]
  node_offline["Offline inference<br/>[run_inference.py]"]
  node_realtime["Realtime inference"]
  node_video_sources["Video sources<br/>[video_sources.py]"]
end

subgraph group_training["Fine-tuning"]
  node_train_args["Training options<br/>[arguments.py]"]
  node_train_data["SFT dataset<br/>[data.py]"]
  node_trainer["SFT trainer<br/>[train.py]"]
end

subgraph group_quantization["Quantization"]
  node_fp8["FP8 quantizer"]
  node_nf4["NF4 quantizer"]
  node_kv8["KV8 packager<br/>[package_kv8_hqq.py]"]
end

subgraph group_runtime["Runtime support"]
  node_device["Device utilities<br/>[device_utils.py]"]
end

node_user(("User"))
node_model["MOSS-VL checkpoints"]
node_quantized_checkpoint["Quantized checkpoint"]

node_user -->|"runs offline"| node_offline
node_user -->|"starts streaming"| node_realtime
node_user -->|"fine-tunes"| node_trainer
node_offline -.->|"runs inference"| node_model
node_realtime -.->|"runs inference"| node_model
node_realtime -->|"builds frames"| node_video_sources
node_trainer -->|"uses options"| node_train_args
node_trainer -->|"loads dataset"| node_train_data
node_trainer -.->|"fine-tunes checkpoint"| node_model
node_user -->|"quantizes checkpoint"| node_fp8
node_user -->|"quantizes checkpoint"| node_nf4
node_user -->|"packages KV cache"| node_kv8
node_fp8 -.->|"writes checkpoint"| node_quantized_checkpoint
node_nf4 -.->|"writes checkpoint"| node_quantized_checkpoint
node_kv8 -.->|"packages checkpoint"| node_quantized_checkpoint
node_offline -->|"uses device helpers"| node_device
node_realtime -->|"uses device helpers"| node_device

click node_offline "https://github.com/openmoss/moss-vl/blob/main/inference/offline/run_inference.py"
click node_realtime "https://github.com/openmoss/moss-vl/blob/main/inference/realtime/run_online_inference.py"
click node_video_sources "https://github.com/openmoss/moss-vl/blob/main/inference/realtime/video_sources.py"
click node_train_args "https://github.com/openmoss/moss-vl/blob/main/finetune/arguments.py"
click node_train_data "https://github.com/openmoss/moss-vl/blob/main/finetune/data.py"
click node_trainer "https://github.com/openmoss/moss-vl/blob/main/finetune/train.py"
click node_fp8 "https://github.com/openmoss/moss-vl/blob/main/quantization/quantize_fp8_dynamic.py"
click node_nf4 "https://github.com/openmoss/moss-vl/blob/main/quantization/quantize_nf4_keep_ends.py"
click node_kv8 "https://github.com/openmoss/moss-vl/blob/main/quantization/package_kv8_hqq.py"
click node_device "https://github.com/openmoss/moss-vl/blob/main/device_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_offline,node_realtime,node_video_sources,node_user toneBlue
class node_train_args,node_train_data,node_trainer toneAmber
class node_fp8,node_nf4,node_kv8 toneMint
class node_device toneRose
class node_model,node_quantized_checkpoint toneIndigo
```

## Components

### Inference

- [Offline inference](https://github.com/openmoss/moss-vl/blob/main/inference/offline/run_inference.py): `inference/offline/run_inference.py`
- [Realtime inference](https://github.com/openmoss/moss-vl/blob/main/inference/realtime/run_online_inference.py): `inference/realtime/run_online_inference.py`
- [Video sources](https://github.com/openmoss/moss-vl/blob/main/inference/realtime/video_sources.py): `inference/realtime/video_sources.py`

### Fine-tuning

- [Training options](https://github.com/openmoss/moss-vl/blob/main/finetune/arguments.py): `finetune/arguments.py`
- [SFT dataset](https://github.com/openmoss/moss-vl/blob/main/finetune/data.py): `finetune/data.py`
- [SFT trainer](https://github.com/openmoss/moss-vl/blob/main/finetune/train.py): `finetune/train.py`

### Quantization

- [FP8 quantizer](https://github.com/openmoss/moss-vl/blob/main/quantization/quantize_fp8_dynamic.py): `quantization/quantize_fp8_dynamic.py`
- [NF4 quantizer](https://github.com/openmoss/moss-vl/blob/main/quantization/quantize_nf4_keep_ends.py): `quantization/quantize_nf4_keep_ends.py`
- [KV8 packager](https://github.com/openmoss/moss-vl/blob/main/quantization/package_kv8_hqq.py): `quantization/package_kv8_hqq.py`

### Runtime support

- [Device utilities](https://github.com/openmoss/moss-vl/blob/main/device_utils.py): `device_utils.py`

### Other components

- **User**
- **MOSS-VL checkpoints**
- **Quantized checkpoint**

## Connections

- User → Offline inference: runs offline (evidence: `README.md`)
- User → Realtime inference: starts streaming (evidence: `README.md`)
- User → SFT trainer: fine-tunes (evidence: `README.md`)
- Offline inference → MOSS-VL checkpoints: runs inference
- Realtime inference → MOSS-VL checkpoints: runs inference
- Realtime inference → Video sources: builds frames (evidence: `inference/realtime/run_online_inference.py`)
- SFT trainer → Training options: uses options (evidence: `finetune/train.py`)
- SFT trainer → SFT dataset: loads dataset (evidence: `finetune/train.py`)
- SFT trainer → MOSS-VL checkpoints: fine-tunes checkpoint
- User → FP8 quantizer: quantizes checkpoint (evidence: `README.md`)
- User → NF4 quantizer: quantizes checkpoint (evidence: `README.md`)
- User → KV8 packager: packages KV cache (evidence: `README.md`)
- FP8 quantizer → Quantized checkpoint: writes checkpoint
- NF4 quantizer → Quantized checkpoint: writes checkpoint
- KV8 packager → Quantized checkpoint: packages checkpoint
- Offline inference → Device utilities: uses device helpers (evidence: `inference/offline/run_inference.py`)
- Realtime inference → Device utilities: uses device helpers (evidence: `inference/realtime/run_online_inference.py`)