# amusementclub/vs-mlrt architecture

> Architecture diagram of the GitHub repository amusementclub/vs-mlrt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amusementclub/vs-mlrt
- Repository: https://github.com/amusementclub/vs-mlrt

## Overview

vs-mlrt provides VapourSynth machine-learning filters with backend-specific inference plugins and a Python wrapper that presents bundled models through a unified backend-selection interface. The typical workflow supplies video clips and a model to a VapourSynth filter, which prepares tiled input, runs inference, and returns processed frames. README-documented backends cover OpenVINO, ONNX Runtime, TensorRT, MIGraphX, and NCNN/Vulkan. Runtime excerpts show frame processing and tiling for several plugins; wiring for unsampled backends and wrapper internals is based on README descriptions.

Main entry points: [`vsort/vs_onnxruntime.cpp`](https://github.com/amusementclub/vs-mlrt/blob/master/vsort/vs_onnxruntime.cpp) (ONNX Runtime), [`vstrt/vs_tensorrt.cpp`](https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/vs_tensorrt.cpp) (TensorRT runtime), [`vsncnn/vs_ncnn.cpp`](https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/vs_ncnn.cpp) (NCNN Vulkan runtime), [`vsov/vs_openvino.cpp`](https://github.com/amusementclub/vs-mlrt/blob/master/vsov/vs_openvino.cpp) (OpenVINO runtime)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User Interface"]
  node_python["Unified wrapper<br/>[vsmlrt.py]"]
  node_clips["Video clips"]
  node_model["Model"]
  node_filter["VapourSynth filter"]
  node_processed["Processed frames"]
end

subgraph group_runtime["Inference Runtimes"]
  node_openvino["OpenVINO runtime<br/>[vs_openvino.cpp]"]
  node_onnxruntime["ONNX Runtime<br/>[vs_onnxruntime.cpp]"]
  node_tensorrt["TensorRT runtime<br/>[vs_tensorrt.cpp]"]
  node_migx["MIGraphX runtime<br/>[vs_migraphx.cpp]"]
  node_ncnn["NCNN Vulkan runtime<br/>[vs_ncnn.cpp]"]
end

subgraph group_shared["Shared Processing"]
  node_onnx["ONNX model"]
  node_tile["Tile inference<br/>[inference_helper.h]"]
  node_conversion["ONNX conversion<br/>[onnx2ncnn.cpp]"]
  node_float16["Float16 conversion"]
  node_onnx_utils["ONNX utilities<br/>[onnx_utils.cpp]"]
end

node_user(("VapourSynth user"))
node_openvino_service["OpenVINO"]
node_ort_service["ONNX Runtime"]
node_trt_service["TensorRT"]
node_migx_service["MIGraphX"]
node_ncnn_service["NCNN / Vulkan"]

node_user -->|"selects backend"| node_python
node_user -->|"invokes filter"| node_filter
node_clips -->|"supplies frames"| node_filter
node_model -->|"supplies model"| node_filter
node_python -->|"configures runtime"| node_filter
node_filter -->|"dispatches inference"| node_openvino
node_filter -->|"dispatches inference"| node_onnxruntime
node_filter -->|"dispatches inference"| node_tensorrt
node_filter -->|"dispatches inference"| node_migx
node_filter -->|"dispatches inference"| node_ncnn
node_openvino -->|"runs inference"| node_openvino_service
node_onnxruntime -->|"runs inference"| node_ort_service
node_tensorrt -->|"runs inference"| node_trt_service
node_migx -->|"runs inference"| node_migx_service
node_ncnn -->|"runs inference"| node_ncnn_service
node_onnx -.->|"converts model"| node_conversion
node_ncnn -->|"converts ONNX"| node_conversion
node_onnx -.->|"uses utilities"| node_onnx_utils
node_onnxruntime -->|"uses utilities"| node_onnx_utils
node_onnxruntime -->|"converts precision"| node_float16
node_tensorrt -->|"runs tiled inference"| node_tile
node_openvino -->|"returns frames"| node_processed
node_onnxruntime -->|"returns frames"| node_processed
node_tensorrt -->|"returns frames"| node_processed
node_migx -->|"returns frames"| node_processed
node_ncnn -->|"returns frames"| node_processed

click node_python "https://github.com/amusementclub/vs-mlrt/blob/master/scripts/vsmlrt.py"
click node_openvino "https://github.com/amusementclub/vs-mlrt/blob/master/vsov/vs_openvino.cpp"
click node_onnxruntime "https://github.com/amusementclub/vs-mlrt/blob/master/vsort/vs_onnxruntime.cpp"
click node_tensorrt "https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/vs_tensorrt.cpp"
click node_migx "https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/vs_migraphx.cpp"
click node_ncnn "https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/vs_ncnn.cpp"
click node_tile "https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/inference_helper.h"
click node_conversion "https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/onnx2ncnn.cpp"
click node_float16 "https://github.com/amusementclub/vs-mlrt/blob/master/common/convert_float_to_float16.cpp"
click node_onnx_utils "https://github.com/amusementclub/vs-mlrt/blob/master/common/onnx_utils.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_python,node_clips,node_model,node_filter,node_processed,node_user toneBlue
class node_openvino,node_onnxruntime,node_tensorrt,node_migx,node_ncnn toneAmber
class node_onnx,node_tile,node_conversion,node_float16,node_onnx_utils toneMint
class node_openvino_service,node_ort_service,node_trt_service,node_migx_service,node_ncnn_service toneIndigo
```

## Components

### User Interface

- [Unified wrapper](https://github.com/amusementclub/vs-mlrt/blob/master/scripts/vsmlrt.py): `scripts/vsmlrt.py`
- **Video clips**
- **Model**
- **VapourSynth filter**
- **Processed frames**

### Inference Runtimes

- [OpenVINO runtime](https://github.com/amusementclub/vs-mlrt/blob/master/vsov/vs_openvino.cpp): `vsov/vs_openvino.cpp`
- [ONNX Runtime](https://github.com/amusementclub/vs-mlrt/blob/master/vsort/vs_onnxruntime.cpp): `vsort/vs_onnxruntime.cpp`
- [TensorRT runtime](https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/vs_tensorrt.cpp): `vstrt/vs_tensorrt.cpp`
- [MIGraphX runtime](https://github.com/amusementclub/vs-mlrt/blob/master/vsmigx/vs_migraphx.cpp): `vsmigx/vs_migraphx.cpp`
- [NCNN Vulkan runtime](https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/vs_ncnn.cpp): `vsncnn/vs_ncnn.cpp`

### Shared Processing

- **ONNX model**
- [Tile inference](https://github.com/amusementclub/vs-mlrt/blob/master/vstrt/inference_helper.h): `vstrt/inference_helper.h`
- [ONNX conversion](https://github.com/amusementclub/vs-mlrt/blob/master/vsncnn/onnx2ncnn.cpp): `vsncnn/onnx2ncnn.cpp`
- [Float16 conversion](https://github.com/amusementclub/vs-mlrt/blob/master/common/convert_float_to_float16.cpp): `common/convert_float_to_float16.cpp`
- [ONNX utilities](https://github.com/amusementclub/vs-mlrt/blob/master/common/onnx_utils.cpp): `common/onnx_utils.cpp`

### Other components

- **VapourSynth user**
- **OpenVINO**
- **ONNX Runtime**
- **TensorRT**
- **MIGraphX**
- **NCNN / Vulkan**

## Connections

- VapourSynth user → Unified wrapper: selects backend
- VapourSynth user → VapourSynth filter: invokes filter
- Video clips → VapourSynth filter: supplies frames
- Model → VapourSynth filter: supplies model
- Unified wrapper → VapourSynth filter: configures runtime
- VapourSynth filter → OpenVINO runtime: dispatches inference
- VapourSynth filter → ONNX Runtime: dispatches inference
- VapourSynth filter → TensorRT runtime: dispatches inference
- VapourSynth filter → MIGraphX runtime: dispatches inference
- VapourSynth filter → NCNN Vulkan runtime: dispatches inference
- OpenVINO runtime → OpenVINO: runs inference
- ONNX Runtime → ONNX Runtime: runs inference
- TensorRT runtime → TensorRT: runs inference
- MIGraphX runtime → MIGraphX: runs inference
- NCNN Vulkan runtime → NCNN / Vulkan: runs inference
- ONNX model → ONNX conversion: converts model
- NCNN Vulkan runtime → ONNX conversion: converts ONNX
- ONNX model → ONNX utilities: uses utilities
- ONNX Runtime → ONNX utilities: uses utilities
- ONNX Runtime → Float16 conversion: converts precision
- TensorRT runtime → Tile inference: runs tiled inference
- OpenVINO runtime → Processed frames: returns frames
- ONNX Runtime → Processed frames: returns frames
- TensorRT runtime → Processed frames: returns frames
- MIGraphX runtime → Processed frames: returns frames
- NCNN Vulkan runtime → Processed frames: returns frames