How amusementclub/vs-mlrt works
vs-mlrt provides VapourSynth machine-learning filters with backend-specific inference plugins and a Python wrapper that presents bundled models through a unified backend-selection interface. The typical workflow supplies video clips and a model to a VapourSynth filter, which prepares tiled input, runs inference, and returns processed frames. README-documented backends cover OpenVINO, ONNX Runtime, TensorRT, MIGraphX, and NCNN/Vulkan. Runtime excerpts show frame processing and tiling for several plugins; wiring for unsampled backends and wrapper internals is based on README descriptions.
Main entry points: vs_onnxruntime.cpp ONNX Runtime · vs_tensorrt.cpp TensorRT runtime · vs_ncnn.cpp NCNN Vulkan runtime · vs_openvino.cpp OpenVINO runtime
Components 21
User Interface
- Unified wrapper
scripts/vsmlrt.py - Video clips
- Model
- VapourSynth filter
- Processed frames
Inference Runtimes
- OpenVINO runtime
vsov/vs_openvino.cpp - ONNX Runtime
vsort/vs_onnxruntime.cpp - TensorRT runtime
vstrt/vs_tensorrt.cpp - MIGraphX runtime
vsmigx/vs_migraphx.cpp - NCNN Vulkan runtime
vsncnn/vs_ncnn.cpp
Shared Processing
- ONNX model
- Tile inference
vstrt/inference_helper.h - ONNX conversion
vsncnn/onnx2ncnn.cpp - Float16 conversion
common/convert_float_to_float16.cpp - ONNX utilities
common/onnx_utils.cpp
Other components
- VapourSynth user
- OpenVINO
- ONNX Runtime
- TensorRT
- MIGraphX
- NCNN / Vulkan
Connections 26
- VapourSynth user → Unified wrapper: selects backend
- VapourSynth user → VapourSynth filter: invokes filter
- Video clips → VapourSynth filter: supplies frames
- Model → VapourSynth filter: supplies model
- Unified wrapper → VapourSynth filter: configures runtime
- VapourSynth filter → OpenVINO runtime: dispatches inference
- VapourSynth filter → ONNX Runtime: dispatches inference
- VapourSynth filter → TensorRT runtime: dispatches inference
- VapourSynth filter → MIGraphX runtime: dispatches inference
- VapourSynth filter → NCNN Vulkan runtime: dispatches inference
- OpenVINO runtime → OpenVINO: runs inference
- ONNX Runtime → ONNX Runtime: runs inference
- TensorRT runtime → TensorRT: runs inference
- MIGraphX runtime → MIGraphX: runs inference
- NCNN Vulkan runtime → NCNN / Vulkan: runs inference
- ONNX model → ONNX conversion: converts model
- NCNN Vulkan runtime → ONNX conversion: converts ONNX
- ONNX model → ONNX utilities: uses utilities
- ONNX Runtime → ONNX utilities: uses utilities
- ONNX Runtime → Float16 conversion: converts precision
- TensorRT runtime → Tile inference: runs tiled inference
- OpenVINO runtime → Processed frames: returns frames
- ONNX Runtime → Processed frames: returns frames
- TensorRT runtime → Processed frames: returns frames
- MIGraphX runtime → Processed frames: returns frames
- NCNN Vulkan runtime → Processed frames: returns frames