# tntwise/real-video-enhancer architecture

> Architecture diagram of the GitHub repository tntwise/real-video-enhancer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tntwise/real-video-enhancer
- Repository: https://github.com/tntwise/real-video-enhancer

## Overview

##### Main Components
- Main Application Entry Point – The file “REAL-Video-Enhancer.py” acts as the launcher, coordinating between the frontend and backend.
- Frontend/UI Layer – Found in the “src” directory (which includes UI modules like Backendhandler, DownloadModels, InputHandler, and the “src/ui” folder containing tabs, animations, and updater logic). This layer handles user interactions, video selection, settings, and preview displays.
- Backend Processing Layer – Located in the “backend” directory. This layer contains business logic for video processing such as:
– FFmpeg wrapper modules (FFmpeg.py, FFmpegBuffers.py, RenderVideo.py) for video decoding/encoding.
– Inference modules for frame interpolation and upscaling available in subdirectories like “ncnn”, “onnx”, and “pytorch”. Notice the extensive organization within “backend/src/pytorch” (which includes support for multiple interpolation models like RIFE, GIMM, GMFSS and even a “spandrel” architecture with many sub-models) illustrating a modular design to support different inference backends.
- Utility and Helper Modules – Additional functionality such as version tracking (version.py in both src and backend), constants, and utilities in “src/Util.py” and “backend/src/utils” for tasks like scene detection, border detection, and model/command generation.
- External Dependencies and Integration – The README makes clear dependencies on external tools (FFmpeg, Discord RPC, CUDA/ROCm, etc.) and build systems (using pyinstaller, cx_freeze, or nuitka). This should be represented as part of the environment or external services the application relies on.

##### Relationships and Interactions
- The frontend (GUI) layer collects user input (e.g., selected video files, desired enhancements, backend selection) and forwards commands to the backend processing components.
- The backend uses FFmpeg for handling video/video frame data. It then calls one or more inference modules based on available hardware (TensorRT, PyTorch, NCNN, ONNX) to perform interpolation/upscaling.
- Internally, the backend’s modules are organized by technology platform. For example, under “pytorch” many subdirectories indicate support for different models – they can be grouped inside an “Inference Engine” or “Model Manager” component.
- Utility modules serve as bridges – handling configuration, model downloading, version management, and even updating UI elements (e.g., preview frames, Discord RPC integration).

##### Architectural Patterns and Design Principles
- Layered Architecture: Separation between the UI (presentation layer) and backend processing (business logic layer) ensures modularity.
- Modular Design: The backend is split into multiple directories for different inference techniques (NCNN, ONNX, and PyTorch) as well as helper utilities, which promotes scalability and easier integration of new models.
- Plugin or Strategy Pattern: The support for multiple interpolation/upscaling backends (selectable at runtime) hints at a flexible strategy pattern which can be diagrammatically shown as interchangeable “backend inference modules.”

##### Relevant Technologies and Frameworks
- Python is the core programming language, and dependencies include FFmpeg (multimedia processing), PyTorch (deep learning / inference), NCNN and ONNX (alternative inference frameworks), and Discord RPC (for user engagement).
- Build tools such as PyInstaller, cx_Freeze, and nuitka are used for deployment based on OS requirements.

##### Final High-Level Structure
- One towards the “Frontend/UI Layer” (the modules in src) handling user commands and display.
- One towards the “Backend Processing Layer” (the backend folder) composed of:
▪ FFmpeg and video handling modules.
▪ Multiple inference engine modules (grouped by framework: PyTorch, NCNN, ONNX).
▪ Utility modules managing configurations, versioning, and model downloads.

## Diagram (Mermaid)

```mermaid
flowchart TD
    User("User"):::external
    RUNNER("Main Application Runner (REAL-Video-Enhancer.py)"):::backend

    subgraph "Frontend/UI Layer"
        UI("Frontend/UI Layer (src)"):::frontend
        UTILITY_FRONT("Utility Modules (Frontend)"):::utility
    end

    subgraph "Backend Processing Layer"
        FFMPEG("FFmpeg Video Processing Modules"):::backend
        DECISION("Inference Engine Selection"):::backend
        PYTORCH("Inference Engine – PyTorch"):::backend
        NCNN("Inference Engine – NCNN"):::backend
        ONNX("Inference Engine – ONNX"):::backend
        PROCESSED("Processed Video"):::backend
        UTILITY_BACK("Utility Modules (Backend)"):::utility
    end

    %% Main flow
    User -->|"launch"| RUNNER
    RUNNER -->|"initUI"| UI
    RUNNER -->|"initBackend"| FFMPEG
    RUNNER -->|"loadConfig"| UTILITY_FRONT

    UI -->|"sendVideo"| FFMPEG
    UTILITY_FRONT -.->|"support"| UI

    FFMPEG -->|"prepareFrames"| DECISION
    UTILITY_BACK -.->|"assist"| DECISION

    DECISION -->|"PyTorch"| PYTORCH
    DECISION -->|"NCNN"| NCNN
    DECISION -->|"ONNX"| ONNX

    PYTORCH -->|"processedFrames"| PROCESSED
    NCNN -->|"processedFrames"| PROCESSED
    ONNX -->|"processedFrames"| PROCESSED

    PROCESSED -->|"display"| UI

    %% Styles
    classDef frontend fill:#ADD8E6,stroke:#000,stroke-width:1px,color:#000;
    classDef backend fill:#F4A460,stroke:#000,stroke-width:1px,color:#000;
    classDef utility fill:#90EE90,stroke:#000,stroke-width:1px,color:#000;
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:1px,color:#000;

    %% Click Events
    click RUNNER "https://github.com/tntwise/real-video-enhancer/blob/2.0/REAL-Video-Enhancer.py"
    click UI "https://github.com/tntwise/real-video-enhancer/tree/2.0/src"
    click FFMPEG "https://github.com/tntwise/real-video-enhancer/blob/2.0/backend/src/FFmpeg.py"
    click PYTORCH "https://github.com/tntwise/real-video-enhancer/tree/2.0/backend/src/pytorch"
    click NCNN "https://github.com/tntwise/real-video-enhancer/tree/2.0/backend/src/ncnn"
    click ONNX "https://github.com/tntwise/real-video-enhancer/tree/2.0/backend/src/onnx"
    click UTILITY_FRONT "https://github.com/tntwise/real-video-enhancer/blob/2.0/src/Util.py"
    click UTILITY_BACK "https://github.com/tntwise/real-video-enhancer/tree/2.0/backend/src/utils"
```
