# ayamir/tiled-vr-dash-platform architecture

> Architecture diagram of the GitHub repository ayamir/tiled-vr-dash-platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ayamir/tiled-vr-dash-platform
- Repository: https://github.com/ayamir/tiled-vr-dash-platform

## Overview

##### Project Type and Purpose
- The project is a full-scale video streaming platform that incorporates both server-side and client-side components.
- It handles video processing – primarily cutting, transcoding (using H.264), and DASH packaging on the server – while on the client side, it supports two different players: one for PC (react-xrplayer) and one for VR headsets (webxr-player). The VR player even supports viewport prediction (VP), whereas the PC variant does not.
- Additionally, the platform contains modules for viewport prediction (vp), emulation of network conditions (emulate), and analysis of client-generated JSON data (analysis).

##### File Structure and Map out Components
- Server Components:
- stream: Contains scripts (e.g., crop_video.py, dash_tiles.py, multi_version.py) for video segmentation, encoding, and possibly live DASH streaming. It even includes a README and assets that suggest its role is to prepare video content for delivery.
- vp: Provides viewport prediction functionalities, indicated by directories like “trace-based-fov-predict.”
- emulate: Simulates network conditions (with subdirectories for “4g” and “5g” emulation), likely leveraging real-world tc command data.
- analysis: Processes JSON data coming from the client to generate graphs or visual insights using Python scripts (cmder.py, deploy.py, visual.py).

- Client Components:
- react-xrplayer: A single-page application (SPA) for PC-based video viewing that does not include VP prediction.
- webxr-player: A more complex client tailored for VR headsets (like Meta Quest 2), consisting of several detailed subdirectories such as:
▪ css for styling,
▪ js libraries for various functionalities including DASH integration, matrix math, 3D rendering, and scene management.
▪ Third-party libraries like gl-matrix and three.js for WebGL rendering, as well as the webxr-polyfill to ensure proper XR support.
▪ Utility scripts and experimental modules (such as hit-test.js, stereo-util.js) that support VR interactions.

##### Main Components and Their Interactions
- Server Layer:
- Video Processor (stream module): Cuts and transcodes original videos; outputs DASH-compatible segments.
- Viewport Predictor (vp module): Uses trace-based algorithms to estimate the Viewport (VP) for VR optimizations.
- Network Emulator (emulate module): Simulates different network bandwidths using real network data to test the system’s resilience.
- Data Analyzer (analysis module): Receives client-collected JSON data, analyzes it, and produces visualizations or performance metrics.
- Client Layer:
- PC Viewer (react-xrplayer): Connects to the streaming server and receives video streams for PC.
- VR Player (webxr-player):
▪ Rendering Engine: Uses matrix libraries, shader programs, and WebGL renderers (found in subdirectories like render/core, render/geometry, etc.) to display video in immersive environments.
▪ Interactivity & Utility: Includes modules for handling inputs, hit tests, and UI elements (found in js/render/nodes).
▪ Third-Party Dependencies: Integrates with libraries like three.js and gl-matrix to handle 3D operations.
- Data Flow and Interaction:
▪ The video processing server sends the DASH-packaged video to both client types.
▪ The VR player can leverage the viewport prediction results from the vp module to optimize video delivery.
▪ The analysis module gets feedback data (JSON format) from client sessions, which helps in system performance analysis.

##### Architectural Patterns and Key Technologies
- Separation of Concerns: The clear partition between server modules (stream, vp, emulate, analysis) and client modules (react-xrplayer, webxr-player) highlights a layered architecture.
- Modularity: Each server module focuses on a specific function (video processing, prediction, emulation, analysis) while the client’s structure separates basic video playback from advanced VR features.
- Technology Stack:
▪ Video Encoding: H.264.
▪ Video Streaming: DASH.
▪ Frontend Libraries: React (react-xrplayer) and WebXR with three.js, gl-matrix, and webxr-polyfill (webxr-player).
▪ Server-Side Tools: Python scripts for analysis, emulation, and streaming.
▪ Additional: Use of matrix math libraries and rendering pipelines in the VR player.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Server Layer
    subgraph "Server Layer"
        VP1["Video Processor\n(H.264, DASH packaging)"]:::server
        VPredict["Viewport Predictor\n(trace-based algorithms)"]:::server
        Nem["Network Emulator\n(4G/5G simulation)"]:::server
        DAn["Data Analyzer\n(JSON analysis & visualization)"]:::server
    end

    %% Client Layer
    subgraph "Client Layer"
        subgraph "PC Client"
            PC["PC Viewer\n(react-xrplayer)"]:::client
        end
        subgraph "VR Client"
            VR["VR Player"]:::client
            RE["Rendering Engine"]:::vrsub
            IU["Interactivity & Utility"]:::vrsub
            TP["Third-Party Dependencies\n(three.js, gl-matrix, webxr-polyfill)"]:::vrsub
        end
    end

    %% Connections between Server and Clients
    VP1 -->|"streamsVideo"| PC
    VP1 -->|"streamsVideo"| VR
    VPredict -->|"providesVP"| VR
    PC -->|"sendsFeedback"| DAn
    VR -->|"sendsFeedback"| DAn
    Nem -->|"appliesNetwork"| VP1

    %% Internal connections within VR Player details
    VR -->|"contains"| RE
    VR -->|"contains"| IU
    VR -->|"contains"| TP

    %% Click Events
    click VP1 "https://github.com/ayamir/tiled-vr-dash-platform/tree/main/server/stream"
    click VPredict "https://github.com/ayamir/tiled-vr-dash-platform/tree/main/server/vp/trace-based-fov-predict"
    click Nem "https://github.com/ayamir/tiled-vr-dash-platform/tree/main/server/emulate"
    click DAn "https://github.com/ayamir/tiled-vr-dash-platform/tree/main/server/analysis"
    click PC "https://github.com/ayamir/tiled-vr-dash-platform/tree/main/client/react-xrplayer"
    click VR "https://github.com/ayamir/tiled-vr-dash-platform/tree/main/client/webxr-player"

    %% Styles
    classDef server fill:#AED6F1,stroke:#1B4F72,stroke-width:2px;
    classDef client fill:#A9DFBF,stroke:#196F3D,stroke-width:2px;
    classDef vrsub fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
```
