# homuler/mediapipeunityplugin architecture

> Architecture diagram of the GitHub repository homuler/mediapipeunityplugin, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/homuler/mediapipeunityplugin
- Repository: https://github.com/homuler/mediapipeunityplugin

## Overview

##### Project Type and Purpose
- This project is a Unity native plugin that “ports the MediaPipe API (C++) one by one to C#.” It enables a Unity (2022.3 or later) project to call MediaPipe functions natively and even run official and custom solutions.
- The design makes it possible to run MediaPipe’s graph-based processing (calculators, graphs, packets) from Unity C# code. In brief, its purpose is to expose MediaPipe’s advanced multimedia processing (e.g., face detection, object detection, landmark detection) to Unity applications.

##### Key Components and Layers
a. Unity Application Layer:
– “Assets” in the file tree include examples, sample scenes, editor scripts, and UI components.
– Scripts in the Assets (e.g., HelloWorld.cs) illustrate how MonoBehaviour components instantiate and use classes like CalculatorGraph and PacketPoller.
– The plugin integrates into Unity’s build process and execution environment (both UnityEditor and runtime on different platforms).

b. C# API / Wrapper Layer:
– Located mainly in the “Packages” directory under “com.github.homuler.mediapipe”, this portion defines the C# wrappers and assemblies (and .asmdef files) for exposing MediaPipe functionalities.
– Here you have classes for CalculatorGraph, Packet, and various other APIs that mirror the native (C++) MediaPipe interfaces.
– This layer is critical for bridging Unity’s C# environment with the underlying native libraries.

c. Native / C++ / MediaPipe API Layer:
– The “mediapipe_api” directory contains the C++ code (with BUILD files, headers, and sources) that implement the port of MediaPipe’s calculators, graphs, formats, GPU helpers, etc.
– It leverages several third-party libraries and dependencies (e.g., absl, gl, protobuf) that are visible under “mediapipe_api/external” and “third_party”.
– This native layer is compiled (using Bazel and possibly Docker for cross-platform builds) into shared libraries that are then loaded at runtime using Unity’s native plugin mechanism.

d. Build and Integration Tools:
– Files such as WORKSPACE, .bazel* configuration files, build.py, and Docker folders indicate that there is a sophisticated build setup based on Bazel.
– These tools and configuration scripts generate the native binaries required by Unity.
– Additionally there are helper scripts (e.g., issue-template-version-updater.js) and documentation files in “docs” that support build and usage.

##### Relationships and Data Flow
- The primary flow begins in the Unity application (via a MonoBehaviour script such as HelloWorld.cs) that instantiates a CalculatorGraph.
- The C# API layer calls into the native layer (using DllImport and Unity native plugin interfaces) to create and manage MediaPipe graphs.
- MediaPipe’s native calculators process data (packets) through a graph; results are polled back into C# via the PacketPoller instance.
- The plugin also takes care of loading required assets (e.g., models or configuration files) via the StreamingAssets folder or local assets.
- The build system (Bazel, Docker, npm scripts) and directories for “mediapipe_api” and “third_party” show a clear separation: the native libraries are built independently and then integrated into the Unity project as a plugin.

##### Architectural Patterns and Design Principles
- Layered Architecture: A clear separation exists between the Unity (C#) presentation layer, the C# API/wrapper layer, and the native C++ implementation layer.
- Cross-platform Modular Design: The project is organized to support a range of platforms by abstracting platform-specific details (e.g., differences for Android, iOS, Windows, macOS, Linux).
- Build Integration: Utilizing Bazel, Docker and npm for packaging (tarballs/unitypackages) illustrates a systematic build and deployment pipeline.

##### Relevant Technologies and Frameworks
- Unity (C#) for the front-end/game engine layer.
- C# Native Plugins (via DllImport) for bridging to native C++ libraries.
- C++ with Bazel as a build tool for MediaPipe components.
- Third-party libraries such as absl, protobuf, OpenGL/Vulkan (for GPU, where applicable), and platform-specific libraries.
- Docker scripts and gradle templates for Android integration.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Unity Application Layer
    subgraph "Unity Application Layer"
        U1["Unity Client (MonoBehaviour)"]:::unity
        U2["Unity Samples & Scenes"]:::unity
    end

    %% C# API / Wrapper Layer
    subgraph "C# API / Wrapper Layer"
        C1["MediaPipe C# Wrapper"]:::csharp
    end

    %% Native C++ MediaPipe API Layer
    subgraph "Native C++ MediaPipe API Layer"
        N1["MediaPipe Native API"]:::native
    end

    %% Build & Deployment Tools
    subgraph "Build & Deployment Tools"
        B1[".bazelrc"]:::build
        B2[".bazelversion"]:::build
        B3["WORKSPACE"]:::build
        B4["build.py"]:::build
        B5["docker folder"]:::build
    end

    %% Relationships
    U1 -->|"calls"| C1
    U2 -->|"provides_assets"| C1

    C1 -->|"calls_native"| N1
    N1 -->|"returns_packets"| C1
    C1 -->|"updates_UI"| U1

    %% Build Tools relationship to Native API
    B3 -->|"builds_native_binaries"| N1

    %% Click Events
    click U1 "https://github.com/homuler/mediapipeunityplugin/tree/master/Assets/MediaPipeUnity"
    click U2 "https://github.com/homuler/mediapipeunityplugin/tree/master/Assets/MediaPipeUnity/Samples"
    click C1 "https://github.com/homuler/mediapipeunityplugin/blob/master/Packages/com.github.homuler.mediapipe"
    click N1 "https://github.com/homuler/mediapipeunityplugin/tree/master/mediapipe_api"
    click B1 "https://github.com/homuler/mediapipeunityplugin/blob/master/.bazelrc"
    click B2 "https://github.com/homuler/mediapipeunityplugin/blob/master/.bazelversion"
    click B3 "https://github.com/homuler/mediapipeunityplugin/tree/master/WORKSPACE"
    click B4 "https://github.com/homuler/mediapipeunityplugin/blob/master/build.py"
    click B5 "https://github.com/homuler/mediapipeunityplugin/tree/master/docker"

    %% Styles
    classDef unity fill:#f9c74f,stroke:#f9844a,stroke-width:2px;
    classDef csharp fill:#90be6d,stroke:#43aa8b,stroke-width:2px;
    classDef native fill:#f94144,stroke:#f3722c,stroke-width:2px;
    classDef build fill:#577590,stroke:#277da1,stroke-width:2px;
```
