# 2bab/mediapiper architecture

> Architecture diagram of the GitHub repository 2bab/mediapiper, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/2bab/mediapiper
- Repository: https://github.com/2bab/mediapiper

## Overview

##### Project Type and Purpose
- This is a Kotlin Multiplatform mobile sample app (Android + iOS) that demonstrates using Google’s MediaPipe tasks (LLM inference and object detection) in a shared codebase.
- The `app` module is a KMM library containing common business logic, UI definitions (Compose), and platform-specific implementations via expect/actual. The `iosApp` is a pure iOS Xcode project that consumes the generated KMM framework via CocoaPods.

##### File-Structure Highlights & Architectural Implications
- Top-level modules:
- `app/`: KMM library module (build.gradle.kts, src/commonMain, src/androidMain, src/iosMain)
- `iosApp/`: Xcode workspace & Pod integration, native Swift/SwiftUI entrypoints, MediaPipe pods
- `commonMain`: Contains core application entry (`App.kt`, `Startup.kt`), UI abstraction (Compose screens under `llm/`), view‐models (`ChatViewModel`), and the LLM operator interface (`LLMOperator.kt`).
- `androidMain` & `iosMain`: Provide platform-specific actualizations for `Platform`, `LLMOperator`, `Theme`, and `App` entrypoints.
- Pods & Native Frameworks: Under `iosApp/Pods/MediaPipeTasksGenAI.xcframework` and `MediaPipeTasksGenAIC.xcframework`, plus local `.podspec` for the KMM framework.

a. Shared KMM Core
- Business Logic: LLMOperator (expect/actual), ChatViewModel, AiRoute
- UI Layer: Compose UI screens (ChatScreen, LoadingScreen)
- Theme & Resource Layer
b. Android App
- Android entry (`App.android.kt`), `Platform.android.kt`
- Compose UI host, Gradle/Kotlin plugin
c. iOS App
- SwiftUI/Swift host (`iosApp.swift`, `MediapipeLLMInferenceDelegate.swift`)
- CocoaPods integration point, Pod‐installed MediaPipe frameworks
d. Native MediaPipe Frameworks
- MediaPipeTasksGenAI: ML & LLM inference engine
- MediaPipeTasksGenAIC: CPU/GPU optimized LLM cores, C headers
e. Device & Model Artifacts
- On-device LLM model files (pushed via adb on Android / folder ref on iOS)
- OS layers (Android SDK, iOS frameworks)

##### Component Relationships & Data Flow
1. User interacts with the App UI (Compose or SwiftUI)
2. UI invokes ChatViewModel → orchestrates LLMOperator
3. LLMOperator bridges to native MediaPipeTasksGenAI/C frameworks (via JNI on Android, via generated KMM framework + pods on iOS)
4. Native frameworks load local model files and perform inference
5. Inference results are returned back up through LLMOperator to the ViewModel → UI updates

##### Architectural Patterns & Design Principles
- Layered modularization: UI layer, ViewModel (MVVM), operator interface, native library
- Expect/Actual: abstraction of platform specifics behind a common API
- Single source of truth: all business logic and UI definitions reside in `commonMain`
- Dependency via CocoaPods (iOS) and Gradle (Android) for native libs

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Top-level layers
    subgraph "Android App"
        direction TB
        AppAndroid["App.android.kt"]:::android
        PlatformAndroid["Platform.android.kt"]:::android
        LLMOpAndroid["LLMOperator.android.kt"]:::android
        ThemeAndroid["Theme.android.kt"]:::android
        AndroidRes["AndroidManifest.xml + res/"]:::android
    end

    subgraph "Shared KMM Library"
        direction TB
        subgraph "Core Entry" 
            direction TB
            App["App.kt"]:::shared
            Startup["Startup.kt"]:::shared
        end
        subgraph "UI Layer"
            direction TB
            ChatScreen["ChatScreen.kt"]:::shared
            LoadingScreen["LoadingScreen.kt"]:::shared
        end
        subgraph "ViewModel Layer"
            direction TB
            ChatVM["ChatViewModel.kt"]:::shared
            AiRoute["AiRoute.kt"]:::shared
        end
        subgraph "Operator Interface"
            direction TB
            LLMOp["LLMOperator.kt"]:::shared
        end
        subgraph "Theme & Resources"
            direction TB
            Theme["Theme.kt"]:::shared
            ColorDefs["Color.kt"]:::shared
            ComposeRes["composeResources"]:::shared
        end
    end

    subgraph "iOS App"
        direction TB
        IOSSwift["iosApp.swift"]:::ios
        IOSDelegate["MediapipeLLMInferenceDelegate.swift"]:::ios
        Podfile["Podfile"]:::ios
    end

    subgraph "Native MediaPipe Frameworks & Assets"
        direction TB
        MPGenAI["MediaPipeTasksGenAI.xcframework"]:::native
        MPGenAIC["MediaPipeTasksGenAIC.xcframework"]:::native
        ModelFiles["LLM Model Files"]:::asset
    end

    %% Data Flows
    ChatScreen -->|"userInput"| ChatVM
    LoadingScreen -->|"userInput"| ChatVM
    ChatVM -->|"invokeInference()"| LLMOp
    LLMOp -->|"JNI / KMM bridge"| MPGenAI
    LLMOp -->|"JNI / KMM bridge"| MPGenAIC
    MPGenAI -->|"load model"| ModelFiles
    MPGenAIC -->|"load model"| ModelFiles
    MPGenAI -->|"return result"| LLMOp
    MPGenAIC -->|"return result"| LLMOp
    LLMOp -->|"returnInference"| ChatVM
    ChatVM -->|"updateUI"| ChatScreen
    ChatVM -->|"updateUI"| LoadingScreen

    %% Expect/Actual bindings
    LLMOp -.-> LLMOpAndroid
    LLMOp -.-> LLMOp["LLMOperator.ios.kt"]:::ios
    Theme -.-> ThemeAndroid
    Theme -.-> Theme["Theme.ios.kt"]:::ios
    App -.-> AppAndroid
    App -.-> IOSSwift
    PlatformAndroid -.-> PlatformAndroid
    PlatformAndroid -.-> PlatformAndroid
    Startup -.-> IOSSwift

    %% Click Events
    click App "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/App.kt"
    click Startup "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/Startup.kt"
    click LLMOp "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/llm/LLMOperator.kt"
    click ChatVM "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/llm/ChatViewModel.kt"
    click AiRoute "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/llm/AiRoute.kt"
    click ChatScreen "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/llm/ChatScreen.kt"
    click LoadingScreen "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/llm/LoadingScreen.kt"
    click Theme "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/theme/Theme.kt"
    click ColorDefs "https://github.com/2bab/mediapiper/blob/main/app/src/commonMain/kotlin/me/xx2bab/mediapiper/theme/Color.kt"
    click ComposeRes "https://github.com/2bab/mediapiper/tree/main/app/src/commonMain/composeResources"
    click AppAndroid "https://github.com/2bab/mediapiper/blob/main/app/src/androidMain/kotlin/me/xx2bab/mediapiper/App.android.kt"
    click PlatformAndroid "https://github.com/2bab/mediapiper/blob/main/app/src/androidMain/kotlin/me/xx2bab/mediapiper/Platform.android.kt"
    click LLMOpAndroid "https://github.com/2bab/mediapiper/blob/main/app/src/androidMain/kotlin/me/xx2bab/mediapiper/llm/LLMOperator.android.kt"
    click ThemeAndroid "https://github.com/2bab/mediapiper/blob/main/app/src/androidMain/kotlin/me/xx2bab/mediapiper/theme/Theme.android.kt"
    click AndroidRes "https://github.com/2bab/mediapiper/blob/main/app/src/androidMain/AndroidManifest.xml"
    click IOSSwift "https://github.com/2bab/mediapiper/blob/main/iosApp/iosApp/iosApp.swift"
    click IOSDelegate "https://github.com/2bab/mediapiper/blob/main/iosApp/iosApp/MediapipeLLMInferenceDelegate.swift"
    click Podfile "https://github.com/2bab/mediapiper/tree/main/iosApp/Podfile"
    click MPGenAI "https://github.com/2bab/mediapiper/blob/main/iosApp/Pods/MediaPipeTasksGenAI/frameworks/MediaPipeTasksGenAI.xcframework"
    click MPGenAIC "https://github.com/2bab/mediapiper/blob/main/iosApp/Pods/MediaPipeTasksGenAIC/frameworks/MediaPipeTasksGenAIC.xcframework"
    click ComposeRes "https://github.com/2bab/mediapiper/tree/main/app/src/commonMain/composeResources"
    click LLMOp "https://github.com/2bab/mediapiper/blob/main/app/src/iosMain/kotlin/me/xx2bab/mediapiper/llm/LLMOperator.ios.kt"
    click Theme "https://github.com/2bab/mediapiper/blob/main/app/src/iosMain/kotlin/me/xx2bab/mediapiper/theme/Theme.ios.kt"
    click PlatformIOS "https://github.com/2bab/mediapiper/blob/main/app/src/iosMain/kotlin/me/xx2bab/mediapiper/Platform.ios.kt"
    click MainIOS "https://github.com/2bab/mediapiper/blob/main/app/src/iosMain/kotlin/main.kt"
    click Podspec "https://github.com/2bab/mediapiper/blob/main/app/Mediapiper.podspec"

    %% Styles
    classDef shared fill:#e0f7fa,stroke:#006064;
    classDef android fill:#e8f5e9,stroke:#1b5e20;
    classDef ios fill:#fff3e0,stroke:#e65100;
    classDef native fill:#f3e5f5,stroke:#4a148c;
    classDef asset fill:#fffde7,stroke:#f9a825;
```
