# xkeyc/fl_caption architecture

> Architecture diagram of the GitHub repository xkeyc/fl_caption, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-16). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/xkeyc/fl_caption
- Repository: https://github.com/xkeyc/fl_caption

## Overview

##### Project Overview and Main Components
- Frontend (Flutter):
- This is the user interface layer found under the lib directory. It consists of pages (e.g., settings, dialogs) and widgets that provide controls for model selection, download progress, language settings, and display of captions.
- It also handles user interactions such as setting up LLM API information and triggering actions (like starting the captioning or initiating a model download).
- Backend (Rust):
- The heavy lifting for audio processing, Whisper inference, voice activity detection (VAD), and text utilities is implemented in Rust. The Rust source code is structured in the rust folder and includes modules for whisper, vad, and text utilities.
- The backend handles the inference logic, communicates with audio models stored as assets, and processes results to be sent back to the Flutter frontend.
- Flutter-Rust Bridge:
- The integration between the Flutter UI and the Rust backend is managed via the flutter_rust_bridge. This enables efficient communication (e.g., function calls and data passing) between the two different language ecosystems.
- Assets:
- Assets (models, fonts, Whisper config files) reside in the assets directory (and similarly within the rust/assets subfolder). These provide the necessary models and resources used by the Rust backend when performing inference.
- Platform-specific Layers:
- The project contains platform-specific folders (linux, macos, windows) with appropriate build configurations and native code integration. This ensures that the compiled binary wraps and deploys the combined Flutter and Rust solution as native desktop applications.

##### Relationships and Interactions
- Data Flow:
- The Flutter UI triggers user actions (e.g., selecting a language, initiating model download, starting caption inference).
- These actions call functions via the Flutter-Rust Bridge to pass control and necessary parameters to the Rust backend.
- The Rust backend then performs the audio inference and text processing, and responds with processed caption data.
- Model download and configuration are managed from the Flutter side, with links to assets (for example, using URLs or file paths as specified in the code and README).
- Dependency & Build Flow:
- Build scripts (see the scripts folder and various CMakeLists.txt files) indicate that the project uses a combination of Flutter’s build tools and Rust’s Cargo system.
- The rust_builder folder contains tools to build and integrate the Rust library into different platforms (Android, iOS, macOS, Windows).

##### Architectural Patterns and Design Principles
- Separation of Concerns:
- The architecture cleanly separates UI logic (Flutter) from audio processing/inference (Rust), ensuring that heavy computations do not block the UI.
- Cross-Platform Compatibility:
- By maintaining platform-specific folders and leveraging Flutter’s cross-platform capabilities paired with native integrations via rust_bridge, the project supports multiple operating systems.
- Modular Design:
- The directory structure (with clear subdivisions like common, dialogs, pages, widgets, and various Rust modules) indicates a modular setup, making it easier to update, test, or extend specific components (e.g., updating the Whisper inference code without changing the UI).

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend Layer
    subgraph "Flutter Frontend"
        A["Flutter Frontend (UI)\n(pages, dialogs, widgets)"]:::flutter
    end

    %% Flutter-Rust Bridge Layer
    subgraph "Flutter-Rust Bridge"
        direction TB
        B1["Bridge Config\n(flutter_rust_bridge.yaml)"]:::bridge
        B2["FRB Dart Files\n(lib/common/rust: frb_generated.dart, frb_generated.io.dart)"]:::bridge
        B3["FRB Rust Generated\n(rust/src/frb_generated.rs)"]:::bridge
        B1 --> B2 --> B3
    end

    %% Rust Backend Layer
    subgraph "Rust Backend"
        direction TB
        R_main["Rust Backend"]:::rustMain
        R1["Whisper Inference"]:::rust
        R2["Voice Activity Detection (VAD)"]:::rust
        R3["Text Utilities"]:::rust
        R4["Assets (rust/assets)"]:::assetsRust
        R_main --> R1 --> R2 --> R3
        R3 --- R4
    end

    %% Assets Repository
    D["Assets Repository"]:::assets

    %% Platform-specific Layers
    subgraph "Platform-specific Layers"
        direction TB
        LINUX["Linux"]:::platform
        MACOS["macOS"]:::platform
        WINDOWS["Windows"]:::platform
        LINUX --> MACOS --> WINDOWS
    end

    %% Build and Integration Tools
    subgraph "Build & Integration Tools"
        direction TB
        SCRIPTS["Scripts"]:::build
        RUST_BUILDER["Rust Builder"]:::build
        SCRIPTS --> RUST_BUILDER
    end

    %% Connections / Data Flow
    A -->|"UICommands/InferenceRequests"| B1
    B3 -->|"BridgeData"| R_main
    R_main -->|"CaptionDataResponse"| A

    A ---|"AccessAssets"| D
    R_main ---|"AccessAssets"| D

    SCRIPTS ---|"BuildsFor"| LINUX

    %% Click Events
    click A "https://github.com/xkeyc/fl_caption/tree/main/lib"
    click B1 "https://github.com/xkeyc/fl_caption/blob/main/flutter_rust_bridge.yaml"
    click B2 "https://github.com/xkeyc/fl_caption/tree/main/lib/common/rust"
    click B3 "https://github.com/xkeyc/fl_caption/blob/main/rust/src/frb_generated.rs"
    click R_main "https://github.com/xkeyc/fl_caption/tree/main/rust"
    click D "https://github.com/xkeyc/fl_caption/tree/main/assets"
    click LINUX "https://github.com/xkeyc/fl_caption/tree/main/linux"
    click MACOS "https://github.com/xkeyc/fl_caption/tree/main/macos"
    click WINDOWS "https://github.com/xkeyc/fl_caption/tree/main/windows"
    click SCRIPTS "https://github.com/xkeyc/fl_caption/tree/main/scripts"
    click RUST_BUILDER "https://github.com/xkeyc/fl_caption/tree/main/rust_builder"

    %% Styles
    classDef flutter fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef rust fill:#ffe5b4,stroke:#d35400,stroke-width:2px;
    classDef rustMain fill:#ffcc99,stroke:#d35400,stroke-width:2px;
    classDef bridge fill:#d1e7dd,stroke:#0f5132,stroke-width:2px;
    classDef assets fill:#e2e3e5,stroke:#41464b,stroke-width:2px;
    classDef assetsRust fill:#F0E68C,stroke:#B8860B,stroke-width:2px;
    classDef platform fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef build fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;
```
