# picovoice/picollm architecture

> Architecture diagram of the GitHub repository picovoice/picollm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/picovoice/picollm
- Repository: https://github.com/picovoice/picollm

## Overview

##### Project type and purpose
- This is an open-source, cross-platform SDK and inference engine (“picoLLM”) for running quantized large language models locally.
- It ships a native core library (C/C++ compiled for CPU/GPU on Linux, Windows, macOS, Raspberry Pi, Android, iOS, WebAssembly) plus language- and platform-specific bindings (Python, .NET, Node.js, Android/Java, iOS/Swift, Web/TypeScript, C).
- The README emphasizes platform coverage, model support, AccessKey authentication against Picovoice license servers, and a suite of demo applications.

##### File structure
- Top-level artifacts:
- / + lib/ → native headers and pre-built binaries per OS/architecture/CPU & WebAssembly
- binding/ → subdirectories for each SDK binding (android, dotnet, ios, nodejs, python, web, c) containing wrapper code, build files, tests
- demo/ → end-user sample apps in each language/platform demonstrating chat & completion
- resources/ → lint and test configs
- .github/workflows → CI pipelines for building, linting, testing across all platforms and languages
- Package.swift, README.md, LICENSE, .gitignore etc.

- Architectural patterns:
- Core “engine” library + thin wrapper layers
- Plugin-style multi-language binding modules
- Demo clients that exercise the same binding APIs
- CI/CD as orchestration layer for building and validating every binding

##### README for additional insights
- Describes AccessKey flow: bindings must call license server to validate token even though inference is offline.
- Demonstrates usage patterns: builder-style Java API, Python context management, Swift error handling, JS worker for WebAssembly.
- Lists supported models and platforms, confirming the cross-compilation matrix.

##### Elements
- Binding → Core Engine calls
- Core Engine → License Server (AccessKey validation)
- Application → Binding API → Engine → Application
- CI/CD → build/publish → artifacts → bindings
- Native C/C++ Engine (e.g., dark grey)
- Binding modules (e.g., blue)
- Demo apps (e.g., green)
- External services (e.g., purple)
- CI/CD workflows (e.g., orange)
- Annotations for multi-platform binaries under lib/ and include/.

##### Detail guidance
- Platform matrix under the native engine (Linux, macOS, Windows, Raspberry Pi, Android, iOS, WASM).
- Don’t omit the AccessKey flow to license server.
- Web SDK running in a browser host + Web Worker.
- Common sequence: Application → Binding → Engine → Output.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% CI/CD Pipelines
    subgraph "CI/CD Pipelines"
        CI["GitHub Actions Workflows"]:::ci
    end

    %% Demo Applications
    subgraph "Demo Applications"
        DM_PY["Python Demos"]:::demo
        DM_DOT[".NET Demos"]:::demo
        DM_NODE["Node.js Demos"]:::demo
        DM_ANDROID["Android Demos"]:::demo
        DM_IOS["iOS Demos"]:::demo
        DM_WEB["Web Demos"]:::demo
        DM_C["C Demo"]:::demo
    end

    %% Binding Modules
    subgraph "Binding Modules"
        PY_SDK["Python SDK\nbinding/python"]:::binding
        DOTNET_SDK[".NET SDK\nbinding/dotnet"]:::binding
        NODE_SDK["Node.js SDK\nbinding/nodejs"]:::binding
        AND_SDK["Android SDK\nbinding/android"]:::binding
        IOS_SDK["iOS SDK\nbinding/ios"]:::binding
        WEB_SDK["Web SDK\nbinding/web"]:::binding
        C_SDK["C SDK\ninclude/pv_picollm.h"]:::binding
    end

    %% Core Engine & External
    subgraph "Core Engine & Services"
        CORE["Native Core Engine\ninclude/ + lib/"]:::core
        LIC["Picovoice License Server"]:::external
    end

    %% Relationships
    CI -->|build & test| CORE
    CI -->|orchestrates| PY_SDK
    CI -->|orchestrates| DOTNET_SDK
    CI -->|orchestrates| NODE_SDK
    CI -->|orchestrates| AND_SDK
    CI -->|orchestrates| IOS_SDK
    CI -->|orchestrates| WEB_SDK
    CI -->|orchestrates| C_SDK

    DM_PY -->|uses| PY_SDK
    DM_DOT -->|uses| DOTNET_SDK
    DM_NODE -->|uses| NODE_SDK
    DM_ANDROID -->|uses| AND_SDK
    DM_IOS -->|uses| IOS_SDK
    DM_WEB -->|uses| WEB_SDK
    DM_C -->|uses| C_SDK

    PY_SDK -->|calls| CORE
    DOTNET_SDK -->|calls| CORE
    NODE_SDK -->|calls| CORE
    AND_SDK -->|calls| CORE
    IOS_SDK -->|calls| CORE
    WEB_SDK -->|calls| CORE
    C_SDK -->|calls| CORE

    PY_SDK -->|validates AccessKey| LIC
    DOTNET_SDK -->|validates AccessKey| LIC
    NODE_SDK -->|validates AccessKey| LIC
    AND_SDK -->|validates AccessKey| LIC
    IOS_SDK -->|validates AccessKey| LIC
    WEB_SDK -->|validates AccessKey| LIC
    C_SDK -->|validates AccessKey| LIC

    CORE -->|platform binaries| CORE
    CORE -->|AccessKey check| LIC

    %% Click Events
    click CI "https://github.com/picovoice/picollm/tree/main/.github/workflows/"
    click PY_SDK "https://github.com/picovoice/picollm/blob/main/binding/python/__init__.py"
    click DOTNET_SDK "https://github.com/picovoice/picollm/blob/main/binding/dotnet/PicoLLM.sln"
    click NODE_SDK "https://github.com/picovoice/picollm/blob/main/binding/nodejs/package.json"
    click AND_SDK "https://github.com/picovoice/picollm/tree/main/binding/android/PicoLLM/"
    click IOS_SDK "https://github.com/picovoice/picollm/blob/main/binding/ios/PicoLLM.swift"
    click WEB_SDK "https://github.com/picovoice/picollm/blob/main/binding/web/package.json"
    click C_SDK "https://github.com/picovoice/picollm/blob/main/include/pv_picollm.h"
    click DM_PY "https://github.com/picovoice/picollm/tree/main/demo/python/"
    click DM_DOT "https://github.com/picovoice/picollm/blob/main/demo/dotnet/PicoLLMDemo/ChatDemo.cs"
    click DM_NODE "https://github.com/picovoice/picollm/blob/main/demo/nodejs/chat.js"
    click DM_ANDROID "https://github.com/picovoice/picollm/tree/main/demo/android/Chat"
    click DM_IOS "https://github.com/picovoice/picollm/tree/main/demo/ios/Chat/PicoLLMChatDemo"
    click DM_WEB "https://github.com/picovoice/picollm/tree/main/demo/web/chat/"
    click DM_C "https://github.com/picovoice/picollm/blob/main/demo/c/picollm_demo_completion.c"

    %% Styles
    classDef core fill:#555,stroke:#333,stroke-width:2px,color:#fff
    classDef binding fill:#1E90FF,stroke:#104E8B,stroke-width:1px,color:#fff
    classDef demo fill:#3CB371,stroke:#2E8B57,stroke-width:1px,color:#fff
    classDef external fill:#9370DB,stroke:#5D478B,stroke-width:1px,color:#fff
    classDef ci fill:#FFA500,stroke:#CC8400,stroke-width:1px,color:#000
```
