# qualcomm/geniex architecture

> Architecture diagram of the GitHub repository qualcomm/geniex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-07-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/qualcomm/geniex
- Repository: https://github.com/qualcomm/geniex

## Overview

#### Purpose and Entry Points
GenieX is an on-device LLM/VLM inference stack for Snapdragon Windows ARM64, Linux ARM64, and Android. A C/C++ SDK is the core; consumers enter through the Go CLI/server, Python package, Android Kotlin/JNI SDK, or Go bindings. Core public API: `sdk/include/geniex.h`; CLI executable: `cli/cmd/geniex/main.go`; Python API: `bindings/python/geniex/modeling.py`.

#### Core SDK and Runtime Selection
The SDK owns device discovery, runtime enumeration, model lifecycle, inference, VLM support, logging, and plugin registration. It presents common LLM/VLM interfaces while selecting an implementation based on model/runtime and available compute units. Primary paths: `sdk/src/llm.cpp`, `sdk/src/vlm.cpp`, `sdk/src/registry.cpp`, `sdk/include/plugin/Plugin.h`.

#### Inference Plugin Boundary
Inference engines are dynamically separated behind SDK plugin interfaces. The `llama_cpp` plugin executes GGUF models across CPU, Adreno GPU, and Hexagon/HTP paths; the `qairt` plugin executes Qualcomm AI Engine Direct bundles on the NPU. Plugin-specific model loading and generation stay below this boundary, allowing all upper interfaces to use the same SDK API. Paths: `sdk/plugins/llama_cpp/src/plugin.cpp`, `sdk/plugins/llama_cpp/src/llm.cpp`, `sdk/plugins/qairt/src/plugin.cpp`, `sdk/plugins/qairt/src/llm.cpp`.

#### Model Acquisition and Local Store
A Rust model-manager subsystem resolves model references, queries metadata, downloads and validates artifacts, resumes transfers, and persists locally installed models. It supports Qualcomm AI Hub, Hugging Face, Docker Hub, and local filesystem sources. Its FFI layer exposes these operations to the SDK and language clients. Paths: `sdk/model-manager/crates/core/src/pull.rs`, `sdk/model-manager/crates/core/src/source/`, `sdk/model-manager/crates/core/src/store.rs`, `sdk/model-manager/crates/ffi/src/lib.rs`.

#### CLI and Local OpenAI-Compatible Service
The Go CLI coordinates model pull/run/infer workflows with the SDK and provides interactive terminal behavior. Its `serve` command hosts a local OpenAI-compatible HTTP API, routing chat and model requests through services to the same inference layer; middleware provides authentication, CORS, and concurrency/GIL-related controls. Paths: `cli/cmd/geniex/main.go`, `cli/cmd/geniex/infer.go`, `cli/server/server.go`, `cli/server/handler/chat.go`.

#### Language and Android Integration
Python provides Transformers-like model construction, tokenization, streaming generation, and C FFI access. Android provides Kotlin wrappers over JNI bridges for LLM, VLM, and model management. Go bindings offer another native-language SDK surface. These are adapters rather than independent inference engines. Paths: `bindings/python/geniex/_ffi/_api.py`, `bindings/python/geniex/modeling.py`, `bindings/android/app/src/main/cpp/llm_bridge_jni.cpp`, `bindings/android/app/src/main/java/com/geniex/sdk/GenieXSdk.kt`.

#### Runtimes, Platforms, and Delivery
Native SDK/plugin code is C/C++ built with CMake and Bazel; model management is Rust; CLI and Go bindings are Go; Python is packaged separately; Android uses Gradle, Kotlin, and JNI. The stack depends on Qualcomm hardware/runtime infrastructure including Hexagon NPU/HTP, Adreno GPU, and Qualcomm AI Engine Direct, plus remote artifact services from AI Hub, Hugging Face, and Docker Hub. Build roots: `sdk/CMakeLists.txt`, `BUILD.bazel`, `bindings/android/app/build.gradle.kts`, `sdk/model-manager/Cargo.toml`.

Main entry points: [`sdk/include/geniex.h`](https://github.com/qualcomm/geniex/blob/main/sdk/include/geniex.h) (SDK public API), [`sdk/include/plugin/Plugin.h`](https://github.com/qualcomm/geniex/blob/main/sdk/include/plugin/Plugin.h) (Plugin interface), [`sdk/model-manager/crates/core/src/pull.rs`](https://github.com/qualcomm/geniex/blob/main/sdk/model-manager/crates/core/src/pull.rs) (Artifact resolver and pull), [`sdk/src/registry.cpp`](https://github.com/qualcomm/geniex/blob/main/sdk/src/registry.cpp) (Runtime registry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clients["Client Surfaces"]
  node_cli["Go CLI<br/>Go command<br/>[main.go]"]
  node_infer_command["CLI inference flow<br/>Go command<br/>[infer.go]"]
  node_local_service["OpenAI-compatible service<br/>Go HTTP server<br/>[server.go]"]
  node_chat_handler["Chat handler<br/>HTTP handler<br/>[chat.go]"]
  node_python_api["Python modeling API<br/>Python API<br/>[modeling.py]"]
  node_python_ffi["Python C FFI<br/>Python FFI<br/>[_api.py]"]
  node_android_sdk["Android Kotlin SDK<br/>Kotlin API<br/>[GenieXSdk.kt]"]
  node_android_jni["Android JNI bridge<br/>C++ JNI<br/>[llm_bridge_jni.cpp]"]
end

subgraph group_sdk["Native SDK"]
  node_sdk_api["SDK public API<br/>C/C++ API<br/>[geniex.h]"]
  node_sdk_llm["LLM lifecycle<br/>C++ subsystem<br/>[llm.cpp]"]
  node_sdk_vlm["VLM lifecycle<br/>C++ subsystem<br/>[vlm.cpp]"]
  node_registry["Runtime registry<br/>C++ registry<br/>[registry.cpp]"]
  node_plugin_api{{"Plugin interface<br/>C++ plugin API<br/>[Plugin.h]"}}
end

subgraph group_models["Model Management"]
  node_model_ffi["Model manager FFI<br/>Rust FFI<br/>[lib.rs]"]
  node_model_pull["Artifact resolver and pull<br/>Rust subsystem<br/>[pull.rs]"]
  node_model_sources["Artifact sources<br/>Rust source adapters"]
  node_model_store[("Local model store<br/>Rust store<br/>[store.rs]")]
end

subgraph group_engines["Inference Engines"]
  node_llama_plugin{{"llama.cpp plugin<br/>C++ inference plugin<br/>[plugin.cpp]"}}
  node_qairt_plugin{{"QAIRT plugin<br/>C++ inference plugin<br/>[plugin.cpp]"}}
end

node_cli -->|"implements"| node_infer_command
node_cli -->|"serve command"| node_local_service
node_infer_command -->|"invokes"| node_sdk_api
node_local_service -->|"routes chat"| node_chat_handler
node_chat_handler -->|"runs inference"| node_sdk_api
node_python_api -->|"uses"| node_python_ffi
node_python_ffi -->|"calls"| node_sdk_api
node_android_sdk -->|"uses"| node_android_jni
node_android_jni -->|"calls"| node_sdk_api
node_sdk_api -->|"LLM requests"| node_sdk_llm
node_sdk_api -->|"VLM requests"| node_sdk_vlm
node_sdk_llm -->|"selects runtime"| node_registry
node_sdk_vlm -->|"selects runtime"| node_registry
node_registry -->|"loads implementation"| node_plugin_api
node_plugin_api -->|"implements"| node_llama_plugin
node_plugin_api -->|"implements"| node_qairt_plugin
node_sdk_api -->|"model operations"| node_model_ffi
node_model_ffi -->|"pulls models"| node_model_pull
node_model_pull -->|"fetches from"| node_model_sources
node_model_pull -->|"installs into"| node_model_store

click node_cli "https://github.com/qualcomm/geniex/blob/main/cli/cmd/geniex/main.go"
click node_infer_command "https://github.com/qualcomm/geniex/blob/main/cli/cmd/geniex/infer.go"
click node_local_service "https://github.com/qualcomm/geniex/blob/main/cli/server/server.go"
click node_chat_handler "https://github.com/qualcomm/geniex/blob/main/cli/server/handler/chat.go"
click node_python_api "https://github.com/qualcomm/geniex/blob/main/bindings/python/geniex/modeling.py"
click node_python_ffi "https://github.com/qualcomm/geniex/blob/main/bindings/python/geniex/_ffi/_api.py"
click node_android_sdk "https://github.com/qualcomm/geniex/blob/main/bindings/android/app/src/main/java/com/geniex/sdk/GenieXSdk.kt"
click node_android_jni "https://github.com/qualcomm/geniex/blob/main/bindings/android/app/src/main/cpp/llm_bridge_jni.cpp"
click node_sdk_api "https://github.com/qualcomm/geniex/blob/main/sdk/include/geniex.h"
click node_sdk_llm "https://github.com/qualcomm/geniex/blob/main/sdk/src/llm.cpp"
click node_sdk_vlm "https://github.com/qualcomm/geniex/blob/main/sdk/src/vlm.cpp"
click node_registry "https://github.com/qualcomm/geniex/blob/main/sdk/src/registry.cpp"
click node_plugin_api "https://github.com/qualcomm/geniex/blob/main/sdk/include/plugin/Plugin.h"
click node_model_ffi "https://github.com/qualcomm/geniex/blob/main/sdk/model-manager/crates/ffi/src/lib.rs"
click node_model_pull "https://github.com/qualcomm/geniex/blob/main/sdk/model-manager/crates/core/src/pull.rs"
click node_model_store "https://github.com/qualcomm/geniex/blob/main/sdk/model-manager/crates/core/src/store.rs"
click node_llama_plugin "https://github.com/qualcomm/geniex/blob/main/sdk/plugins/llama_cpp/src/plugin.cpp"
click node_qairt_plugin "https://github.com/qualcomm/geniex/blob/main/sdk/plugins/qairt/src/plugin.cpp"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_infer_command,node_local_service,node_chat_handler,node_python_api,node_python_ffi,node_android_sdk,node_android_jni toneBlue
class node_sdk_api,node_sdk_llm,node_sdk_vlm,node_registry,node_plugin_api toneAmber
class node_model_ffi,node_model_pull,node_model_sources,node_model_store toneMint
class node_llama_plugin,node_qairt_plugin toneRose
```

## Components

### Client Surfaces

- [Go CLI](https://github.com/qualcomm/geniex/blob/main/cli/cmd/geniex/main.go): Go command · Coordinates pull, run, and inference workflows. · `cli/cmd/geniex/main.go`
- [CLI inference flow](https://github.com/qualcomm/geniex/blob/main/cli/cmd/geniex/infer.go): Go command · `cli/cmd/geniex/infer.go`
- [OpenAI-compatible service](https://github.com/qualcomm/geniex/blob/main/cli/server/server.go): Go HTTP server · Serves local chat and model requests. · `cli/server/server.go`
- [Chat handler](https://github.com/qualcomm/geniex/blob/main/cli/server/handler/chat.go): HTTP handler · `cli/server/handler/chat.go`
- [Python modeling API](https://github.com/qualcomm/geniex/blob/main/bindings/python/geniex/modeling.py): Python API · Transformers-like model construction and streaming generation. · `bindings/python/geniex/modeling.py`
- [Python C FFI](https://github.com/qualcomm/geniex/blob/main/bindings/python/geniex/_ffi/_api.py): Python FFI · `bindings/python/geniex/_ffi/_api.py`
- [Android Kotlin SDK](https://github.com/qualcomm/geniex/blob/main/bindings/android/app/src/main/java/com/geniex/sdk/GenieXSdk.kt): Kotlin API · `bindings/android/app/src/main/java/com/geniex/sdk/GenieXSdk.kt`
- [Android JNI bridge](https://github.com/qualcomm/geniex/blob/main/bindings/android/app/src/main/cpp/llm_bridge_jni.cpp): C++ JNI · `bindings/android/app/src/main/cpp/llm_bridge_jni.cpp`

### Native SDK

- [SDK public API](https://github.com/qualcomm/geniex/blob/main/sdk/include/geniex.h): C/C++ API · Common LLM and VLM interface. · `sdk/include/geniex.h`
- [LLM lifecycle](https://github.com/qualcomm/geniex/blob/main/sdk/src/llm.cpp): C++ subsystem · Selects implementations and manages LLM inference. · `sdk/src/llm.cpp`
- [VLM lifecycle](https://github.com/qualcomm/geniex/blob/main/sdk/src/vlm.cpp): C++ subsystem · Manages vision-language inference. · `sdk/src/vlm.cpp`
- [Runtime registry](https://github.com/qualcomm/geniex/blob/main/sdk/src/registry.cpp): C++ registry · Discovers devices and enumerates available runtimes. · `sdk/src/registry.cpp`
- [Plugin interface](https://github.com/qualcomm/geniex/blob/main/sdk/include/plugin/Plugin.h): C++ plugin API · Dynamic boundary for inference engines. · `sdk/include/plugin/Plugin.h`

### Model Management

- [Model manager FFI](https://github.com/qualcomm/geniex/blob/main/sdk/model-manager/crates/ffi/src/lib.rs): Rust FFI · Exposes acquisition and local-model operations to native clients. · `sdk/model-manager/crates/ffi/src/lib.rs`
- [Artifact resolver and pull](https://github.com/qualcomm/geniex/blob/main/sdk/model-manager/crates/core/src/pull.rs): Rust subsystem · Resolves references, downloads, validates, and resumes artifacts. · `sdk/model-manager/crates/core/src/pull.rs`
- **Artifact sources**: Rust source adapters · AI Hub, Hugging Face, Docker Hub, and local filesystem sources.
- [Local model store](https://github.com/qualcomm/geniex/blob/main/sdk/model-manager/crates/core/src/store.rs): Rust store · Persists installed model artifacts. · `sdk/model-manager/crates/core/src/store.rs`

### Inference Engines

- [llama.cpp plugin](https://github.com/qualcomm/geniex/blob/main/sdk/plugins/llama_cpp/src/plugin.cpp): C++ inference plugin · Runs GGUF models on CPU, Adreno GPU, and Hexagon HTP. · `sdk/plugins/llama_cpp/src/plugin.cpp`
- [QAIRT plugin](https://github.com/qualcomm/geniex/blob/main/sdk/plugins/qairt/src/plugin.cpp): C++ inference plugin · Runs Qualcomm AI Engine Direct bundles on the NPU. · `sdk/plugins/qairt/src/plugin.cpp`

## Connections

- Go CLI → CLI inference flow: implements
- Go CLI → OpenAI-compatible service: serve command
- CLI inference flow → SDK public API: invokes
- OpenAI-compatible service → Chat handler: routes chat
- Chat handler → SDK public API: runs inference
- Python modeling API → Python C FFI: uses
- Python C FFI → SDK public API: calls
- Android Kotlin SDK → Android JNI bridge: uses
- Android JNI bridge → SDK public API: calls
- SDK public API → LLM lifecycle: LLM requests
- SDK public API → VLM lifecycle: VLM requests
- LLM lifecycle → Runtime registry: selects runtime
- VLM lifecycle → Runtime registry: selects runtime
- Runtime registry → Plugin interface: loads implementation
- Plugin interface → llama.cpp plugin: implements
- Plugin interface → QAIRT plugin: implements
- SDK public API → Model manager FFI: model operations
- Model manager FFI → Artifact resolver and pull: pulls models
- Artifact resolver and pull → Artifact sources: fetches from
- Artifact resolver and pull → Local model store: installs into