qualcomm/geniex

Purpose and Entry Points

GenieX is an on-device LLM/VLM inference stack for Snapdragon Windows ARM64, Linux ARM64, and Android. A C/C++ SDK is the core; consumers enter through the Go CLI/server, Python package, Android Kotlin/JNI SDK, or Go bindings. Core public API: sdk/include/geniex.h; CLI executable: cli/cmd/geniex/main.go; Python API: bindings/python/geniex/modeling.py.

Core SDK and Runtime Selection

The SDK owns device discovery, runtime enumeration, model lifecycle, inference, VLM support, logging, and plugin registration. It presents common LLM/VLM interfaces while selecting an implementation based on model/runtime and available compute units. Primary paths: sdk/src/llm.cpp, sdk/src/vlm.cpp, sdk/src/registry.cpp, sdk/include/plugin/Plugin.h.

Inference Plugin Boundary

Inference engines are dynamically separated behind SDK plugin interfaces. The llama_cpp plugin executes GGUF models across CPU, Adreno GPU, and Hexagon/HTP paths; the qairt plugin executes Qualcomm AI Engine Direct bundles on the NPU. Plugin-specific model loading and generation stay below this boundary, allowing all upper interfaces to use the same SDK API. Paths: sdk/plugins/llama_cpp/src/plugin.cpp, sdk/plugins/llama_cpp/src/llm.cpp, sdk/plugins/qairt/src/plugin.cpp, sdk/plugins/qairt/src/llm.cpp.

Model Acquisition and Local Store

A Rust model-manager subsystem resolves model references, queries metadata, downloads and validates artifacts, resumes transfers, and persists locally installed models. It supports Qualcomm AI Hub, Hugging Face, Docker Hub, and local filesystem sources. Its FFI layer exposes these operations to the SDK and language clients. Paths: sdk/model-manager/crates/core/src/pull.rs, sdk/model-manager/crates/core/src/source/, sdk/model-manager/crates/core/src/store.rs, sdk/model-manager/crates/ffi/src/lib.rs.

CLI and Local OpenAI-Compatible Service

The Go CLI coordinates model pull/run/infer workflows with the SDK and provides interactive terminal behavior. Its serve command hosts a local OpenAI-compatible HTTP API, routing chat and model requests through services to the same inference layer; middleware provides authentication, CORS, and concurrency/GIL-related controls. Paths: cli/cmd/geniex/main.go, cli/cmd/geniex/infer.go, cli/server/server.go, cli/server/handler/chat.go.

Language and Android Integration

Python provides Transformers-like model construction, tokenization, streaming generation, and C FFI access. Android provides Kotlin wrappers over JNI bridges for LLM, VLM, and model management. Go bindings offer another native-language SDK surface. These are adapters rather than independent inference engines. Paths: bindings/python/geniex/_ffi/_api.py, bindings/python/geniex/modeling.py, bindings/android/app/src/main/cpp/llm_bridge_jni.cpp, bindings/android/app/src/main/java/com/geniex/sdk/GenieXSdk.kt.

Runtimes, Platforms, and Delivery

Native SDK/plugin code is C/C++ built with CMake and Bazel; model management is Rust; CLI and Go bindings are Go; Python is packaged separately; Android uses Gradle, Kotlin, and JNI. The stack depends on Qualcomm hardware/runtime infrastructure including Hexagon NPU/HTP, Adreno GPU, and Qualcomm AI Engine Direct, plus remote artifact services from AI Hub, Hugging Face, and Docker Hub. Build roots: sdk/CMakeLists.txt, BUILD.bazel, bindings/android/app/build.gradle.kts, sdk/model-manager/Cargo.toml.