# kimi-syl/pocketllm architecture

> Architecture diagram of the GitHub repository kimi-syl/pocketllm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kimi-syl/pocketllm
- Repository: https://github.com/kimi-syl/pocketllm

## Overview

PocketLLM is an on-device GGUF inference app and OpenAI-compatible server, with a shared JVM core also used by a desktop CLI. The main flow is model acquisition/loading, chat or agent generation, then a streamed or JSON response to the user or API client. Android also includes a companion overlay; the tree contains a substantial Live2D Cubism SDK used for character rendering. Edges follow README statements and sampled references; unsampled wiring is omitted rather than inferred.

Main entry points: [`core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt`](https://github.com/kimi-syl/pocketllm/blob/main/core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt) (Llama inference), [`core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt`](https://github.com/kimi-syl/pocketllm/blob/main/core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt) (Agent loop), [`cli/src/main/kotlin/com/pocketllm/cli/Main.kt`](https://github.com/kimi-syl/pocketllm/blob/main/cli/src/main/kotlin/com/pocketllm/cli/Main.kt) (CLI commands), [`app/src/main/java/com/pocketllm/models/ModelRepository.kt`](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/models/ModelRepository.kt) (Model discovery)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_app["Android app"]
  node_app_ui["Compose app"]
  node_model_repo["Model discovery<br/>[ModelRepository.kt]"]
  node_api_server["API server<br/>[ApiServer.kt]"]
  node_keys[("API keys")]
  node_usage[("Usage logs<br/>[UsageRepository.kt]")]
  node_companion["Companion overlay"]
end

subgraph group_core["Shared engine"]
  node_agent["Agent loop<br/>[AgentLoop.kt]"]
  node_router["Tool routing<br/>[ToolRouter.kt]"]
  node_engine_api["Chat engine API<br/>[ChatEngine.kt]"]
  node_llama_engine["Llama inference<br/>[LlamaEngine.kt]"]
  node_native_bridge["JNI bridge<br/>[LlamaBridge.kt]"]
  node_hf["HF discovery"]
end

subgraph group_cli["Desktop CLI"]
  node_cli_entry["CLI commands<br/>[Main.kt]"]
end

subgraph group_cubism["Live2D framework"]
  node_cubism_framework["Cubism lifecycle"]
  node_cubism_model["Cubism model<br/>[CubismModel.java]"]
  node_cubism_motion["Motion system<br/>[ACubismMotion.java]"]
  node_cubism_render["Android renderer"]
  node_clip_manager["Clipping masks"]
end

node_user(("User"))
node_api_client(("API client"))

node_user -->|"uses"| node_app_ui
node_user -->|"downloads model"| node_model_repo
node_model_repo -.->|"browses models"| node_hf
node_user -->|"sends prompt"| node_api_client
node_api_client -->|"requests completion"| node_api_server
node_api_server -.->|"generates response"| node_llama_engine
node_llama_engine -->|"calls JNI"| node_native_bridge
node_engine_api -->|"implemented by"| node_llama_engine
node_agent -.->|"routes tools"| node_router
node_agent -.->|"generates with"| node_engine_api
node_cli_entry -->|"runs agent"| node_agent
node_cli_entry -->|"runs chat"| node_llama_engine
node_companion -->|"uses engine"| node_llama_engine
node_cubism_framework -.->|"initializes IDs"| node_cubism_model
node_cubism_render -.->|"renders masks"| node_clip_manager

click node_app_ui "https://github.com/kimi-syl/pocketllm/tree/main/app/src/main/java/com/pocketllm/ui"
click node_model_repo "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/models/ModelRepository.kt"
click node_api_server "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/server/ApiServer.kt"
click node_keys "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/keys/ApiKeyRepository.kt"
click node_usage "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/usage/UsageRepository.kt"
click node_companion "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/companion/CompanionOverlayService.kt"
click node_agent "https://github.com/kimi-syl/pocketllm/blob/main/core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt"
click node_router "https://github.com/kimi-syl/pocketllm/blob/main/core/src/commonMain/kotlin/com/pocketllm/agent/ToolRouter.kt"
click node_engine_api "https://github.com/kimi-syl/pocketllm/blob/main/core/src/commonMain/kotlin/com/pocketllm/llm/ChatEngine.kt"
click node_llama_engine "https://github.com/kimi-syl/pocketllm/blob/main/core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt"
click node_native_bridge "https://github.com/kimi-syl/pocketllm/blob/main/core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaBridge.kt"
click node_hf "https://github.com/kimi-syl/pocketllm/blob/main/core/src/jvmMain/kotlin/com/pocketllm/hf/HuggingFaceClient.kt"
click node_cli_entry "https://github.com/kimi-syl/pocketllm/blob/main/cli/src/main/kotlin/com/pocketllm/cli/Main.kt"
click node_cubism_framework "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/CubismFramework.java"
click node_cubism_model "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/model/CubismModel.java"
click node_cubism_motion "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/motion/ACubismMotion.java"
click node_cubism_render "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/rendering/android/CubismRendererAndroid.java"
click node_clip_manager "https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/rendering/ACubismClippingManager.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app_ui,node_model_repo,node_api_server,node_keys,node_usage,node_companion,node_user,node_api_client toneBlue
class node_agent,node_router,node_engine_api,node_llama_engine,node_native_bridge,node_hf toneAmber
class node_cli_entry toneMint
class node_cubism_framework,node_cubism_model,node_cubism_motion,node_cubism_render,node_clip_manager toneRose
```

## Components

### Android app

- [Compose app](https://github.com/kimi-syl/pocketllm/tree/main/app/src/main/java/com/pocketllm/ui): `app/src/main/java/com/pocketllm/ui`
- [Model discovery](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/models/ModelRepository.kt): `app/src/main/java/com/pocketllm/models/ModelRepository.kt`
- [API server](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/server/ApiServer.kt): `app/src/main/java/com/pocketllm/server/ApiServer.kt`
- [API keys](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/keys/ApiKeyRepository.kt): `app/src/main/java/com/pocketllm/keys/ApiKeyRepository.kt`
- [Usage logs](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/usage/UsageRepository.kt): `app/src/main/java/com/pocketllm/usage/UsageRepository.kt`
- [Companion overlay](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/pocketllm/companion/CompanionOverlayService.kt): `app/src/main/java/com/pocketllm/companion/CompanionOverlayService.kt`

### Shared engine

- [Agent loop](https://github.com/kimi-syl/pocketllm/blob/main/core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt): `core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt`
- [Tool routing](https://github.com/kimi-syl/pocketllm/blob/main/core/src/commonMain/kotlin/com/pocketllm/agent/ToolRouter.kt): `core/src/commonMain/kotlin/com/pocketllm/agent/ToolRouter.kt`
- [Chat engine API](https://github.com/kimi-syl/pocketllm/blob/main/core/src/commonMain/kotlin/com/pocketllm/llm/ChatEngine.kt): `core/src/commonMain/kotlin/com/pocketllm/llm/ChatEngine.kt`
- [Llama inference](https://github.com/kimi-syl/pocketllm/blob/main/core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt): `core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt`
- [JNI bridge](https://github.com/kimi-syl/pocketllm/blob/main/core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaBridge.kt): `core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaBridge.kt`
- [HF discovery](https://github.com/kimi-syl/pocketllm/blob/main/core/src/jvmMain/kotlin/com/pocketllm/hf/HuggingFaceClient.kt): `core/src/jvmMain/kotlin/com/pocketllm/hf/HuggingFaceClient.kt`

### Desktop CLI

- [CLI commands](https://github.com/kimi-syl/pocketllm/blob/main/cli/src/main/kotlin/com/pocketllm/cli/Main.kt): `cli/src/main/kotlin/com/pocketllm/cli/Main.kt`

### Live2D framework

- [Cubism lifecycle](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/CubismFramework.java): `app/src/main/java/com/live2d/sdk/cubism/framework/CubismFramework.java`
- [Cubism model](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/model/CubismModel.java): `app/src/main/java/com/live2d/sdk/cubism/framework/model/CubismModel.java`
- [Motion system](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/motion/ACubismMotion.java): `app/src/main/java/com/live2d/sdk/cubism/framework/motion/ACubismMotion.java`
- [Android renderer](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/rendering/android/CubismRendererAndroid.java): `app/src/main/java/com/live2d/sdk/cubism/framework/rendering/android/CubismRendererAndroid.java`
- [Clipping masks](https://github.com/kimi-syl/pocketllm/blob/main/app/src/main/java/com/live2d/sdk/cubism/framework/rendering/ACubismClippingManager.java): `app/src/main/java/com/live2d/sdk/cubism/framework/rendering/ACubismClippingManager.java`

### Other components

- **User**
- **API client**

## Connections

- User → Compose app: uses (evidence: `README.md`)
- User → Model discovery: downloads model (evidence: `README.md`)
- Model discovery → HF discovery: browses models
- User → API client: sends prompt (evidence: `README.md`)
- API client → API server: requests completion (evidence: `README.md`)
- API server → Llama inference: generates response
- Llama inference → JNI bridge: calls JNI (evidence: `core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt`)
- Chat engine API → Llama inference: implemented by (evidence: `core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt`)
- Agent loop → Tool routing: routes tools (evidence: `core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt`)
- Agent loop → Chat engine API: generates with (evidence: `core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt`)
- CLI commands → Agent loop: runs agent (evidence: `cli/src/main/kotlin/com/pocketllm/cli/Main.kt`)
- CLI commands → Llama inference: runs chat (evidence: `cli/src/main/kotlin/com/pocketllm/cli/Main.kt`)
- Companion overlay → Llama inference: uses engine (evidence: `app/src/main/java/com/pocketllm/companion/CompanionOverlayService.kt`)
- Cubism lifecycle → Cubism model: initializes IDs (evidence: `app/src/main/java/com/live2d/sdk/cubism/framework/model/CubismModel.java`)
- Android renderer → Clipping masks: renders masks
