How kimi-syl/pocketllm works
PocketLLM is an on-device GGUF inference app and OpenAI-compatible server, with a shared JVM core also used by a desktop CLI. The main flow is model acquisition/loading, chat or agent generation, then a streamed or JSON response to the user or API client. Android also includes a companion overlay; the tree contains a substantial Live2D Cubism SDK used for character rendering. Edges follow README statements and sampled references; unsampled wiring is omitted rather than inferred.
Main entry points: LlamaEngine.kt Llama inference · AgentLoop.kt Agent loop · Main.kt CLI commands · ModelRepository.kt Model discovery
Components 20
Android app
- Compose app
app/src/main/java/com/pocketllm/ui - Model discovery
app/src/main/java/com/pocketllm/models/ModelRepository.kt - API server
app/src/main/java/com/pocketllm/server/ApiServer.kt - API keys
app/src/main/java/com/pocketllm/keys/ApiKeyRepository.kt - Usage logs
app/src/main/java/com/pocketllm/usage/UsageRepository.kt - Companion overlay
app/src/main/java/com/pocketllm/companion/CompanionOverlayService.kt
Shared engine
- Agent loop
core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt - Tool routing
core/src/commonMain/kotlin/com/pocketllm/agent/ToolRouter.kt - Chat engine API
core/src/commonMain/kotlin/com/pocketllm/llm/ChatEngine.kt - Llama inference
core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt - JNI bridge
core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaBridge.kt - HF discovery
core/src/jvmMain/kotlin/com/pocketllm/hf/HuggingFaceClient.kt
Desktop CLI
- CLI commands
cli/src/main/kotlin/com/pocketllm/cli/Main.kt
Live2D framework
- Cubism lifecycle
app/src/main/java/com/live2d/sdk/cubism/framework/CubismFramework.java - Cubism model
app/src/main/java/com/live2d/sdk/cubism/framework/model/CubismModel.java - Motion system
app/src/main/java/com/live2d/sdk/cubism/framework/motion/ACubismMotion.java - Android renderer
app/src/main/java/com/live2d/sdk/cubism/framework/rendering/android/CubismRendererAndroid.java - Clipping masks
app/src/main/java/com/live2d/sdk/cubism/framework/rendering/ACubismClippingManager.java
Other components
- User
- API client
Connections 15
- User → Compose app: uses
README.md - User → Model discovery: downloads model
README.md - Model discovery → HF discovery: browses models
- User → API client: sends prompt
README.md - API client → API server: requests completion
README.md - API server → Llama inference: generates response
- Llama inference → JNI bridge: calls JNI
core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt - Chat engine API → Llama inference: implemented by
core/src/jvmMain/kotlin/com/pocketllm/llm/LlamaEngine.kt - Agent loop → Tool routing: routes tools
core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt - Agent loop → Chat engine API: generates with
core/src/commonMain/kotlin/com/pocketllm/agent/AgentLoop.kt - CLI commands → Agent loop: runs agent
cli/src/main/kotlin/com/pocketllm/cli/Main.kt - CLI commands → Llama inference: runs chat
cli/src/main/kotlin/com/pocketllm/cli/Main.kt - Companion overlay → Llama inference: uses engine
app/src/main/java/com/pocketllm/companion/CompanionOverlayService.kt - Cubism lifecycle → Cubism model: initializes IDs
app/src/main/java/com/live2d/sdk/cubism/framework/model/CubismModel.java - Android renderer → Clipping masks: renders masks