# bhangun/gollek architecture

> Architecture diagram of the GitHub repository bhangun/gollek, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-03). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhangun/gollek
- Repository: https://github.com/bhangun/gollek

## Overview

Gollek is an AI inference engine and serving layer. Its user-facing workflow is to obtain or select a model, submit a prompt through the CLI or SDK, run inference on a supported backend, and return generated output; the CLI also exposes an API server. Major documented capabilities include model management, multimodal and speech/vision runners, quantization, and observability. The source excerpts show parts of runtime execution, model registration, and prefill/decode optimization; connections not established by those excerpts are included only where the README documents them.

Main entry points: [`runtime/core/gollek-runtime-core/src/main/java/tech/kayys/gollek/runtime/ExecutionEngine.java`](https://github.com/bhangun/gollek/blob/main/runtime/core/gollek-runtime-core/src/main/java/tech/kayys/gollek/runtime/ExecutionEngine.java) (Execution Runtime), [`framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceEngine.java`](https://github.com/bhangun/gollek/blob/main/framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceEngine.java) (Inference SPI), [`ui/gollek-cli`](https://github.com/bhangun/gollek/tree/main/ui/gollek-cli) (CLI), [`framework/sdk/gollek-sdk-core/src/main/java/tech/kayys/gollek/sdk/core/GollekChatService.java`](https://github.com/bhangun/gollek/blob/main/framework/sdk/gollek-sdk-core/src/main/java/tech/kayys/gollek/sdk/core/GollekChatService.java) (SDK Chat)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_clients["Clients and Serving"]
  node_cli["CLI"]
  node_sdk["SDK Chat"]
  node_api["API Server"]
  node_grpc["gRPC Service"]
end

subgraph group_models["Model Management"]
  node_model_registry["Model Registry"]
  node_model_download["Model Download"]
  node_hf_repo["Hugging Face"]
end

subgraph group_inference["Inference Runtime"]
  node_request["Inference Request"]
  node_engine_spi["Inference SPI"]
  node_runtime["Execution Runtime"]
  node_prefill_decode["Prefill Decode"]
  node_core["Tensor and IR Core<br/>[GValueId.java]"]
  node_response["Inference Result"]
end

subgraph group_backends["Execution Backends"]
  node_gguf["GGUF Runner"]
  node_safetensor["SafeTensor Runner"]
  node_onnx["ONNX Runner<br/>[OnnxEngine.java]"]
  node_litert["LiteRT Runner"]
  node_torch["LibTorch Runner"]
end

subgraph group_operations["Operations"]
  node_observability["Metrics and Audit"]
end

node_user(("User"))

node_user -->|"invokes"| node_cli
node_user -->|"uses"| node_sdk
node_user -->|"starts"| node_api
node_cli -->|"pulls models"| node_model_download
node_model_download -->|"downloads from"| node_hf_repo
node_cli -->|"lists models"| node_model_registry
node_sdk -->|"builds request"| node_request
node_sdk -.->|"submits inference"| node_engine_spi
node_grpc -->|"serves inference"| node_engine_spi
node_api -.->|"serves inference"| node_engine_spi
node_request -->|"specifies input"| node_engine_spi
node_engine_spi -.->|"executes via"| node_runtime
node_runtime -->|"uses operators"| node_core
node_runtime -.->|"dispatches to"| node_safetensor
node_runtime -.->|"dispatches to"| node_gguf
node_runtime -.->|"dispatches to"| node_onnx
node_runtime -.->|"dispatches to"| node_litert
node_runtime -.->|"dispatches to"| node_torch
node_prefill_decode -->|"reads request"| node_request
node_prefill_decode -.->|"produces output"| node_response
node_engine_spi -->|"returns result"| node_response
node_engine_spi -.->|"reports metrics"| node_observability

click node_cli "https://github.com/bhangun/gollek/tree/main/ui/gollek-cli"
click node_sdk "https://github.com/bhangun/gollek/blob/main/framework/sdk/gollek-sdk-core/src/main/java/tech/kayys/gollek/sdk/core/GollekChatService.java"
click node_api "https://github.com/bhangun/gollek/tree/main/ui/gollek-api"
click node_grpc "https://github.com/bhangun/gollek/blob/main/ui/gollek-sdk-protobuf/src/main/java/tech/kayys/gollek/protobuf/GollekGrpcService.java"
click node_model_registry "https://github.com/bhangun/gollek/blob/main/framework/repository/gollek-model-database/src/main/java/tech/kayys/gollek/model/registry/ModelRegistryService.java"
click node_model_download "https://github.com/bhangun/gollek/blob/main/framework/repository/gollek-model-database/src/main/java/tech/kayys/gollek/model/domain/download/ModelDownloadManager.java"
click node_hf_repo "https://github.com/bhangun/gollek/blob/main/runtime/repository/gollek-model-repo-hf/src/main/java/tech/kayys/gollek/model/repo/hf/HuggingFaceRepository.java"
click node_request "https://github.com/bhangun/gollek/blob/main/framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceRequest.java"
click node_engine_spi "https://github.com/bhangun/gollek/blob/main/framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceEngine.java"
click node_runtime "https://github.com/bhangun/gollek/blob/main/runtime/core/gollek-runtime-core/src/main/java/tech/kayys/gollek/runtime/ExecutionEngine.java"
click node_prefill_decode "https://github.com/bhangun/gollek/blob/main/runtime/optimization/gollek-plugin-prefill-decode/src/main/java/tech/kayys/gollek/prefilldecode/PrefillDecodeDisaggService.java"
click node_core "https://github.com/bhangun/gollek/blob/main/framework/core/gollek-core/src/main/java/tech/kayys/gollek/ir/GValueId.java"
click node_gguf "https://github.com/bhangun/gollek/tree/main/runtime/runner/gguf/gollek-runner-gguf"
click node_safetensor "https://github.com/bhangun/gollek/tree/main/runtime/runner/safetensor/gollek-safetensor-engine"
click node_onnx "https://github.com/bhangun/gollek/blob/main/runtime/runner/onnx/gollek-runner-onnx/src/main/java/tech/kayys/gollek/provider/onnx/OnnxEngine.java"
click node_litert "https://github.com/bhangun/gollek/tree/main/runtime/runner/litert/gollek-runner-litert"
click node_torch "https://github.com/bhangun/gollek/blob/main/runtime/runner/torch/gollek-runner-libtorch/src/main/java/tech/kayys/gollek/inference/libtorch/LibTorchEngine.java"
click node_observability "https://github.com/bhangun/gollek/tree/main/framework/observability/gollek-observability"
click node_response "https://github.com/bhangun/gollek/blob/main/framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceResponse.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_sdk,node_api,node_grpc,node_user toneBlue
class node_model_registry,node_model_download,node_hf_repo toneAmber
class node_request,node_engine_spi,node_runtime,node_prefill_decode,node_core,node_response toneMint
class node_gguf,node_safetensor,node_onnx,node_litert,node_torch toneRose
class node_observability toneIndigo
```

## Components

### Clients and Serving

- [CLI](https://github.com/bhangun/gollek/tree/main/ui/gollek-cli): `ui/gollek-cli`
- [SDK Chat](https://github.com/bhangun/gollek/blob/main/framework/sdk/gollek-sdk-core/src/main/java/tech/kayys/gollek/sdk/core/GollekChatService.java): `framework/sdk/gollek-sdk-core/src/main/java/tech/kayys/gollek/sdk/core/GollekChatService.java`
- [API Server](https://github.com/bhangun/gollek/tree/main/ui/gollek-api): `ui/gollek-api`
- [gRPC Service](https://github.com/bhangun/gollek/blob/main/ui/gollek-sdk-protobuf/src/main/java/tech/kayys/gollek/protobuf/GollekGrpcService.java): `ui/gollek-sdk-protobuf/src/main/java/tech/kayys/gollek/protobuf/GollekGrpcService.java`

### Model Management

- [Model Registry](https://github.com/bhangun/gollek/blob/main/framework/repository/gollek-model-database/src/main/java/tech/kayys/gollek/model/registry/ModelRegistryService.java): `framework/repository/gollek-model-database/src/main/java/tech/kayys/gollek/model/registry/ModelRegistryService.java`
- [Model Download](https://github.com/bhangun/gollek/blob/main/framework/repository/gollek-model-database/src/main/java/tech/kayys/gollek/model/domain/download/ModelDownloadManager.java): `framework/repository/gollek-model-database/src/main/java/tech/kayys/gollek/model/domain/download/ModelDownloadManager.java`
- [Hugging Face](https://github.com/bhangun/gollek/blob/main/runtime/repository/gollek-model-repo-hf/src/main/java/tech/kayys/gollek/model/repo/hf/HuggingFaceRepository.java): `runtime/repository/gollek-model-repo-hf/src/main/java/tech/kayys/gollek/model/repo/hf/HuggingFaceRepository.java`

### Inference Runtime

- [Inference Request](https://github.com/bhangun/gollek/blob/main/framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceRequest.java): `framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceRequest.java`
- [Inference SPI](https://github.com/bhangun/gollek/blob/main/framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceEngine.java): `framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceEngine.java`
- [Execution Runtime](https://github.com/bhangun/gollek/blob/main/runtime/core/gollek-runtime-core/src/main/java/tech/kayys/gollek/runtime/ExecutionEngine.java): `runtime/core/gollek-runtime-core/src/main/java/tech/kayys/gollek/runtime/ExecutionEngine.java`
- [Prefill Decode](https://github.com/bhangun/gollek/blob/main/runtime/optimization/gollek-plugin-prefill-decode/src/main/java/tech/kayys/gollek/prefilldecode/PrefillDecodeDisaggService.java): `runtime/optimization/gollek-plugin-prefill-decode/src/main/java/tech/kayys/gollek/prefilldecode/PrefillDecodeDisaggService.java`
- [Tensor and IR Core](https://github.com/bhangun/gollek/blob/main/framework/core/gollek-core/src/main/java/tech/kayys/gollek/ir/GValueId.java): `framework/core/gollek-core/src/main/java/tech/kayys/gollek/ir/GValueId.java`
- [Inference Result](https://github.com/bhangun/gollek/blob/main/framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceResponse.java): `framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceResponse.java`

### Execution Backends

- [GGUF Runner](https://github.com/bhangun/gollek/tree/main/runtime/runner/gguf/gollek-runner-gguf): `runtime/runner/gguf/gollek-runner-gguf`
- [SafeTensor Runner](https://github.com/bhangun/gollek/tree/main/runtime/runner/safetensor/gollek-safetensor-engine): `runtime/runner/safetensor/gollek-safetensor-engine`
- [ONNX Runner](https://github.com/bhangun/gollek/blob/main/runtime/runner/onnx/gollek-runner-onnx/src/main/java/tech/kayys/gollek/provider/onnx/OnnxEngine.java): `runtime/runner/onnx/gollek-runner-onnx/src/main/java/tech/kayys/gollek/provider/onnx/OnnxEngine.java`
- [LiteRT Runner](https://github.com/bhangun/gollek/tree/main/runtime/runner/litert/gollek-runner-litert): `runtime/runner/litert/gollek-runner-litert`
- [LibTorch Runner](https://github.com/bhangun/gollek/blob/main/runtime/runner/torch/gollek-runner-libtorch/src/main/java/tech/kayys/gollek/inference/libtorch/LibTorchEngine.java): `runtime/runner/torch/gollek-runner-libtorch/src/main/java/tech/kayys/gollek/inference/libtorch/LibTorchEngine.java`

### Operations

- [Metrics and Audit](https://github.com/bhangun/gollek/tree/main/framework/observability/gollek-observability): `framework/observability/gollek-observability`

### Other components

- **User**

## Connections

- User → CLI: invokes (evidence: `README.md`)
- User → SDK Chat: uses (evidence: `README.md`)
- User → API Server: starts (evidence: `README.md`)
- CLI → Model Download: pulls models (evidence: `README.md`)
- Model Download → Hugging Face: downloads from (evidence: `README.md`)
- CLI → Model Registry: lists models (evidence: `README.md`)
- SDK Chat → Inference Request: builds request (evidence: `framework/sdk/gollek-sdk-core/src/main/java/tech/kayys/gollek/sdk/core/GollekChatService.java`)
- SDK Chat → Inference SPI: submits inference
- gRPC Service → Inference SPI: serves inference (evidence: `ui/gollek-sdk-protobuf/src/main/java/tech/kayys/gollek/protobuf/GollekGrpcService.java`)
- API Server → Inference SPI: serves inference
- Inference Request → Inference SPI: specifies input (evidence: `framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceEngine.java`)
- Inference SPI → Execution Runtime: executes via
- Execution Runtime → Tensor and IR Core: uses operators (evidence: `runtime/core/gollek-runtime-core/src/main/java/tech/kayys/gollek/runtime/ExecutionEngine.java`)
- Execution Runtime → SafeTensor Runner: dispatches to
- Execution Runtime → GGUF Runner: dispatches to
- Execution Runtime → ONNX Runner: dispatches to
- Execution Runtime → LiteRT Runner: dispatches to
- Execution Runtime → LibTorch Runner: dispatches to
- Prefill Decode → Inference Request: reads request (evidence: `runtime/optimization/gollek-plugin-prefill-decode/src/main/java/tech/kayys/gollek/prefilldecode/PrefillDecodeDisaggService.java`)
- Prefill Decode → Inference Result: produces output (evidence: `runtime/optimization/gollek-plugin-prefill-decode/src/main/java/tech/kayys/gollek/prefilldecode/PrefillDecodeDisaggService.java`)
- Inference SPI → Inference Result: returns result (evidence: `framework/spi/gollek-spi-inference/src/main/java/tech/kayys/gollek/spi/inference/InferenceEngine.java`)
- Inference SPI → Metrics and Audit: reports metrics (evidence: `framework/observability/gollek-observability/src/main/java/tech/kayys/gollek/observability/InferenceEngineHealthCheck.java`)
