# 9cog/koboldcpp architecture

> Architecture diagram of the GitHub repository 9cog/koboldcpp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/9cog/koboldcpp
- Repository: https://github.com/9cog/koboldcpp

## Overview

KoboldCpp is a self-contained local AI application built around llama.cpp, with text generation as its principal workflow: a user launches the program, supplies a model and prompt, and receives generated text through a UI or compatible API. Its documented capabilities also include image generation, speech recognition and synthesis, embeddings, and image understanding. The graph distinguishes the application/server layer, model inference, and major modality subsystems. Some integration paths are documented only at product level or were not sampled, so those links are dashed and unverified.

Main entry points: [`src/llama-context.cpp`](https://github.com/9cog/koboldcpp/blob/concedo/src/llama-context.cpp) (LLM Engine), [`tools/server/server.cpp`](https://github.com/9cog/koboldcpp/blob/concedo/tools/server/server.cpp) (HTTP Server), [`koboldcpp.py`](https://github.com/9cog/koboldcpp/blob/concedo/koboldcpp.py) (KoboldCpp App), [`tools/server/webui/src/lib/components/app/chat/ChatScreen/ChatScreen.svelte`](https://github.com/9cog/koboldcpp/blob/concedo/tools/server/webui/src/lib/components/app/chat/ChatScreen/ChatScreen.svelte) (Server Web UI)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_application["Application Interfaces"]
  node_launcher["KoboldCpp App<br/>[koboldcpp.py]"]
  node_server["HTTP Server<br/>[server.cpp]"]
  node_webui["Server Web UI<br/>[ChatScreen.svelte]"]
  node_api["Compatible APIs<br/>[server.cpp]"]
end

subgraph group_inference["Text Inference"]
  node_llama["LLM Engine<br/>[llama-context.cpp]"]
  node_sampler["Sampling<br/>[sampling.h]"]
end

subgraph group_modalities["Media Capabilities"]
  node_imagegen["Image Generation"]
  node_stablediffusion_cli["Image Generation CLI<br/>[main.cpp]"]
  node_whisper["Speech Recognition<br/>[whisper.cpp]"]
  node_whisper_cli["Whisper Interface<br/>[main.cpp]"]
  node_tts["Text To Speech<br/>[ttscpp.cpp]"]
  node_outetts["OuteTTS<br/>[tts-outetts.py]"]
  node_embeddings["Embeddings"]
  node_vision["Image Understanding<br/>[mtmd.cpp]"]
  node_clip["Vision Encoder<br/>[clip.cpp]"]
end

subgraph group_runtime["Model Runtime"]
  node_ggml["Tensor Compute<br/>[ggml.cpp]"]
  node_backends["Compute Backends<br/>[ggml-backend.cpp]"]
  node_gguf["GGUF Conversion"]
end

node_user(("User"))
node_model[("GGUF Model")]

node_user -->|"launches"| node_launcher
node_launcher -.->|"starts"| node_server
node_user -.->|"submits prompts"| node_webui
node_webui -.->|"sends requests"| node_server
node_user -.->|"sends requests"| node_api
node_api -.->|"routes requests"| node_server
node_server -->|"initializes backend"| node_llama
node_model -.->|"loads weights"| node_llama
node_llama -->|"executes graphs"| node_ggml
node_llama -.->|"samples tokens"| node_sampler
node_ggml -.->|"dispatches compute"| node_backends
node_user -.->|"requests images"| node_imagegen
node_stablediffusion_cli -.->|"runs generation"| node_imagegen
node_user -.->|"provides audio"| node_whisper
node_whisper_cli -.->|"transcribes audio"| node_whisper
node_user -.->|"requests speech"| node_tts
node_user -.->|"requests speech"| node_outetts
node_user -.->|"requests embeddings"| node_embeddings
node_user -.->|"provides images"| node_vision
node_vision -.->|"uses encoder"| node_clip
node_gguf -.->|"produces format"| node_model

click node_launcher "https://github.com/9cog/koboldcpp/blob/concedo/koboldcpp.py"
click node_server "https://github.com/9cog/koboldcpp/blob/concedo/tools/server/server.cpp"
click node_webui "https://github.com/9cog/koboldcpp/blob/concedo/tools/server/webui/src/lib/components/app/chat/ChatScreen/ChatScreen.svelte"
click node_api "https://github.com/9cog/koboldcpp/blob/concedo/tools/server/server.cpp"
click node_llama "https://github.com/9cog/koboldcpp/blob/concedo/src/llama-context.cpp"
click node_sampler "https://github.com/9cog/koboldcpp/blob/concedo/common/sampling.h"
click node_ggml "https://github.com/9cog/koboldcpp/blob/concedo/ggml/src/ggml.cpp"
click node_backends "https://github.com/9cog/koboldcpp/blob/concedo/ggml/src/ggml-backend.cpp"
click node_imagegen "https://github.com/9cog/koboldcpp/blob/concedo/otherarch/sdcpp/stable-diffusion.cpp"
click node_stablediffusion_cli "https://github.com/9cog/koboldcpp/blob/concedo/otherarch/sdcpp/main.cpp"
click node_whisper "https://github.com/9cog/koboldcpp/blob/concedo/otherarch/whispercpp/whisper.cpp"
click node_whisper_cli "https://github.com/9cog/koboldcpp/blob/concedo/otherarch/whispercpp/main.cpp"
click node_tts "https://github.com/9cog/koboldcpp/blob/concedo/otherarch/ttscpp/src/ttscpp.cpp"
click node_outetts "https://github.com/9cog/koboldcpp/blob/concedo/tools/tts/tts-outetts.py"
click node_embeddings "https://github.com/9cog/koboldcpp/blob/concedo/otherarch/embeddings_adapter.cpp"
click node_vision "https://github.com/9cog/koboldcpp/blob/concedo/tools/mtmd/mtmd.cpp"
click node_clip "https://github.com/9cog/koboldcpp/blob/concedo/tools/mtmd/clip.cpp"
click node_gguf "https://github.com/9cog/koboldcpp/blob/concedo/convert_hf_to_gguf.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_launcher,node_server,node_webui,node_api,node_user toneBlue
class node_llama,node_sampler,node_model toneAmber
class node_imagegen,node_stablediffusion_cli,node_whisper,node_whisper_cli,node_tts,node_outetts,node_embeddings,node_vision,node_clip toneMint
class node_ggml,node_backends,node_gguf toneRose
```

## Components

### Application Interfaces

- [KoboldCpp App](https://github.com/9cog/koboldcpp/blob/concedo/koboldcpp.py): `koboldcpp.py`
- [HTTP Server](https://github.com/9cog/koboldcpp/blob/concedo/tools/server/server.cpp): `tools/server/server.cpp`
- [Server Web UI](https://github.com/9cog/koboldcpp/blob/concedo/tools/server/webui/src/lib/components/app/chat/ChatScreen/ChatScreen.svelte): `tools/server/webui/src/lib/components/app/chat/ChatScreen/ChatScreen.svelte`
- [Compatible APIs](https://github.com/9cog/koboldcpp/blob/concedo/tools/server/server.cpp): `tools/server/server.cpp`

### Text Inference

- [LLM Engine](https://github.com/9cog/koboldcpp/blob/concedo/src/llama-context.cpp): `src/llama-context.cpp`
- [Sampling](https://github.com/9cog/koboldcpp/blob/concedo/common/sampling.h): `common/sampling.h`

### Media Capabilities

- [Image Generation](https://github.com/9cog/koboldcpp/blob/concedo/otherarch/sdcpp/stable-diffusion.cpp): `otherarch/sdcpp/stable-diffusion.cpp`
- [Image Generation CLI](https://github.com/9cog/koboldcpp/blob/concedo/otherarch/sdcpp/main.cpp): `otherarch/sdcpp/main.cpp`
- [Speech Recognition](https://github.com/9cog/koboldcpp/blob/concedo/otherarch/whispercpp/whisper.cpp): `otherarch/whispercpp/whisper.cpp`
- [Whisper Interface](https://github.com/9cog/koboldcpp/blob/concedo/otherarch/whispercpp/main.cpp): `otherarch/whispercpp/main.cpp`
- [Text To Speech](https://github.com/9cog/koboldcpp/blob/concedo/otherarch/ttscpp/src/ttscpp.cpp): `otherarch/ttscpp/src/ttscpp.cpp`
- [OuteTTS](https://github.com/9cog/koboldcpp/blob/concedo/tools/tts/tts-outetts.py): `tools/tts/tts-outetts.py`
- [Embeddings](https://github.com/9cog/koboldcpp/blob/concedo/otherarch/embeddings_adapter.cpp): `otherarch/embeddings_adapter.cpp`
- [Image Understanding](https://github.com/9cog/koboldcpp/blob/concedo/tools/mtmd/mtmd.cpp): `tools/mtmd/mtmd.cpp`
- [Vision Encoder](https://github.com/9cog/koboldcpp/blob/concedo/tools/mtmd/clip.cpp): `tools/mtmd/clip.cpp`

### Model Runtime

- [Tensor Compute](https://github.com/9cog/koboldcpp/blob/concedo/ggml/src/ggml.cpp): `ggml/src/ggml.cpp`
- [Compute Backends](https://github.com/9cog/koboldcpp/blob/concedo/ggml/src/ggml-backend.cpp): `ggml/src/ggml-backend.cpp`
- [GGUF Conversion](https://github.com/9cog/koboldcpp/blob/concedo/convert_hf_to_gguf.py): `convert_hf_to_gguf.py`

### Other components

- **User**
- **GGUF Model**

## Connections

- User → KoboldCpp App: launches
- KoboldCpp App → HTTP Server: starts
- User → Server Web UI: submits prompts
- Server Web UI → HTTP Server: sends requests
- User → Compatible APIs: sends requests
- Compatible APIs → HTTP Server: routes requests (evidence: `tools/server/server.cpp`)
- HTTP Server → LLM Engine: initializes backend (evidence: `tools/server/server.cpp`)
- GGUF Model → LLM Engine: loads weights
- LLM Engine → Tensor Compute: executes graphs (evidence: `src/llama-context.cpp`)
- LLM Engine → Sampling: samples tokens
- Tensor Compute → Compute Backends: dispatches compute
- User → Image Generation: requests images
- Image Generation CLI → Image Generation: runs generation
- User → Speech Recognition: provides audio
- Whisper Interface → Speech Recognition: transcribes audio
- User → Text To Speech: requests speech
- User → OuteTTS: requests speech
- User → Embeddings: requests embeddings
- User → Image Understanding: provides images
- Image Understanding → Vision Encoder: uses encoder
- GGUF Conversion → GGUF Model: produces format
