# tcpipuk/llama-go architecture

> Architecture diagram of the GitHub repository tcpipuk/llama-go, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/tcpipuk/llama-go
- Repository: https://github.com/tcpipuk/llama-go

## Overview

llama-go is an idiomatic Go binding for running llama.cpp models locally. The principal workflow loads GGUF model weights, creates independent inference contexts, then performs chat or text generation and returns results to the Go application. The Go API and C++ wrapper bridge to llama.cpp’s model and decoding APIs; GGML supplies tensor and backend abstractions. The README documents additional embedding, streaming, prefix-cache, speculative-decoding, and accelerator capabilities. Some Go implementation files and optional backend wiring were not excerpted, so those documented branches are shown without speculative internal call chains.

Main entry points: [`wrapper.cpp`](https://github.com/tcpipuk/llama-go/blob/main/wrapper.cpp) (C++ wrapper), [`context.go`](https://github.com/tcpipuk/llama-go/blob/main/context.go) (Inference context), [`model.go`](https://github.com/tcpipuk/llama-go/blob/main/model.go) (Model loading), [`chat.go`](https://github.com/tcpipuk/llama-go/blob/main/chat.go) (Chat API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_api["Go API"]
  node_model["Model loading<br/>[model.go]"]
  node_modelopts["Model options<br/>[options_model.go]"]
  node_context["Inference context<br/>[context.go]"]
  node_contextopts["Context options<br/>[options_context.go]"]
  node_generation["Text generation"]
  node_chat["Chat API<br/>[chat.go]"]
  node_chatopts["Chat options<br/>[chat_options.go]"]
  node_tools["Chat tools<br/>[chat_tools.go]"]
  node_streaming["Token streaming"]
  node_stats["Inference stats<br/>[stats.go]"]
end

subgraph group_bridge["CGO bridge"]
  node_wrapper["C++ wrapper<br/>[wrapper.cpp]"]
  node_wrapperapi["Wrapper interface<br/>[wrapper.h]"]
end

subgraph group_engine["Inference engine"]
  node_llama["Model and decoding<br/>[llama.h]"]
  node_chatengine["Template and parsing<br/>[chat.h]"]
  node_speculative["Speculative decoding<br/>[speculative.h]"]
  node_tensor["GGML tensors<br/>[ggml.h]"]
end

subgraph group_acceleration["Compute backends"]
  node_backends["Backend execution<br/>[ggml-backend.h]"]
end

node_app(("Go application"))
node_gpu(("GPU hardware"))
node_modeldata["GGUF model file"]

node_app -->|"loads model"| node_model
node_modelopts -->|"configures"| node_model
node_modeldata -->|"provides weights"| node_model
node_model -->|"creates contexts"| node_context
node_contextopts -->|"configures"| node_context
node_app -->|"submits messages"| node_chat
node_chat -->|"uses context"| node_context
node_app -->|"submits prompt"| node_generation
node_generation -->|"generates with"| node_context
node_chatopts -->|"configures"| node_chat
node_chat -->|"supports tool calls"| node_tools
node_context -->|"streams tokens"| node_streaming
node_context -->|"reports metrics"| node_stats
node_model -->|"calls through CGO"| node_wrapper
node_context -->|"calls through CGO"| node_wrapper
node_wrapper -->|"implements interface"| node_wrapperapi
node_wrapper -->|"calls"| node_llama
node_wrapper -->|"uses chat support"| node_chatengine
node_wrapper -->|"uses decoding support"| node_speculative
node_llama -->|"uses tensor API"| node_tensor
node_llama -->|"uses backend API"| node_backends
node_backends -.->|"dispatches compute"| node_gpu
node_context -.->|"extracts embeddings"| node_generation
node_wrapper -->|"reuses prompt prefix"| node_wrapper

click node_model "https://github.com/tcpipuk/llama-go/blob/main/model.go"
click node_modelopts "https://github.com/tcpipuk/llama-go/blob/main/options_model.go"
click node_context "https://github.com/tcpipuk/llama-go/blob/main/context.go"
click node_contextopts "https://github.com/tcpipuk/llama-go/blob/main/options_context.go"
click node_generation "https://github.com/tcpipuk/llama-go/blob/main/options_generate.go"
click node_chat "https://github.com/tcpipuk/llama-go/blob/main/chat.go"
click node_chatopts "https://github.com/tcpipuk/llama-go/blob/main/chat_options.go"
click node_tools "https://github.com/tcpipuk/llama-go/blob/main/chat_tools.go"
click node_streaming "https://github.com/tcpipuk/llama-go/blob/main/progress_callback.go"
click node_stats "https://github.com/tcpipuk/llama-go/blob/main/stats.go"
click node_wrapper "https://github.com/tcpipuk/llama-go/blob/main/wrapper.cpp"
click node_wrapperapi "https://github.com/tcpipuk/llama-go/blob/main/wrapper.h"
click node_llama "https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/include/llama.h"
click node_chatengine "https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/common/chat.h"
click node_speculative "https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/common/speculative.h"
click node_tensor "https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/ggml/include/ggml.h"
click node_backends "https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/ggml/include/ggml-backend.h"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_model,node_modelopts,node_context,node_contextopts,node_generation,node_chat,node_chatopts,node_tools,node_streaming,node_stats toneBlue
class node_wrapper,node_wrapperapi toneAmber
class node_llama,node_chatengine,node_speculative,node_tensor toneMint
class node_backends toneRose
class node_app,node_gpu,node_modeldata toneIndigo
```

## Components

### Go API

- [Model loading](https://github.com/tcpipuk/llama-go/blob/main/model.go): `model.go`
- [Model options](https://github.com/tcpipuk/llama-go/blob/main/options_model.go): `options_model.go`
- [Inference context](https://github.com/tcpipuk/llama-go/blob/main/context.go): `context.go`
- [Context options](https://github.com/tcpipuk/llama-go/blob/main/options_context.go): `options_context.go`
- [Text generation](https://github.com/tcpipuk/llama-go/blob/main/options_generate.go): `options_generate.go`
- [Chat API](https://github.com/tcpipuk/llama-go/blob/main/chat.go): `chat.go`
- [Chat options](https://github.com/tcpipuk/llama-go/blob/main/chat_options.go): `chat_options.go`
- [Chat tools](https://github.com/tcpipuk/llama-go/blob/main/chat_tools.go): `chat_tools.go`
- [Token streaming](https://github.com/tcpipuk/llama-go/blob/main/progress_callback.go): `progress_callback.go`
- [Inference stats](https://github.com/tcpipuk/llama-go/blob/main/stats.go): `stats.go`

### CGO bridge

- [C++ wrapper](https://github.com/tcpipuk/llama-go/blob/main/wrapper.cpp): `wrapper.cpp`
- [Wrapper interface](https://github.com/tcpipuk/llama-go/blob/main/wrapper.h): `wrapper.h`

### Inference engine

- [Model and decoding](https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/include/llama.h): `cgo_headers/llama.cpp/include/llama.h`
- [Template and parsing](https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/common/chat.h): `cgo_headers/llama.cpp/common/chat.h`
- [Speculative decoding](https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/common/speculative.h): `cgo_headers/llama.cpp/common/speculative.h`
- [GGML tensors](https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/ggml/include/ggml.h): `cgo_headers/llama.cpp/ggml/include/ggml.h`

### Compute backends

- [Backend execution](https://github.com/tcpipuk/llama-go/blob/main/cgo_headers/llama.cpp/ggml/include/ggml-backend.h): `cgo_headers/llama.cpp/ggml/include/ggml-backend.h`

### Other components

- **Go application**
- **GPU hardware**
- **GGUF model file**

## Connections

- Go application → Model loading: loads model (evidence: `README.md`)
- Model options → Model loading: configures (evidence: `README.md`)
- GGUF model file → Model loading: provides weights (evidence: `README.md`)
- Model loading → Inference context: creates contexts (evidence: `README.md`)
- Context options → Inference context: configures (evidence: `README.md`)
- Go application → Chat API: submits messages (evidence: `README.md`)
- Chat API → Inference context: uses context (evidence: `README.md`)
- Go application → Text generation: submits prompt (evidence: `README.md`)
- Text generation → Inference context: generates with (evidence: `README.md`)
- Chat options → Chat API: configures (evidence: `README.md`)
- Chat API → Chat tools: supports tool calls (evidence: `README.md`)
- Inference context → Token streaming: streams tokens (evidence: `README.md`)
- Inference context → Inference stats: reports metrics (evidence: `README.md`)
- Model loading → C++ wrapper: calls through CGO (evidence: `README.md`)
- Inference context → C++ wrapper: calls through CGO (evidence: `README.md`)
- C++ wrapper → Wrapper interface: implements interface (evidence: `wrapper.cpp`)
- C++ wrapper → Model and decoding: calls (evidence: `wrapper.cpp`)
- C++ wrapper → Template and parsing: uses chat support (evidence: `wrapper.cpp`)
- C++ wrapper → Speculative decoding: uses decoding support (evidence: `wrapper.cpp`)
- Model and decoding → GGML tensors: uses tensor API (evidence: `cgo_headers/llama.cpp/include/llama.h`)
- Model and decoding → Backend execution: uses backend API (evidence: `cgo_headers/llama.cpp/include/llama.h`)
- Backend execution → GPU hardware: dispatches compute
- Inference context → Text generation: extracts embeddings
- C++ wrapper → C++ wrapper: reuses prompt prefix (evidence: `wrapper.cpp`)
