How tcpipuk/llama-go works
llama-go is an idiomatic Go binding for running llama.cpp models locally. The principal workflow loads GGUF model weights, creates independent inference contexts, then performs chat or text generation and returns results to the Go application. The Go API and C++ wrapper bridge to llama.cpp’s model and decoding APIs; GGML supplies tensor and backend abstractions. The README documents additional embedding, streaming, prefix-cache, speculative-decoding, and accelerator capabilities. Some Go implementation files and optional backend wiring were not excerpted, so those documented branches are shown without speculative internal call chains.
Main entry points: wrapper.cpp C++ wrapper · context.go Inference context · model.go Model loading · chat.go Chat API
Components 20
Go API
- Model loading
model.go - Model options
options_model.go - Inference context
context.go - Context options
options_context.go - Text generation
options_generate.go - Chat API
chat.go - Chat options
chat_options.go - Chat tools
chat_tools.go - Token streaming
progress_callback.go - Inference stats
stats.go
CGO bridge
- C++ wrapper
wrapper.cpp - Wrapper interface
wrapper.h
Inference engine
- Model and decoding
cgo_headers/llama.cpp/include/llama.h - Template and parsing
cgo_headers/llama.cpp/common/chat.h - Speculative decoding
cgo_headers/llama.cpp/common/speculative.h - GGML tensors
cgo_headers/llama.cpp/ggml/include/ggml.h
Compute backends
- Backend execution
cgo_headers/llama.cpp/ggml/include/ggml-backend.h
Other components
- Go application
- GPU hardware
- GGUF model file
Connections 24
- Go application → Model loading: loads model
README.md - Model options → Model loading: configures
README.md - GGUF model file → Model loading: provides weights
README.md - Model loading → Inference context: creates contexts
README.md - Context options → Inference context: configures
README.md - Go application → Chat API: submits messages
README.md - Chat API → Inference context: uses context
README.md - Go application → Text generation: submits prompt
README.md - Text generation → Inference context: generates with
README.md - Chat options → Chat API: configures
README.md - Chat API → Chat tools: supports tool calls
README.md - Inference context → Token streaming: streams tokens
README.md - Inference context → Inference stats: reports metrics
README.md - Model loading → C++ wrapper: calls through CGO
README.md - Inference context → C++ wrapper: calls through CGO
README.md - C++ wrapper → Wrapper interface: implements interface
wrapper.cpp - C++ wrapper → Model and decoding: calls
wrapper.cpp - C++ wrapper → Template and parsing: uses chat support
wrapper.cpp - C++ wrapper → Speculative decoding: uses decoding support
wrapper.cpp - Model and decoding → GGML tensors: uses tensor API
cgo_headers/llama.cpp/include/llama.h - Model and decoding → Backend execution: uses backend API
cgo_headers/llama.cpp/include/llama.h - Backend execution → GPU hardware: dispatches compute
- Inference context → Text generation: extracts embeddings
- C++ wrapper → C++ wrapper: reuses prompt prefix
wrapper.cpp