How while-basic/celaya works
Project type and purpose
• This is a multi-component local LLM platform (“Ollama”) that includes:
• A command-line client (cmd/, main.go) and desktop UI (app/, macapp/, app/tray/)
• A long-running local daemon or server (server/, llm/, ml/) exposing a REST API
• A model execution engine (wrapping llama.cpp via ml/backend, convert/, model/, kvcache/, sample/)
• File-system and cache management (fs/, server/internal/cache, app/store/)
• External model registry integration (server/internal/client/ollama, api/)
File structure
• Top-level CLI client: main.go, cmd/, runner/.
• Daemon/server: server/ hosts routes, handlers, internal caches, registry, scheduler.
• Model loader & executor: convert/, ml/backend/ggml, model/, kvcache/, sample/.
• GPU/CPU discovery: discover/.
• UI layers: native tray (app/tray/), cross-platform GUI assets (app/assets/), Electron/mac app (macapp/).
• Configuration & build: Dockerfile, Makefile.sync, CMakeLists.txt, scripts/.
• Integration & unit tests sprinkled across modules.
Key insights from README
• CLI commands (ollama run, create, pull, serve) communicate with a local server (default localhost:11434).
• REST API for generate/chat/manage models (api/).
• Supports model import (GGUF, Safetensors) via a Modelfile.
• Desktop UX: system tray, native menus.
Relationships & interactions
• CLI/Desktop UI → HTTP/gRPC → Local Server
• Server → reads/writes → Local FS (models, blobs, cache)
• Server → HTTP → External Registry (pull/push models)
• Server → In-process calls → Model loader, convertors, inference engine
• Inference engine → native calls → llama.cpp (CPU/GPU backends)
• GPU discovery module → OS/driver APIs → detect hardware capabilities
• Server → Scheduler → dispatch jobs to model runner
• Server → REST API → client SDKs & examples
Architectural patterns & design principles
• Layered architecture: Presentation → Application → Domain → Infrastructure
• Modular packages: separation of concerns (convert, model, runner, server, UI)
• Adapter pattern: ml/backend and discover abstract over hardware backends
• Cache and registry abstractions for pluggable storage backends
Technology call-outs
• Go for core implementation
• C/C++ (llama.cpp) bridged via cgo
• Electron/TypeScript for macapp (macapp/)
• GGML for inference (ml/backend/ggml)
• REST API served by Go’s net/http (server/)
• File I/O and blob cache (fs/, server/internal/cache/blob)