# mostlygeek/llama-swap architecture

> Architecture diagram of the GitHub repository mostlygeek/llama-swap, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-14). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/mostlygeek/llama-swap
- Repository: https://github.com/mostlygeek/llama-swap

## Overview

#### Purpose and Entry Points
llama-swap is a Go local inference gateway and process orchestrator. It exposes OpenAI-, Anthropic-, llama.cpp-, SDAPI-, audio.cpp-, and ComfyUI-compatible HTTP routes, chooses a configured model, starts/stops its backing inference server, then reverse-proxies the request. Production entry point: `llama-swap.go`. Configuration begins from YAML and its generated/validated schema: `config.example.yaml`, `config-schema.json`.

#### Configuration Compilation
The configuration subsystem loads platform-aware YAML, merges profiles, expands macros, resolves model aliases/selectors, command templates, request filters, upstreams, and matrix rules into runtime model definitions. This is the principal declarative boundary between user configuration and orchestration behavior. Paths: `internal/config/load.go`, `internal/config/model_config.go`, `internal/config/matrix.go`.

#### HTTP API and Proxy Boundary
The server owns listener setup, authentication, endpoint grouping, request model extraction/selection, request rewriting, direct upstream access, captures, and API/UI responses. It must preserve streaming behavior while forwarding arbitrary compatible API payloads to selected upstream processes. Paths: `internal/server/server.go`, `internal/server/api.go`, `internal/server/auth.go`.

#### Routing, Scheduling, and Swap Decisions
The router maps requested model IDs (including aliases/groups/peers) to a loading decision. Basic routing swaps incompatible running models; matrix routing evaluates the configuration DSL to decide which concurrent model set can remain loaded. Scheduler coordination serializes or queues load work and protects capacity/resource decisions. Paths: `internal/router/router.go`, `internal/router/matrix_solver.go`, `internal/router/scheduler/scheduler.go`.

#### Model Process Lifecycle
The process layer launches configured commands with dynamically assigned ports and environment, tracks readiness/events, forwards process output, and performs graceful termination or platform-specific tree cleanup. Router loading uses this layer to materialize an upstream server before proxying and unloads models due to replacement, manual APIs, or TTL. Paths: `internal/router/loading.go`, `internal/process/process.go`, `internal/process/process_command.go`.

#### Runtime State, Events, and Observability
In-memory caches/chains coordinate active model state and request flow. Event/log monitoring maintains buffered history plus live streams for proxy and upstream output. Performance monitoring detects hardware and exposes system/GPU Prometheus metrics; activity is persisted through a migration-backed store for UI/API consumption. Paths: `internal/event/event.go`, `internal/logmon/logging.go`, `internal/perf/prometheus.go`, `internal/store/store.go`.

#### UI and Deployment
A Svelte/Vite single-page UI is embedded or served by the Go server, providing models, activity, hardware, performance, logs, settings, and multimodal playgrounds over server APIs and event streams. The system runs as a native Go binary across OSes or in Docker/Podman; unified images bundle inference runtimes such as llama-server, stable-diffusion.cpp, and whisper. Paths: `ui-svelte/src/main.ts`, `ui-svelte/src/App.svelte`, `internal/server/ui.go`, `docker/unified/Dockerfile`.

Main entry points: [`internal/server/server.go`](https://github.com/mostlygeek/llama-swap/blob/main/internal/server/server.go) (HTTP server), [`internal/server/api.go`](https://github.com/mostlygeek/llama-swap/blob/main/internal/server/api.go) (API & request selection), [`ui-svelte/src/App.svelte`](https://github.com/mostlygeek/llama-swap/blob/main/ui-svelte/src/App.svelte) (Svelte UI), [`internal/config/load.go`](https://github.com/mostlygeek/llama-swap/blob/main/internal/config/load.go) (Config loader)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_config["Configuration"]
  node_config_input["YAML &amp; schema<br/>configuration"]
  node_config_loader["Config loader<br/>config compiler<br/>[load.go]"]
  node_model_definitions["Runtime model definitions<br/>compiled config<br/>[model_config.go]"]
  node_matrix_rules["Matrix rules<br/>routing DSL<br/>[matrix.go]"]
end

subgraph group_gateway["Gateway"]
  node_http_server["HTTP server<br/>gateway server<br/>[server.go]"]
  node_api["API &amp; request selection<br/>HTTP API<br/>[api.go]"]
  node_auth["Authentication<br/>HTTP middleware<br/>[auth.go]"]
end

subgraph group_orchestration["Orchestration"]
  node_router["Model router<br/>routing engine<br/>[router.go]"]
  node_matrix_solver["Matrix solver<br/>capacity planner<br/>[matrix_solver.go]"]
  node_scheduler["Load scheduler<br/>work coordinator<br/>[scheduler.go]"]
  node_loading["Loading lifecycle<br/>model loader<br/>[loading.go]"]
  node_process_manager["Inference process manager<br/>process supervisor<br/>[process.go]"]
  node_upstream_servers{{"Managed inference servers<br/>local upstreams<br/>[process_command.go]"}}
end

subgraph group_runtime["Runtime State"]
  node_events["Event stream<br/>in-memory events<br/>[event.go]"]
  node_log_monitor["Log monitor<br/>observability<br/>[logging.go]"]
  node_metrics["Prometheus metrics<br/>hardware monitoring<br/>[prometheus.go]"]
  node_activity_store[("Activity store<br/>persistent store<br/>[store.go]")]
end

subgraph group_clients["UI &amp; Deployment"]
  node_ui["Svelte UI<br/>single-page app<br/>[App.svelte]"]
  node_ui_serving["UI serving<br/>asset server<br/>[ui.go]"]
  node_unified_image{{"Unified container image<br/>deployment image"}}
end

node_entry["Go entry point<br/>bootstrap<br/>[llama-swap.go]"]

node_entry -->|"loads configuration"| node_config_loader
node_entry -->|"starts gateway"| node_http_server
node_config_input -->|"defines input"| node_config_loader
node_config_loader -->|"compiles"| node_model_definitions
node_config_loader -->|"expands rules"| node_matrix_rules
node_http_server -->|"applies"| node_auth
node_http_server -->|"routes requests"| node_api
node_http_server -->|"serves UI"| node_ui_serving
node_ui_serving -->|"delivers SPA"| node_ui
node_ui -->|"uses APIs and streams"| node_api
node_api -->|"selects model"| node_router
node_model_definitions -->|"supplies models"| node_router
node_router -->|"evaluates matrix mode"| node_matrix_solver
node_matrix_rules -->|"defines constraints"| node_matrix_solver
node_router -->|"coordinates load"| node_scheduler
node_scheduler -->|"authorizes work"| node_loading
node_loading -->|"starts or stops"| node_process_manager
node_process_manager -->|"supervises"| node_upstream_servers
node_api -->|"reverse proxies"| node_upstream_servers
node_loading -->|"publishes state"| node_events
node_process_manager -->|"forwards output"| node_log_monitor
node_http_server -->|"exposes metrics"| node_metrics
node_api -->|"records activity"| node_activity_store
node_ui -->|"subscribes live"| node_events
node_ui -->|"views history"| node_activity_store
node_unified_image -.->|"packages runtime"| node_entry

click node_entry "https://github.com/mostlygeek/llama-swap/blob/main/llama-swap.go"
click node_config_input "https://github.com/mostlygeek/llama-swap/blob/main/config.example.yaml"
click node_config_loader "https://github.com/mostlygeek/llama-swap/blob/main/internal/config/load.go"
click node_model_definitions "https://github.com/mostlygeek/llama-swap/blob/main/internal/config/model_config.go"
click node_matrix_rules "https://github.com/mostlygeek/llama-swap/blob/main/internal/config/matrix.go"
click node_http_server "https://github.com/mostlygeek/llama-swap/blob/main/internal/server/server.go"
click node_api "https://github.com/mostlygeek/llama-swap/blob/main/internal/server/api.go"
click node_auth "https://github.com/mostlygeek/llama-swap/blob/main/internal/server/auth.go"
click node_router "https://github.com/mostlygeek/llama-swap/blob/main/internal/router/router.go"
click node_matrix_solver "https://github.com/mostlygeek/llama-swap/blob/main/internal/router/matrix_solver.go"
click node_scheduler "https://github.com/mostlygeek/llama-swap/blob/main/internal/router/scheduler/scheduler.go"
click node_loading "https://github.com/mostlygeek/llama-swap/blob/main/internal/router/loading.go"
click node_process_manager "https://github.com/mostlygeek/llama-swap/blob/main/internal/process/process.go"
click node_upstream_servers "https://github.com/mostlygeek/llama-swap/blob/main/internal/process/process_command.go"
click node_events "https://github.com/mostlygeek/llama-swap/blob/main/internal/event/event.go"
click node_log_monitor "https://github.com/mostlygeek/llama-swap/blob/main/internal/logmon/logging.go"
click node_metrics "https://github.com/mostlygeek/llama-swap/blob/main/internal/perf/prometheus.go"
click node_activity_store "https://github.com/mostlygeek/llama-swap/blob/main/internal/store/store.go"
click node_ui "https://github.com/mostlygeek/llama-swap/blob/main/ui-svelte/src/App.svelte"
click node_ui_serving "https://github.com/mostlygeek/llama-swap/blob/main/internal/server/ui.go"
click node_unified_image "https://github.com/mostlygeek/llama-swap/blob/main/docker/unified/Dockerfile"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config_input,node_config_loader,node_model_definitions,node_matrix_rules toneBlue
class node_http_server,node_api,node_auth toneAmber
class node_router,node_matrix_solver,node_scheduler,node_loading,node_process_manager,node_upstream_servers toneMint
class node_events,node_log_monitor,node_metrics,node_activity_store toneRose
class node_ui,node_ui_serving,node_unified_image toneIndigo
class node_entry toneNeutral
```

## Components

### Configuration

Compiles declarative model and routing definitions.

- [YAML & schema](https://github.com/mostlygeek/llama-swap/blob/main/config.example.yaml): configuration · User-facing configuration contract. · `config.example.yaml`
- [Config loader](https://github.com/mostlygeek/llama-swap/blob/main/internal/config/load.go): config compiler · Loads platform-aware YAML and profiles. · `internal/config/load.go`
- [Runtime model definitions](https://github.com/mostlygeek/llama-swap/blob/main/internal/config/model_config.go): compiled config · Resolves aliases, templates, filters, and upstreams. · `internal/config/model_config.go`
- [Matrix rules](https://github.com/mostlygeek/llama-swap/blob/main/internal/config/matrix.go): routing DSL · Defines permitted concurrent model sets. · `internal/config/matrix.go`

### Gateway

HTTP compatibility, authentication, and proxy boundary.

- [HTTP server](https://github.com/mostlygeek/llama-swap/blob/main/internal/server/server.go): gateway server · Owns listeners, compatible routes, and responses. · `internal/server/server.go`
- [API & request selection](https://github.com/mostlygeek/llama-swap/blob/main/internal/server/api.go): HTTP API · Extracts requested models and rewrites requests. · `internal/server/api.go`
- [Authentication](https://github.com/mostlygeek/llama-swap/blob/main/internal/server/auth.go): HTTP middleware · `internal/server/auth.go`

### Orchestration

Model selection, capacity decisions, and process lifecycle.

- [Model router](https://github.com/mostlygeek/llama-swap/blob/main/internal/router/router.go): routing engine · Maps model IDs to loading decisions. · `internal/router/router.go`
- [Matrix solver](https://github.com/mostlygeek/llama-swap/blob/main/internal/router/matrix_solver.go): capacity planner · Evaluates concurrent loading rules. · `internal/router/matrix_solver.go`
- [Load scheduler](https://github.com/mostlygeek/llama-swap/blob/main/internal/router/scheduler/scheduler.go): work coordinator · Serializes load work and protects capacity. · `internal/router/scheduler/scheduler.go`
- [Loading lifecycle](https://github.com/mostlygeek/llama-swap/blob/main/internal/router/loading.go): model loader · Materializes and unloads model upstreams. · `internal/router/loading.go`
- [Inference process manager](https://github.com/mostlygeek/llama-swap/blob/main/internal/process/process.go): process supervisor · Launches, monitors, and terminates configured servers. · `internal/process/process.go`
- [Managed inference servers](https://github.com/mostlygeek/llama-swap/blob/main/internal/process/process_command.go): local upstreams · llama.cpp, SDAPI, audio.cpp, and ComfyUI-compatible processes. · `internal/process/process_command.go`

### Runtime State

Events, logs, metrics, and persisted activity.

- [Event stream](https://github.com/mostlygeek/llama-swap/blob/main/internal/event/event.go): in-memory events · Coordinates active state and live updates. · `internal/event/event.go`
- [Log monitor](https://github.com/mostlygeek/llama-swap/blob/main/internal/logmon/logging.go): observability · Buffers and streams proxy and upstream output. · `internal/logmon/logging.go`
- [Prometheus metrics](https://github.com/mostlygeek/llama-swap/blob/main/internal/perf/prometheus.go): hardware monitoring · Exports system and GPU performance metrics. · `internal/perf/prometheus.go`
- [Activity store](https://github.com/mostlygeek/llama-swap/blob/main/internal/store/store.go): persistent store · Migration-backed activity data for APIs and UI. · `internal/store/store.go`

### UI & Deployment

Operator interface and packaged runtime environment.

- [Svelte UI](https://github.com/mostlygeek/llama-swap/blob/main/ui-svelte/src/App.svelte): single-page app · Models, activity, hardware, logs, settings, and playgrounds. · `ui-svelte/src/App.svelte`
- [UI serving](https://github.com/mostlygeek/llama-swap/blob/main/internal/server/ui.go): asset server · Embeds or serves the SPA from the gateway. · `internal/server/ui.go`
- [Unified container image](https://github.com/mostlygeek/llama-swap/blob/main/docker/unified/Dockerfile): deployment image · Bundles the gateway with inference runtimes. · `docker/unified/Dockerfile`

### Other components

- [Go entry point](https://github.com/mostlygeek/llama-swap/blob/main/llama-swap.go): bootstrap · Starts the local inference gateway. · `llama-swap.go`

## Connections

- Go entry point → Config loader: loads configuration
- Go entry point → HTTP server: starts gateway
- YAML & schema → Config loader: defines input
- Config loader → Runtime model definitions: compiles
- Config loader → Matrix rules: expands rules
- HTTP server → Authentication: applies
- HTTP server → API & request selection: routes requests
- HTTP server → UI serving: serves UI
- UI serving → Svelte UI: delivers SPA
- Svelte UI → API & request selection: uses APIs and streams
- API & request selection → Model router: selects model
- Runtime model definitions → Model router: supplies models
- Model router → Matrix solver: evaluates matrix mode
- Matrix rules → Matrix solver: defines constraints
- Model router → Load scheduler: coordinates load
- Load scheduler → Loading lifecycle: authorizes work
- Loading lifecycle → Inference process manager: starts or stops
- Inference process manager → Managed inference servers: supervises
- API & request selection → Managed inference servers: reverse proxies · Preserves compatible and streaming payloads.
- Loading lifecycle → Event stream: publishes state
- Inference process manager → Log monitor: forwards output
- HTTP server → Prometheus metrics: exposes metrics
- API & request selection → Activity store: records activity
- Svelte UI → Event stream: subscribes live
- Svelte UI → Activity store: views history
- Unified container image → Go entry point: packages runtime