# offbyonebit/arc-llama architecture

> Architecture diagram of the GitHub repository offbyonebit/arc-llama, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/offbyonebit/arc-llama
- Repository: https://github.com/offbyonebit/arc-llama

## Overview

arc-llama is an Intel Arc-focused llama.cpp runtime and model-serving application. Its main workflow detects and configures a GPU, prepares or registers a model, estimates a safe launch recipe, then routes API requests to llama-server. It also provides web and terminal interfaces, plugins, tuning and recipe sharing, plus a coding agent with tools and rollback checkpoints. README statements and SOURCE INDEX references support the major connections shown; several subsystem internals were not sampled, so uncertain links are omitted.

Main entry points: [`src/arc_llama/cli.py`](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/cli.py) (CLI commands), [`src/arc_llama/server.py`](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/server.py) (HTTP API server), [`src/arc_llama/router.py`](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/router.py) (Model router), [`src/arc_llama/launcher.py`](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/launcher.py) (Launch planning)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_setup["Setup and models"]
  node_cli["CLI commands<br/>[cli.py]"]
  node_gpu["GPU discovery<br/>[detect.py]"]
  node_arch["GPU profiles<br/>[arch.py]"]
  node_config[("Config registry<br/>[config.py]")]
  node_runtime["Runtime installer<br/>[runtime.py]"]
  node_models["Model registry<br/>[models.py]"]
  node_gguf["GGUF metadata<br/>[gguf_meta.py]"]
  node_recipes["Launch recipes<br/>[recipes.py]"]
end

subgraph group_serving["Inference serving"]
  node_launcher["Launch planning<br/>[launcher.py]"]
  node_router["Model router<br/>[router.py]"]
  node_server["HTTP API server<br/>[server.py]"]
  node_llama["llama-server"]
  node_plugins["Plugin integrations<br/>[plugins.py]"]
end

subgraph group_experience["User interfaces"]
  node_web["Web UI<br/>[app.js]"]
  node_chatui["Chat UI<br/>[chat.js]"]
  node_tui["Terminal UI<br/>[tui.py]"]
end

subgraph group_optimization["Tuning"]
  node_autotune["Background tuning<br/>[autotune.py]"]
  node_tune["Recipe tuner<br/>[tune.py]"]
  node_workload["Workload profiling<br/>[workload.py]"]
  node_share["Shared recipes<br/>[recipe_share.py]"]
end

subgraph group_agent["Coding agent"]
  node_agent["Coding agent<br/>[loop.py]"]
  node_tools["Agent tools<br/>[tools.py]"]
  node_checkpoints[("Rollback checkpoints<br/>[checkpoints.py]")]
  node_mcp["MCP integrations<br/>[mcp_client.py]"]
end

node_user(("Operator"))
node_client(("API client"))

node_user -->|"runs commands"| node_cli
node_cli -->|"detects GPUs"| node_gpu
node_gpu -->|"identifies generation"| node_arch
node_cli -->|"writes config"| node_config
node_cli -->|"installs runtime"| node_runtime
node_cli -->|"registers models"| node_models
node_models -->|"reads metadata"| node_gguf
node_models -->|"creates recipes"| node_recipes
node_cli -->|"checks launch plan"| node_launcher
node_launcher -->|"estimates fit"| node_gguf
node_launcher -->|"builds launch settings"| node_recipes
node_cli -->|"starts service"| node_router
node_server -->|"routes model requests"| node_router
node_router -->|"plans model launch"| node_launcher
node_router -->|"manages subprocesses"| node_llama
node_client -->|"sends API requests"| node_server
node_server -->|"exposes plugin status"| node_plugins
node_web -->|"uses admin API"| node_server
node_chatui -->|"uses chat API"| node_server
node_tui -->|"controls models"| node_server
node_autotune -->|"schedules sweeps"| node_tune
node_tune -->|"profiles workload"| node_workload
node_tune -->|"benchmarks models"| node_router
node_cli -->|"manages shared recipes"| node_share
node_server -->|"provides agent endpoints"| node_agent
node_agent -->|"executes tools"| node_tools
node_tools -->|"creates checkpoints"| node_checkpoints
node_agent -->|"loads configured MCP"| node_mcp

click node_cli "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/cli.py"
click node_gpu "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/detect.py"
click node_arch "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/arch.py"
click node_config "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/config.py"
click node_runtime "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/runtime.py"
click node_models "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/models.py"
click node_gguf "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/gguf_meta.py"
click node_recipes "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/recipes.py"
click node_launcher "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/launcher.py"
click node_router "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/router.py"
click node_server "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/server.py"
click node_plugins "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/plugins.py"
click node_web "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/static/app.js"
click node_chatui "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/static/chat.js"
click node_tui "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/tui.py"
click node_autotune "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/autotune.py"
click node_tune "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/tune.py"
click node_workload "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/workload.py"
click node_share "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/recipe_share.py"
click node_agent "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/loop.py"
click node_tools "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/tools.py"
click node_checkpoints "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/checkpoints.py"
click node_mcp "https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/mcp_client.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_gpu,node_arch,node_config,node_runtime,node_models,node_gguf,node_recipes,node_client toneBlue
class node_launcher,node_router,node_server,node_llama,node_plugins toneAmber
class node_web,node_chatui,node_tui toneMint
class node_autotune,node_tune,node_workload,node_share toneRose
class node_agent,node_tools,node_checkpoints,node_mcp,node_user toneIndigo
```

## Components

### Setup and models

- [CLI commands](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/cli.py): `src/arc_llama/cli.py`
- [GPU discovery](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/detect.py): `src/arc_llama/detect.py`
- [GPU profiles](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/arch.py): `src/arc_llama/arch.py`
- [Config registry](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/config.py): `src/arc_llama/config.py`
- [Runtime installer](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/runtime.py): `src/arc_llama/runtime.py`
- [Model registry](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/models.py): `src/arc_llama/models.py`
- [GGUF metadata](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/gguf_meta.py): `src/arc_llama/gguf_meta.py`
- [Launch recipes](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/recipes.py): `src/arc_llama/recipes.py`

### Inference serving

- [Launch planning](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/launcher.py): `src/arc_llama/launcher.py`
- [Model router](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/router.py): `src/arc_llama/router.py`
- [HTTP API server](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/server.py): `src/arc_llama/server.py`
- **llama-server**
- [Plugin integrations](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/plugins.py): `src/arc_llama/plugins.py`

### User interfaces

- [Web UI](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/static/app.js): `src/arc_llama/static/app.js`
- [Chat UI](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/static/chat.js): `src/arc_llama/static/chat.js`
- [Terminal UI](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/tui.py): `src/arc_llama/tui.py`

### Tuning

- [Background tuning](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/autotune.py): `src/arc_llama/autotune.py`
- [Recipe tuner](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/tune.py): `src/arc_llama/tune.py`
- [Workload profiling](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/workload.py): `src/arc_llama/workload.py`
- [Shared recipes](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/recipe_share.py): `src/arc_llama/recipe_share.py`

### Coding agent

- [Coding agent](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/loop.py): `src/arc_llama/agent/loop.py`
- [Agent tools](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/tools.py): `src/arc_llama/agent/tools.py`
- [Rollback checkpoints](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/checkpoints.py): `src/arc_llama/agent/checkpoints.py`
- [MCP integrations](https://github.com/offbyonebit/arc-llama/blob/main/src/arc_llama/agent/mcp_client.py): `src/arc_llama/agent/mcp_client.py`

### Other components

- **Operator**
- **API client**

## Connections

- Operator → CLI commands: runs commands
- CLI commands → GPU discovery: detects GPUs (evidence: `src/arc_llama/cli.py`)
- GPU discovery → GPU profiles: identifies generation (evidence: `src/arc_llama/detect.py`)
- CLI commands → Config registry: writes config (evidence: `src/arc_llama/cli.py`)
- CLI commands → Runtime installer: installs runtime (evidence: `src/arc_llama/cli.py`)
- CLI commands → Model registry: registers models (evidence: `src/arc_llama/cli.py`)
- Model registry → GGUF metadata: reads metadata (evidence: `src/arc_llama/models.py`)
- Model registry → Launch recipes: creates recipes (evidence: `src/arc_llama/models.py`)
- CLI commands → Launch planning: checks launch plan
- Launch planning → GGUF metadata: estimates fit (evidence: `src/arc_llama/launcher.py`)
- Launch planning → Launch recipes: builds launch settings (evidence: `src/arc_llama/launcher.py`)
- CLI commands → Model router: starts service (evidence: `src/arc_llama/cli.py`)
- HTTP API server → Model router: routes model requests (evidence: `src/arc_llama/server.py`)
- Model router → Launch planning: plans model launch (evidence: `src/arc_llama/router.py`)
- Model router → llama-server: manages subprocesses (evidence: `src/arc_llama/router.py`)
- API client → HTTP API server: sends API requests
- HTTP API server → Plugin integrations: exposes plugin status (evidence: `src/arc_llama/server.py`)
- Web UI → HTTP API server: uses admin API
- Chat UI → HTTP API server: uses chat API
- Terminal UI → HTTP API server: controls models
- Background tuning → Recipe tuner: schedules sweeps (evidence: `src/arc_llama/autotune.py`)
- Recipe tuner → Workload profiling: profiles workload (evidence: `src/arc_llama/tune.py`)
- Recipe tuner → Model router: benchmarks models (evidence: `src/arc_llama/tune.py`)
- CLI commands → Shared recipes: manages shared recipes (evidence: `src/arc_llama/cli.py`)
- HTTP API server → Coding agent: provides agent endpoints (evidence: `src/arc_llama/server.py`)
- Coding agent → Agent tools: executes tools (evidence: `src/arc_llama/agent/loop.py`)
- Agent tools → Rollback checkpoints: creates checkpoints (evidence: `src/arc_llama/agent/tools.py`)
- Coding agent → MCP integrations: loads configured MCP
