How offbyonebit/arc-llama works
arc-llama is an Intel Arc-focused llama.cpp runtime and model-serving application. Its main workflow detects and configures a GPU, prepares or registers a model, estimates a safe launch recipe, then routes API requests to llama-server. It also provides web and terminal interfaces, plugins, tuning and recipe sharing, plus a coding agent with tools and rollback checkpoints. README statements and SOURCE INDEX references support the major connections shown; several subsystem internals were not sampled, so uncertain links are omitted.
Main entry points: cli.py CLI commands · server.py HTTP API server · router.py Model router · launcher.py Launch planning
Components 26
Setup and models
- CLI commands
src/arc_llama/cli.py - GPU discovery
src/arc_llama/detect.py - GPU profiles
src/arc_llama/arch.py - Config registry
src/arc_llama/config.py - Runtime installer
src/arc_llama/runtime.py - Model registry
src/arc_llama/models.py - GGUF metadata
src/arc_llama/gguf_meta.py - Launch recipes
src/arc_llama/recipes.py
Inference serving
- Launch planning
src/arc_llama/launcher.py - Model router
src/arc_llama/router.py - HTTP API server
src/arc_llama/server.py - llama-server
- Plugin integrations
src/arc_llama/plugins.py
User interfaces
- Web UI
src/arc_llama/static/app.js - Chat UI
src/arc_llama/static/chat.js - Terminal UI
src/arc_llama/tui.py
Tuning
- Background tuning
src/arc_llama/autotune.py - Recipe tuner
src/arc_llama/tune.py - Workload profiling
src/arc_llama/workload.py - Shared recipes
src/arc_llama/recipe_share.py
Coding agent
- Coding agent
src/arc_llama/agent/loop.py - Agent tools
src/arc_llama/agent/tools.py - Rollback checkpoints
src/arc_llama/agent/checkpoints.py - MCP integrations
src/arc_llama/agent/mcp_client.py
Other components
- Operator
- API client
Connections 28
- Operator → CLI commands: runs commands
- CLI commands → GPU discovery: detects GPUs
src/arc_llama/cli.py - GPU discovery → GPU profiles: identifies generation
src/arc_llama/detect.py - CLI commands → Config registry: writes config
src/arc_llama/cli.py - CLI commands → Runtime installer: installs runtime
src/arc_llama/cli.py - CLI commands → Model registry: registers models
src/arc_llama/cli.py - Model registry → GGUF metadata: reads metadata
src/arc_llama/models.py - Model registry → Launch recipes: creates recipes
src/arc_llama/models.py - CLI commands → Launch planning: checks launch plan
- Launch planning → GGUF metadata: estimates fit
src/arc_llama/launcher.py - Launch planning → Launch recipes: builds launch settings
src/arc_llama/launcher.py - CLI commands → Model router: starts service
src/arc_llama/cli.py - HTTP API server → Model router: routes model requests
src/arc_llama/server.py - Model router → Launch planning: plans model launch
src/arc_llama/router.py - Model router → llama-server: manages subprocesses
src/arc_llama/router.py - API client → HTTP API server: sends API requests
- HTTP API server → Plugin integrations: exposes plugin status
src/arc_llama/server.py - Web UI → HTTP API server: uses admin API
- Chat UI → HTTP API server: uses chat API
- Terminal UI → HTTP API server: controls models
- Background tuning → Recipe tuner: schedules sweeps
src/arc_llama/autotune.py - Recipe tuner → Workload profiling: profiles workload
src/arc_llama/tune.py - Recipe tuner → Model router: benchmarks models
src/arc_llama/tune.py - CLI commands → Shared recipes: manages shared recipes
src/arc_llama/cli.py - HTTP API server → Coding agent: provides agent endpoints
src/arc_llama/server.py - Coding agent → Agent tools: executes tools
src/arc_llama/agent/loop.py - Agent tools → Rollback checkpoints: creates checkpoints
src/arc_llama/agent/tools.py - Coding agent → MCP integrations: loads configured MCP