# andreipath26/kiln architecture

> Architecture diagram of the GitHub repository andreipath26/kiln, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/andreipath26/kiln
- Repository: https://github.com/andreipath26/kiln

## Overview

KILN is a local inference runtime and model manager: users discover or download a GGUF, optionally preflight it, then run or activate it for chat/generation through the CLI or loopback UI. Its experimental Linux MoE stream path is a separate opt-in route using an isolated worker and selected expert reads. The graph preserves documented model-library, planning, streaming, and engineering-evaluation capabilities. Edges are limited to relationships visible in excerpts, the source index, or README; unsampled subsystems have no asserted internal call flow.

Main entry points: [`crates/kiln-cli/src/main.rs`](https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/main.rs) (CLI Commands), [`crates/kiln-models/src/library.rs`](https://github.com/andreipath26/kiln/blob/main/crates/kiln-models/src/library.rs) (Model Discovery), [`crates/kiln-runtime/src/backends/moe_streamed.rs`](https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/backends/moe_streamed.rs) (Streamed MoE Backend), [`crates/kiln-runtime/src/runtime_dispatch.rs`](https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/runtime_dispatch.rs) (Runtime Dispatcher)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_entry["User Interfaces"]
  node_cli["CLI Commands<br/>[main.rs]"]
  node_api["HTTP Routes<br/>[routes.rs]"]
  node_server["Loopback Server<br/>[server.rs]"]
  node_ui["Web UI<br/>[ui.js]"]
end

subgraph group_models["Model Management"]
  node_model_library["Model Discovery<br/>[library.rs]"]
  node_gguf["GGUF Parser<br/>[gguf.rs]"]
end

subgraph group_inference["Inference Runtime"]
  node_runtime_dispatch["Runtime Dispatcher"]
  node_llama_backend["Native Backend<br/>[llama_cpp.rs]"]
  node_chat["Chat Sampling<br/>[chat.rs]"]
  node_api_runtime["API Session<br/>[runtime.rs]"]
end

subgraph group_streaming["Expert Streaming"]
  node_stream_cli["Stream Launcher"]
  node_stream_worker["Isolated Worker<br/>[stream_worker.rs]"]
  node_moe_backend["Streamed MoE Backend<br/>[moe_streamed.rs]"]
  node_expert_io["Expert I/O"]
  node_expert_memory["Expert Memory"]
  node_expert_index["Expert Tensor Mapping"]
end

subgraph group_planning["Planning and Evaluation"]
  node_plan["Feasibility Plan<br/>[model_plan.rs]"]
  node_eval["Quality Evaluation<br/>[quality_eval.rs]"]
  node_bench["Runtime Benchmarks"]
  node_spec_bench["Speculative Benchmarks"]
  node_monitor["System Monitor<br/>[monitor.rs]"]
  node_selector["Plan Selector<br/>[selector.rs]"]
end

node_user(("Local User"))
node_ollama(("Hugging Face"))

node_user -->|"runs commands"| node_cli
node_user -->|"uses locally"| node_ui
node_ui -->|"requests"| node_server
node_server -->|"routes to"| node_api
node_cli -->|"resolves models"| node_model_library
node_model_library -->|"inspects GGUF"| node_gguf
node_ollama -->|"discovers manifests"| node_model_library
node_ollama -->|"searches and downloads"| node_model_library
node_cli -->|"runs preflight"| node_plan
node_plan -->|"resolves model"| node_model_library
node_cli -->|"runs evaluation"| node_eval
node_cli -->|"runs benchmarks"| node_bench
node_cli -->|"runs benchmarks"| node_spec_bench
node_api -->|"uses session"| node_api_runtime
node_api -->|"samples responses"| node_chat
node_api_runtime -->|"dispatches inference"| node_runtime_dispatch
node_runtime_dispatch -.->|"dispatches operations"| node_llama_backend
node_cli -->|"dispatches inference"| node_runtime_dispatch
node_runtime_dispatch -.->|"uses system profile"| node_monitor
node_selector -->|"queries monitor"| node_monitor
node_api_runtime -->|"starts worker"| node_stream_worker
node_stream_worker -->|"launches stream command"| node_stream_cli
node_stream_cli -->|"resolves model"| node_model_library
node_stream_cli -.->|"starts inference child"| node_moe_backend
node_moe_backend -.->|"reads expert slabs"| node_expert_io
node_moe_backend -.->|"maps expert tensors"| node_expert_index
node_moe_backend -.->|"caches experts"| node_expert_memory

click node_cli "https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/main.rs"
click node_api "https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/routes.rs"
click node_server "https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/server.rs"
click node_ui "https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/static/ui.js"
click node_model_library "https://github.com/andreipath26/kiln/blob/main/crates/kiln-models/src/library.rs"
click node_gguf "https://github.com/andreipath26/kiln/blob/main/crates/kiln-models/src/gguf.rs"
click node_plan "https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/model_plan.rs"
click node_eval "https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/quality_eval.rs"
click node_bench "https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/phase35_kv_bench.rs"
click node_spec_bench "https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/phase35_spec_bench.rs"
click node_monitor "https://github.com/andreipath26/kiln/blob/main/crates/kiln-core/src/monitor.rs"
click node_selector "https://github.com/andreipath26/kiln/blob/main/crates/kiln-core/src/selector.rs"
click node_runtime_dispatch "https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/runtime_dispatch.rs"
click node_llama_backend "https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/backends/llama_cpp.rs"
click node_chat "https://github.com/andreipath26/kiln/blob/main/crates/kiln-core/src/chat.rs"
click node_api_runtime "https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/runtime.rs"
click node_stream_cli "https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/experimental_stream.rs"
click node_stream_worker "https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/stream_worker.rs"
click node_moe_backend "https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/backends/moe_streamed.rs"
click node_expert_io "https://github.com/andreipath26/kiln/tree/main/crates/kiln-io/src"
click node_expert_memory "https://github.com/andreipath26/kiln/tree/main/crates/kiln-mem/src"
click node_expert_index "https://github.com/andreipath26/kiln/blob/main/crates/kiln-models/src/expert_tensor_map.rs"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_api,node_server,node_ui,node_user toneBlue
class node_model_library,node_gguf toneAmber
class node_runtime_dispatch,node_llama_backend,node_chat,node_api_runtime toneMint
class node_stream_cli,node_stream_worker,node_moe_backend,node_expert_io,node_expert_memory,node_expert_index toneRose
class node_plan,node_eval,node_bench,node_spec_bench,node_monitor,node_selector,node_ollama toneIndigo
```

## Components

### User Interfaces

- [CLI Commands](https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/main.rs): `crates/kiln-cli/src/main.rs`
- [HTTP Routes](https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/routes.rs): `crates/kiln-api/src/routes.rs`
- [Loopback Server](https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/server.rs): `crates/kiln-api/src/server.rs`
- [Web UI](https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/static/ui.js): `crates/kiln-api/static/ui.js`

### Model Management

- [Model Discovery](https://github.com/andreipath26/kiln/blob/main/crates/kiln-models/src/library.rs): `crates/kiln-models/src/library.rs`
- [GGUF Parser](https://github.com/andreipath26/kiln/blob/main/crates/kiln-models/src/gguf.rs): `crates/kiln-models/src/gguf.rs`

### Inference Runtime

- [Runtime Dispatcher](https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/runtime_dispatch.rs): `crates/kiln-runtime/src/runtime_dispatch.rs`
- [Native Backend](https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/backends/llama_cpp.rs): `crates/kiln-runtime/src/backends/llama_cpp.rs`
- [Chat Sampling](https://github.com/andreipath26/kiln/blob/main/crates/kiln-core/src/chat.rs): `crates/kiln-core/src/chat.rs`
- [API Session](https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/runtime.rs): `crates/kiln-api/src/runtime.rs`

### Expert Streaming

- [Stream Launcher](https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/experimental_stream.rs): `crates/kiln-cli/src/experimental_stream.rs`
- [Isolated Worker](https://github.com/andreipath26/kiln/blob/main/crates/kiln-api/src/stream_worker.rs): `crates/kiln-api/src/stream_worker.rs`
- [Streamed MoE Backend](https://github.com/andreipath26/kiln/blob/main/crates/kiln-runtime/src/backends/moe_streamed.rs): `crates/kiln-runtime/src/backends/moe_streamed.rs`
- [Expert I/O](https://github.com/andreipath26/kiln/tree/main/crates/kiln-io/src): `crates/kiln-io/src`
- [Expert Memory](https://github.com/andreipath26/kiln/tree/main/crates/kiln-mem/src): `crates/kiln-mem/src`
- [Expert Tensor Mapping](https://github.com/andreipath26/kiln/blob/main/crates/kiln-models/src/expert_tensor_map.rs): `crates/kiln-models/src/expert_tensor_map.rs`

### Planning and Evaluation

- [Feasibility Plan](https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/model_plan.rs): `crates/kiln-cli/src/model_plan.rs`
- [Quality Evaluation](https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/quality_eval.rs): `crates/kiln-cli/src/quality_eval.rs`
- [Runtime Benchmarks](https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/phase35_kv_bench.rs): `crates/kiln-cli/src/phase35_kv_bench.rs`
- [Speculative Benchmarks](https://github.com/andreipath26/kiln/blob/main/crates/kiln-cli/src/phase35_spec_bench.rs): `crates/kiln-cli/src/phase35_spec_bench.rs`
- [System Monitor](https://github.com/andreipath26/kiln/blob/main/crates/kiln-core/src/monitor.rs): `crates/kiln-core/src/monitor.rs`
- [Plan Selector](https://github.com/andreipath26/kiln/blob/main/crates/kiln-core/src/selector.rs): `crates/kiln-core/src/selector.rs`

### Other components

- **Local User**
- **Hugging Face**

## Connections

- Local User → CLI Commands: runs commands (evidence: `README.md`)
- Local User → Web UI: uses locally (evidence: `README.md`)
- Web UI → Loopback Server: requests (evidence: `crates/kiln-api/src/server.rs`)
- Loopback Server → HTTP Routes: routes to (evidence: `crates/kiln-api/src/server.rs`)
- CLI Commands → Model Discovery: resolves models
- Model Discovery → GGUF Parser: inspects GGUF (evidence: `crates/kiln-models/src/library.rs`)
- Hugging Face → Model Discovery: discovers manifests (evidence: `README.md`)
- Hugging Face → Model Discovery: searches and downloads (evidence: `README.md`)
- CLI Commands → Feasibility Plan: runs preflight (evidence: `crates/kiln-cli/src/main.rs`)
- Feasibility Plan → Model Discovery: resolves model (evidence: `crates/kiln-cli/src/model_plan.rs`)
- CLI Commands → Quality Evaluation: runs evaluation (evidence: `crates/kiln-cli/src/main.rs`)
- CLI Commands → Runtime Benchmarks: runs benchmarks (evidence: `crates/kiln-cli/src/main.rs`)
- CLI Commands → Speculative Benchmarks: runs benchmarks (evidence: `crates/kiln-cli/src/main.rs`)
- HTTP Routes → API Session: uses session (evidence: `crates/kiln-api/src/routes.rs`)
- HTTP Routes → Chat Sampling: samples responses (evidence: `crates/kiln-api/src/routes.rs`)
- API Session → Runtime Dispatcher: dispatches inference (evidence: `crates/kiln-api/src/runtime.rs`)
- Runtime Dispatcher → Native Backend: dispatches operations
- CLI Commands → Runtime Dispatcher: dispatches inference (evidence: `crates/kiln-cli/src/main.rs`)
- Runtime Dispatcher → System Monitor: uses system profile
- Plan Selector → System Monitor: queries monitor (evidence: `crates/kiln-core/src/pipeline.rs`)
- API Session → Isolated Worker: starts worker (evidence: `crates/kiln-api/src/runtime.rs`)
- Isolated Worker → Stream Launcher: launches stream command (evidence: `README.md`)
- Stream Launcher → Model Discovery: resolves model (evidence: `crates/kiln-cli/src/experimental_stream.rs`)
- Stream Launcher → Streamed MoE Backend: starts inference child
- Streamed MoE Backend → Expert I/O: reads expert slabs
- Streamed MoE Backend → Expert Tensor Mapping: maps expert tensors
- Streamed MoE Backend → Expert Memory: caches experts