# bnice00/k3s-hermes-llama architecture

> Architecture diagram of the GitHub repository bnice00/k3s-hermes-llama, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bnice00/k3s-hermes-llama
- Repository: https://github.com/bnice00/k3s-hermes-llama

## Overview

This repository describes a single-user K3s inference stack: a Streamlit chat client sends prompts to a Hermes agent, which uses llama.cpp as its inference backend. The README also documents an earlier vLLM/ROCm setup and GPU scheduling/monitoring components; manifests show additional services, but their runtime wiring is not fully established by the sampled source. Paths for those unsampled deployments point to their exact manifest files.

Main entry points: [`manifests/llama.yaml`](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/llama.yaml) (llama.cpp Vulkan), [`manifests/00-namespace.yaml`](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/00-namespace.yaml) (K3s cluster), [`streamlit/agent.py`](https://github.com/bnice00/k3s-hermes-llama/blob/main/streamlit/agent.py) (Streamlit chat), [`manifests/hermes-agent.yaml`](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/hermes-agent.yaml) (Hermes agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["User interaction"]
  node_chat["Streamlit chat<br/>[agent.py]"]
end

subgraph group_agent["Agent layer"]
  node_hermes["Hermes agent<br/>[hermes-agent.yaml]"]
  node_hermesconfig["Hermes config<br/>[hermes-config.yaml]"]
end

subgraph group_inference["Inference services"]
  node_llama["llama.cpp Vulkan<br/>[llama.yaml]"]
  node_llamaservice["Inference endpoint<br/>[llama.yaml]"]
  node_vllm["vLLM deployment<br/>[vllm.yaml]"]
  node_vllmservice["vLLM service<br/>[vllm-service.yaml]"]
  node_whisper["Whisper service<br/>[whisper.yaml]"]
end

subgraph group_platform["Cluster platform"]
  node_modelstorage[("Model storage<br/>[pvc-hermes.yaml]")]
  node_gpuplugin["GPU device plugin"]
  node_gpulabeller["GPU device labeller"]
  node_k3s["K3s cluster<br/>[00-namespace.yaml]"]
  node_obsidian["Obsidian service<br/>[obsidian.yaml]"]
  node_deb["Debian service<br/>[deb-service.yaml]"]
  node_certman["Certificate manager"]
  node_prometheus["Prometheus monitoring"]
end

node_user(("User"))
node_clusteroperator(("Cluster operator"))

node_user -->|"submits prompt"| node_chat
node_chat -->|"posts chat request"| node_hermes
node_hermes -->|"requests inference"| node_llama
node_llama -->|"serves endpoint"| node_llamaservice
node_llama -->|"uses model storage"| node_modelstorage
node_gpuplugin -->|"advertises GPU"| node_k3s
node_k3s -->|"schedules GPU pod"| node_llama
node_gpulabeller -->|"labels GPU devices"| node_k3s
node_clusteroperator -->|"operates cluster"| node_k3s

click node_chat "https://github.com/bnice00/k3s-hermes-llama/blob/main/streamlit/agent.py"
click node_hermes "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/hermes-agent.yaml"
click node_hermesconfig "https://github.com/bnice00/k3s-hermes-llama/blob/main/hermes-config/hermes-config.yaml"
click node_llama "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/llama.yaml"
click node_llamaservice "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/llama.yaml"
click node_modelstorage "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/pvc-hermes.yaml"
click node_gpuplugin "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/gpu-device-plugin.yaml"
click node_gpulabeller "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/gpu-device-labeller.yaml"
click node_k3s "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/00-namespace.yaml"
click node_vllm "https://github.com/bnice00/k3s-hermes-llama/blob/main/vllm-stack/vllm.yaml"
click node_vllmservice "https://github.com/bnice00/k3s-hermes-llama/blob/main/vllm-stack/vllm-service.yaml"
click node_whisper "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/whisper.yaml"
click node_obsidian "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/obsidian.yaml"
click node_deb "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/deb-service.yaml"
click node_certman "https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/certman-values.yaml"
click node_prometheus "https://github.com/bnice00/k3s-hermes-llama/blob/main/monitoring/prometheus-values.yaml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_chat,node_user toneBlue
class node_hermes,node_hermesconfig toneAmber
class node_llama,node_llamaservice,node_vllm,node_vllmservice,node_whisper toneMint
class node_modelstorage,node_gpuplugin,node_gpulabeller,node_k3s,node_obsidian,node_deb,node_certman,node_prometheus toneRose
class node_clusteroperator toneIndigo
```

## Components

### User interaction

- [Streamlit chat](https://github.com/bnice00/k3s-hermes-llama/blob/main/streamlit/agent.py): `streamlit/agent.py`

### Agent layer

- [Hermes agent](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/hermes-agent.yaml): `manifests/hermes-agent.yaml`
- [Hermes config](https://github.com/bnice00/k3s-hermes-llama/blob/main/hermes-config/hermes-config.yaml): `hermes-config/hermes-config.yaml`

### Inference services

- [llama.cpp Vulkan](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/llama.yaml): `manifests/llama.yaml`
- [Inference endpoint](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/llama.yaml): `manifests/llama.yaml`
- [vLLM deployment](https://github.com/bnice00/k3s-hermes-llama/blob/main/vllm-stack/vllm.yaml): `vllm-stack/vllm.yaml`
- [vLLM service](https://github.com/bnice00/k3s-hermes-llama/blob/main/vllm-stack/vllm-service.yaml): `vllm-stack/vllm-service.yaml`
- [Whisper service](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/whisper.yaml): `manifests/whisper.yaml`

### Cluster platform

- [Model storage](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/pvc-hermes.yaml): `manifests/pvc-hermes.yaml`
- [GPU device plugin](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/gpu-device-plugin.yaml): `manifests/gpu-device-plugin.yaml`
- [GPU device labeller](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/gpu-device-labeller.yaml): `manifests/gpu-device-labeller.yaml`
- [K3s cluster](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/00-namespace.yaml): `manifests/00-namespace.yaml`
- [Obsidian service](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/obsidian.yaml): `manifests/obsidian.yaml`
- [Debian service](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/deb-service.yaml): `manifests/deb-service.yaml`
- [Certificate manager](https://github.com/bnice00/k3s-hermes-llama/blob/main/manifests/certman-values.yaml): `manifests/certman-values.yaml`
- [Prometheus monitoring](https://github.com/bnice00/k3s-hermes-llama/blob/main/monitoring/prometheus-values.yaml): `monitoring/prometheus-values.yaml`

### Other components

- **User**
- **Cluster operator**

## Connections

- User → Streamlit chat: submits prompt
- Streamlit chat → Hermes agent: posts chat request
- Hermes agent → llama.cpp Vulkan: requests inference
- llama.cpp Vulkan → Inference endpoint: serves endpoint
- llama.cpp Vulkan → Model storage: uses model storage
- GPU device plugin → K3s cluster: advertises GPU
- K3s cluster → llama.cpp Vulkan: schedules GPU pod
- GPU device labeller → K3s cluster: labels GPU devices
- Cluster operator → K3s cluster: operates cluster