# ali-ncibi/llm-stack architecture

> Architecture diagram of the GitHub repository ali-ncibi/llm-stack, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ali-ncibi/llm-stack
- Repository: https://github.com/ali-ncibi/llm-stack
- Video tour (about a minute): https://gitdiagram.com/ali-ncibi/llm-stack/video

## Overview

This repository describes a self-hosted Kubernetes LLM chat stack. The main user workflow is browser → public Open WebUI → vLLM → generated response; Open WebUI also uses PostgreSQL/pgvector and Redis. Separate operational branches cover deployment, queue-based and CPU-based scaling, disruption protection, and monitoring. The README documents these connections and capabilities; the manifest itself was not sampled, so the graph does not claim details beyond that documentation.

Main entry points: [`manifests/stack.yaml`](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml) (Open WebUI), [`stack.sh`](https://github.com/ali-ncibi/llm-stack/blob/main/stack.sh) (Stack setup)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_operations["Deployment"]
  node_setup["Stack setup<br/>[stack.sh]"]
  node_stack_manifest["Stack resources<br/>[stack.yaml]"]
end

subgraph group_chat["Chat and inference"]
  node_public_lb{{"Public LoadBalancer<br/>[stack.yaml]"}}
  node_openwebui["Open WebUI<br/>[stack.yaml]"]
  node_vllm["vLLM API<br/>[stack.yaml]"]
end

subgraph group_state["Application state"]
  node_postgres[("Postgres pgvector<br/>[stack.yaml]")]
  node_redis[("Redis<br/>[stack.yaml]")]
end

subgraph group_observability["Monitoring and scaling"]
  node_prometheus[("Prometheus<br/>[stack.yaml]")]
  node_grafana["Grafana<br/>[stack.yaml]"]
  node_keda["KEDA scaler<br/>[stack.yaml]"]
  node_openwebui_hpa["Open WebUI HPA<br/>[stack.yaml]"]
  node_openwebui_pdb["Open WebUI PDB<br/>[stack.yaml]"]
end

node_browser(("Browser user"))
node_operator(("Cluster operator"))

node_operator -->|"runs"| node_setup
node_setup -->|"applies"| node_stack_manifest
node_browser -->|"sends HTTP"| node_public_lb
node_public_lb -->|"routes traffic"| node_openwebui
node_openwebui -->|"requests inference"| node_vllm
node_vllm -->|"returns completions"| node_openwebui
node_openwebui -->|"stores app data"| node_postgres
node_openwebui -->|"coordinates sessions"| node_redis
node_vllm -->|"exposes metrics"| node_prometheus
node_prometheus -->|"provides datasource"| node_grafana
node_keda -->|"queries queue metrics"| node_prometheus
node_keda -.->|"scales replicas"| node_vllm
node_openwebui_hpa -.->|"scales replicas"| node_openwebui
node_openwebui_pdb -.->|"protects availability"| node_openwebui

click node_setup "https://github.com/ali-ncibi/llm-stack/blob/main/stack.sh"
click node_stack_manifest "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_public_lb "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_openwebui "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_vllm "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_postgres "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_redis "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_prometheus "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_grafana "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_keda "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_openwebui_hpa "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"
click node_openwebui_pdb "https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_setup,node_stack_manifest,node_browser toneBlue
class node_public_lb,node_openwebui,node_vllm toneAmber
class node_postgres,node_redis toneMint
class node_prometheus,node_grafana,node_keda,node_openwebui_hpa,node_openwebui_pdb toneRose
class node_operator toneIndigo
```

## Components

### Deployment

- [Stack setup](https://github.com/ali-ncibi/llm-stack/blob/main/stack.sh): `stack.sh`
- [Stack resources](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`

### Chat and inference

- [Public LoadBalancer](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`
- [Open WebUI](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`
- [vLLM API](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`

### Application state

- [Postgres pgvector](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`
- [Redis](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`

### Monitoring and scaling

- [Prometheus](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`
- [Grafana](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`
- [KEDA scaler](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`
- [Open WebUI HPA](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`
- [Open WebUI PDB](https://github.com/ali-ncibi/llm-stack/blob/main/manifests/stack.yaml): `manifests/stack.yaml`

### Other components

- **Browser user**
- **Cluster operator**

## Connections

- Cluster operator → Stack setup: runs (evidence: `README.md`)
- Stack setup → Stack resources: applies (evidence: `README.md`)
- Browser user → Public LoadBalancer: sends HTTP (evidence: `README.md`)
- Public LoadBalancer → Open WebUI: routes traffic (evidence: `README.md`)
- Open WebUI → vLLM API: requests inference (evidence: `README.md`)
- vLLM API → Open WebUI: returns completions (evidence: `README.md`)
- Open WebUI → Postgres pgvector: stores app data (evidence: `README.md`)
- Open WebUI → Redis: coordinates sessions (evidence: `README.md`)
- vLLM API → Prometheus: exposes metrics (evidence: `README.md`)
- Prometheus → Grafana: provides datasource (evidence: `README.md`)
- KEDA scaler → Prometheus: queries queue metrics (evidence: `README.md`)
- KEDA scaler → vLLM API: scales replicas (evidence: `README.md`)
- Open WebUI HPA → Open WebUI: scales replicas (evidence: `README.md`)
- Open WebUI PDB → Open WebUI: protects availability (evidence: `README.md`)
