How ali-ncibi/llm-stack works
This repository describes a self-hosted Kubernetes LLM chat stack. The main user workflow is browser → public Open WebUI → vLLM → generated response; Open WebUI also uses PostgreSQL/pgvector and Redis. Separate operational branches cover deployment, queue-based and CPU-based scaling, disruption protection, and monitoring. The README documents these connections and capabilities; the manifest itself was not sampled, so the graph does not claim details beyond that documentation.
Main entry points: stack.yaml Open WebUI · stack.sh Stack setup
Components 14
Deployment
- Stack setup
stack.sh - Stack resources
manifests/stack.yaml
Chat and inference
- Public LoadBalancer
manifests/stack.yaml - Open WebUI
manifests/stack.yaml - vLLM API
manifests/stack.yaml
Application state
- Postgres pgvector
manifests/stack.yaml - Redis
manifests/stack.yaml
Monitoring and scaling
- Prometheus
manifests/stack.yaml - Grafana
manifests/stack.yaml - KEDA scaler
manifests/stack.yaml - Open WebUI HPA
manifests/stack.yaml - Open WebUI PDB
manifests/stack.yaml
Other components
- Browser user
- Cluster operator
Connections 14
- Cluster operator → Stack setup: runs
README.md - Stack setup → Stack resources: applies
README.md - Browser user → Public LoadBalancer: sends HTTP
README.md - Public LoadBalancer → Open WebUI: routes traffic
README.md - Open WebUI → vLLM API: requests inference
README.md - vLLM API → Open WebUI: returns completions
README.md - Open WebUI → Postgres pgvector: stores app data
README.md - Open WebUI → Redis: coordinates sessions
README.md - vLLM API → Prometheus: exposes metrics
README.md - Prometheus → Grafana: provides datasource
README.md - KEDA scaler → Prometheus: queries queue metrics
README.md - KEDA scaler → vLLM API: scales replicas
README.md - Open WebUI HPA → Open WebUI: scales replicas
README.md - Open WebUI PDB → Open WebUI: protects availability
README.md