How camenduru/non-profit-gpu-cluster works
Project’s Nature and Purpose
• This is an infrastructure-as-code repository for a multi-node GPU cluster running a variety of open-source AI demos (Stable Diffusion, Visual Style Prompting, Instant-ID, Dust3r, TriposR, CRM, Forge, etc.) via containerized UIs (code-server, Jupyter-Lab, web dashboards).
• Server 1 & 2: GPU compute nodes. Server 3: control plane (database, web front-end, dispatcher, scheduler).
• Exposed to the public via Cloudflare Tunnel → NGINX reverse proxy → per-service endpoints.
Top-Level Components
a. Physical/GPU Nodes
• Node 1 & Node 2 (RTX 3090/A5000 GPUs, Docker + NVIDIA drivers)
• Node 3 (Web server, MongoDB database, Dispatcher & Scheduler microservices)
b. Network & Gateway
• Cloudflare Argo Tunnel (“cloudflared” service)
• NGINX reverse proxy (nginx.conf + redirect.py)
• Ports 80/443 → internal service ports (e.g. 8080, 7860, custom ports)
c. Service Orchestration
• systemd units in /services/*.service
• Docker Engine + NVIDIA Container Toolkit
• Dockerfiles under code-server/
d. Microservices / Containers
• code-server (OpenVSCode)
• jupyter-lab
• Stable-Diffusion-WebUI
• Stable-Cascade
• VisualStylePrompting
• Dust3r
• Instant-ID
• TriposR
• CRM, Forge
• Dispatcher & Scheduler (custom Python/Node microservices)
e. Database
• MongoDB (persistent store for web, scheduler state, usage tracking)
Relationships & Data Flows
• External User → HTTPS → Cloudflare Tunnel → NGINX
• NGINX executes redirect.py to route to specific service port via systemd socket or reverse proxy.
• The request reaches the containerized service on Node 1/2 (if GPU-bound) or Node 3 (if web/dashboard).
• For GPU workloads: service container → Docker → NVIDIA drivers → GPU hardware.
• Dispatcher service communicates with Scheduler service and updates MongoDB.
• Web front-end reads/writes data to MongoDB.
• Jupyter-Lab and code-server provide interactive notebooks/editors on GPU nodes.
Architectural Patterns
• Microservices: each AI demo runs in its own container + systemd unit.
• Reverse‐proxy + routing logic: single entry point (NGINX + redirect.py) directs traffic to dozens of UIs.
• Containerization: isolation, GPU access through NVIDIA Container Toolkit.
• Service orchestration via systemd rather than a full Kubernetes stack.
• Central control plane (Scheduler, Dispatcher, DB) separated from compute nodes.
Technologies & Frameworks to Annotate
• Docker Engine, NVIDIA Container Toolkit
• systemd service files for each microservice
• NGINX (reverse proxy) + Python redirect script
• cloudflared (Cloudflare Argo Tunnel)
• Python 3.10 (Torch, xformers), Node.js (some web UIs)
• MongoDB
• CUDA 12.1 drivers, OSS dependencies
• code-server (OpenVSCode), Jupyter-Lab