# camenduru/non-profit-gpu-cluster architecture

> Architecture diagram of the GitHub repository camenduru/non-profit-gpu-cluster, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/camenduru/non-profit-gpu-cluster
- Repository: https://github.com/camenduru/non-profit-gpu-cluster

## Overview

##### Project’s Nature and Purpose
- This is an infrastructure-as-code repository for a multi-node GPU cluster running a variety of open-source AI demos (Stable Diffusion, Visual Style Prompting, Instant-ID, Dust3r, TriposR, CRM, Forge, etc.) via containerized UIs (code-server, Jupyter-Lab, web dashboards).
- Server 1 & 2: GPU compute nodes. Server 3: control plane (database, web front-end, dispatcher, scheduler).
- Exposed to the public via Cloudflare Tunnel → NGINX reverse proxy → per-service endpoints.

##### Top-Level Components
a. Physical/GPU Nodes
- Node 1 & Node 2 (RTX 3090/A5000 GPUs, Docker + NVIDIA drivers)
- Node 3 (Web server, MongoDB database, Dispatcher & Scheduler microservices)
b. Network & Gateway
- Cloudflare Argo Tunnel (“cloudflared” service)
- NGINX reverse proxy (nginx.conf + redirect.py)
- Ports 80/443 → internal service ports (e.g. 8080, 7860, custom ports)
c. Service Orchestration
- systemd units in /services/*.service
- Docker Engine + NVIDIA Container Toolkit
- Dockerfiles under code-server/
d. Microservices / Containers
- code-server (OpenVSCode)
- jupyter-lab
- Stable-Diffusion-WebUI
- Stable-Cascade
- VisualStylePrompting
- Dust3r
- Instant-ID
- TriposR
- CRM, Forge
- Dispatcher & Scheduler (custom Python/Node microservices)
e. Database
- MongoDB (persistent store for web, scheduler state, usage tracking)

##### Relationships & Data Flows
- External User → HTTPS → Cloudflare Tunnel → NGINX
- NGINX executes redirect.py to route to specific service port via systemd socket or reverse proxy.
- The request reaches the containerized service on Node 1/2 (if GPU-bound) or Node 3 (if web/dashboard).
- For GPU workloads: service container → Docker → NVIDIA drivers → GPU hardware.
- Dispatcher service communicates with Scheduler service and updates MongoDB.
- Web front-end reads/writes data to MongoDB.
- Jupyter-Lab and code-server provide interactive notebooks/editors on GPU nodes.

##### Architectural Patterns
- Microservices: each AI demo runs in its own container + systemd unit.
- Reverse‐proxy + routing logic: single entry point (NGINX + redirect.py) directs traffic to dozens of UIs.
- Containerization: isolation, GPU access through NVIDIA Container Toolkit.
- Service orchestration via systemd rather than a full Kubernetes stack.
- Central control plane (Scheduler, Dispatcher, DB) separated from compute nodes.

##### Technologies & Frameworks to Annotate
- Docker Engine, NVIDIA Container Toolkit
- systemd service files for each microservice
- NGINX (reverse proxy) + Python redirect script
- cloudflared (Cloudflare Argo Tunnel)
- Python 3.10 (Torch, xformers), Node.js (some web UIs)
- MongoDB
- CUDA 12.1 drivers, OSS dependencies
- code-server (OpenVSCode), Jupyter-Lab

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External
    ExternalUser["External User"]:::external
    %% Network Layer
    Cloudflare["Cloudflare Argo Tunnel\n(cloudflared)"]:::network
    NGINX["NGINX Reverse Proxy\n(nginx.conf + redirect.py)"]:::network

    ExternalUser -->|"HTTPS"| Cloudflare
    Cloudflare -->|"tunnel"| NGINX

    %% Compute Nodes
    subgraph "Node 1 (GPU Compute)\n4× NVIDIA GPUs" 
        direction TB
        Docker1["Docker Engine +\nNVIDIA Container Toolkit"]:::docker
        subgraph "Containers via systemd" 
            direction TB
            SDWebUI["Stable Diffusion WebUI"]:::compute
            SDcascade["Stable Cascade"]:::compute
            VSP["VisualStylePrompting"]:::compute
            Dust3r["Dust3r"]:::compute
            InstantID["Instant-ID"]:::compute
            TriposR["TriposR"]:::compute
            CRM["CRM"]:::compute
            Forge["Forge"]:::compute
            Jupyter["Jupyter-Lab"]:::compute
            CodeServer["code-server"]:::compute
        end
        SDWebUI --> Docker1
        SDcascade --> Docker1
        VSP --> Docker1
        Dust3r --> Docker1
        InstantID --> Docker1
        TriposR --> Docker1
        CRM --> Docker1
        Forge --> Docker1
        Jupyter --> Docker1
        CodeServer --> Docker1
        Docker1 -->|"CUDA calls"| GPU1["GPUs"]:::compute
    end

    subgraph "Node 2 (GPU Compute)\n4× NVIDIA GPUs"
        direction TB
        Docker2["Docker Engine +\nNVIDIA Container Toolkit"]:::docker
        %% Services mirrored from Node1
        SDWebUI2["Stable Diffusion WebUI"]:::compute
        SDcascade2["Stable Cascade"]:::compute
        SDWebUI2 --> Docker2
        SDcascade2 --> Docker2
        Docker2 -->|"CUDA calls"| GPU2["GPUs"]:::compute
    end

    %% Control Plane
    subgraph "Node 3 (Control Plane)"
        direction TB
        Dispatcher["Dispatcher Service"]:::control
        Scheduler["Scheduler Service"]:::control
        MongoDB["MongoDB"]:::control
        WebUI["Web Front-End"]:::control
        Dispatcher --> Scheduler
        Scheduler --> MongoDB
        WebUI --> MongoDB
    end

    %% Routing to services
    NGINX -->|"routes GPU workloads"| Node1
    NGINX -->|"routes GPU workloads"| Node2
    NGINX -->|"routes dashboards"| Node3

    %% Click events
    click CodeServer "https://github.com/camenduru/non-profit-gpu-cluster/tree/main/code-server/Dockerfile_gpu"
    click CodeServer "https://github.com/camenduru/non-profit-gpu-cluster/tree/main/code-server/Dockerfile_cpu"
    click CodeServer "https://github.com/camenduru/non-profit-gpu-cluster/tree/main/code-server/Dockerfile_user"
    click NGINX "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/nginx.conf"
    click NGINX "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/redirect.py"
    click SDWebUI "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/stable-diffusion-webui.service"
    click SDcascade "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/stable-cascade.service"
    click VSP "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/VisualStylePrompting.service"
    click Dust3r "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/dust3r.service"
    click InstantID "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/instant-id.service"
    click TriposR "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/triposr.service"
    click CRM "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/crm.service"
    click Forge "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/forge.service"
    click Jupyter "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/jupyter-lab.service"
    click CodeServer "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/services/code-server@root.service"
    click InstallerConfig "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/00-installer-config.yaml"
    click README "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/README.md"
    click SERVICES "https://github.com/camenduru/non-profit-gpu-cluster/blob/main/SERVICES.md"
    click SSHConfig "https://github.com/camenduru/non-profit-gpu-cluster/tree/main/sshd_config"

    %% Styles
    classDef compute fill:#e0ffe0,stroke:#008000
    classDef network fill:#cce5ff,stroke:#004080
    classDef control fill:#ffe0b3,stroke:#b36b00
    classDef external fill:#eeeeee,stroke:#888888,stroke-dasharray: 5 5
    classDef docker fill:#f2f2f2,stroke:#666666
    class InstallerConfig,README,SERVICES,SSHConfig internal
```
