# ayaanshaheer/kaust-hpc-inference architecture

> Architecture diagram of the GitHub repository ayaanshaheer/kaust-hpc-inference, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ayaanshaheer/kaust-hpc-inference
- Repository: https://github.com/ayaanshaheer/kaust-hpc-inference

## Overview

This is an infrastructure and operations lab for running LLM inference on a Google Cloud Slurm cluster. The documented workflow is researcher access to the login node, submission of a GPU batch job, Slurm scheduling and GPU-node provisioning, then Apptainer launching vLLM to serve the model. Shared Filestore, monitoring, and cross-layer quota troubleshooting are distinct supporting capabilities. Source excerpts were unavailable, so links reflect README-documented relationships only.

Main entry points: [`kaust-ai-slurm.yaml`](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml) (GPU compute nodes), [`serve-llm.sbatch`](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch) (vLLM server), [`nova.sh`](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/nova.sh) (Cluster monitor)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_infrastructure["Cloud Infrastructure"]
  node_blueprint["Cluster blueprint"]
  node_cloud["Google Cloud"]
  node_vpc["VPC and networking"]
  node_filestore[("Filestore NFS")]
  node_login["Login node"]
  node_controller["Slurm controller"]
  node_gpu["GPU compute nodes"]
  node_partition["GPU partition"]
end

subgraph group_workload["Inference Workload"]
  node_job["Inference batch job<br/>[serve-llm.sbatch]"]
  node_apptainer["Apptainer container<br/>[serve-llm.sbatch]"]
  node_vllm["vLLM server<br/>[serve-llm.sbatch]"]
  node_model["OPT-125M model<br/>[serve-llm.sbatch]"]
  node_researcher_result["Inference service<br/>[serve-llm.sbatch]"]
end

subgraph group_operations["Operations"]
  node_monitor["Cluster monitor<br/>[nova.sh]"]
  node_scheduler_state[("Slurm jobs and nodes")]
  node_quota["GPU quota"]
end

node_researcher(("Researcher"))
node_internet(("Internet"))

node_researcher -->|"SSH / IAP"| node_login
node_login -->|"submits jobs"| node_controller
node_controller -->|"schedules jobs"| node_partition
node_controller -->|"provisions nodes"| node_gpu
node_login -->|"reads / writes"| node_filestore
node_controller -->|"shares storage"| node_filestore
node_gpu -->|"accesses files"| node_filestore
node_job -->|"launches"| node_apptainer
node_apptainer -->|"runs"| node_vllm
node_vllm -->|"serves"| node_model
node_vllm -->|"exposes"| node_researcher_result
node_gpu -->|"runs workload"| node_apptainer
node_monitor -->|"queries squeue / sinfo"| node_scheduler_state
node_vpc -->|"routes via NAT"| node_internet
node_cloud -->|"provisions"| node_gpu
node_cloud -->|"enforces"| node_quota
node_quota -.->|"can block provisioning"| node_gpu

click node_blueprint "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml"
click node_vpc "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml"
click node_filestore "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml"
click node_login "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml"
click node_controller "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml"
click node_gpu "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml"
click node_partition "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml"
click node_job "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch"
click node_apptainer "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch"
click node_vllm "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch"
click node_model "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch"
click node_researcher_result "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch"
click node_monitor "https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/nova.sh"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_blueprint,node_cloud,node_vpc,node_filestore,node_login,node_controller,node_gpu,node_partition toneBlue
class node_job,node_apptainer,node_vllm,node_model,node_researcher_result toneAmber
class node_monitor,node_scheduler_state,node_quota toneMint
class node_researcher,node_internet toneIndigo
```

## Components

### Cloud Infrastructure

- [Cluster blueprint](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml): `kaust-ai-slurm.yaml`
- **Google Cloud**
- [VPC and networking](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml): `kaust-ai-slurm.yaml`
- [Filestore NFS](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml): `kaust-ai-slurm.yaml`
- [Login node](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml): `kaust-ai-slurm.yaml`
- [Slurm controller](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml): `kaust-ai-slurm.yaml`
- [GPU compute nodes](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml): `kaust-ai-slurm.yaml`
- [GPU partition](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/kaust-ai-slurm.yaml): `kaust-ai-slurm.yaml`

### Inference Workload

- [Inference batch job](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch): `serve-llm.sbatch`
- [Apptainer container](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch): `serve-llm.sbatch`
- [vLLM server](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch): `serve-llm.sbatch`
- [OPT-125M model](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch): `serve-llm.sbatch`
- [Inference service](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/serve-llm.sbatch): `serve-llm.sbatch`

### Operations

- [Cluster monitor](https://github.com/ayaanshaheer/kaust-hpc-inference/blob/main/nova.sh): `nova.sh`
- **Slurm jobs and nodes**
- **GPU quota**

### Other components

- **Researcher**
- **Internet**

## Connections

- Researcher → Login node: SSH / IAP
- Login node → Slurm controller: submits jobs
- Slurm controller → GPU partition: schedules jobs
- Slurm controller → GPU compute nodes: provisions nodes
- Login node → Filestore NFS: reads / writes
- Slurm controller → Filestore NFS: shares storage
- GPU compute nodes → Filestore NFS: accesses files
- Inference batch job → Apptainer container: launches
- Apptainer container → vLLM server: runs
- vLLM server → OPT-125M model: serves
- vLLM server → Inference service: exposes
- GPU compute nodes → Apptainer container: runs workload
- Cluster monitor → Slurm jobs and nodes: queries squeue / sinfo
- VPC and networking → Internet: routes via NAT
- Google Cloud → GPU compute nodes: provisions
- Google Cloud → GPU quota: enforces
- GPU quota → GPU compute nodes: can block provisioning