How ayaanshaheer/kaust-hpc-inference works
This is an infrastructure and operations lab for running LLM inference on a Google Cloud Slurm cluster. The documented workflow is researcher access to the login node, submission of a GPU batch job, Slurm scheduling and GPU-node provisioning, then Apptainer launching vLLM to serve the model. Shared Filestore, monitoring, and cross-layer quota troubleshooting are distinct supporting capabilities. Source excerpts were unavailable, so links reflect README-documented relationships only.
Main entry points: kaust-ai-slurm.yaml GPU compute nodes · serve-llm.sbatch vLLM server · nova.sh Cluster monitor
Components 18
Cloud Infrastructure
- Cluster blueprint
kaust-ai-slurm.yaml - Google Cloud
- VPC and networking
kaust-ai-slurm.yaml - Filestore NFS
kaust-ai-slurm.yaml - Login node
kaust-ai-slurm.yaml - Slurm controller
kaust-ai-slurm.yaml - GPU compute nodes
kaust-ai-slurm.yaml - GPU partition
kaust-ai-slurm.yaml
Inference Workload
- Inference batch job
serve-llm.sbatch - Apptainer container
serve-llm.sbatch - vLLM server
serve-llm.sbatch - OPT-125M model
serve-llm.sbatch - Inference service
serve-llm.sbatch
Operations
- Cluster monitor
nova.sh - Slurm jobs and nodes
- GPU quota
Other components
- Researcher
- Internet
Connections 17
- Researcher → Login node: SSH / IAP
- Login node → Slurm controller: submits jobs
- Slurm controller → GPU partition: schedules jobs
- Slurm controller → GPU compute nodes: provisions nodes
- Login node → Filestore NFS: reads / writes
- Slurm controller → Filestore NFS: shares storage
- GPU compute nodes → Filestore NFS: accesses files
- Inference batch job → Apptainer container: launches
- Apptainer container → vLLM server: runs
- vLLM server → OPT-125M model: serves
- vLLM server → Inference service: exposes
- GPU compute nodes → Apptainer container: runs workload
- Cluster monitor → Slurm jobs and nodes: queries squeue / sinfo
- VPC and networking → Internet: routes via NAT
- Google Cloud → GPU compute nodes: provisions
- Google Cloud → GPU quota: enforces
- GPU quota → GPU compute nodes: can block provisioning