How acchapm1/nim-container-demo works
Project type and purpose
• This is a demo repository for “AI Workflows on HPC with NVIDIA NIM.” It shows how to run NVIDIA NIM inference microservices inside Apptainer containers on an HPC cluster, using Slurm for scheduling and also supporting local testing.
• Key indicators:
• README.md describes running “NVIDIA NIM containers using Apptainer on an HPC cluster.”
• Directories like examples/, asciinema/, and nim.md indicate hands-on demos and documentation.
• A slides/ folder hosts a static site (Vue components and markdown) for presentation.
File structure
• Top-level items:
• asciinema/: recorded terminal demos
• examples/: scripts and code to launch Llama/LOL-COW demos via Slurm or locally
• nim.md: deep-dive documentation on NIM usage
• slides/: Vue-based slide deck with Netlify/Vercel configs
• There is no “frontend/backend” MVC, but rather:
• Client interfaces (CLI scripts, Python examples)
• HPC infrastructure (Slurm scheduler, Apptainer container runtime)
• NVIDIA NIM microservice running inside containers
• Deployment/configuration files:
• slides/netlify.toml, slides/vercel.json (slides hosting)
• Slurm job scripts under examples/llama3
README for additional insights
• Architecture is not drawn but implied: client submits jobs, Apptainer runs NIM, NIM serves inference, results returned.
• Dependencies: Apptainer, Slurm, NVIDIA NIM containers, optional local Docker testing.
a. Main components
• Developer Machine (local testing)
• HPC Head Node (Slurm controller)
• Compute Nodes (running Apptainer)
• NVIDIA NIM Container (inference microservice)
• Client Interfaces (bash scripts, Python clients)
• Container Registry (source of NIM container images)
b. Relationships & interactions
1. User runs runLlama.sh or Slurm script → submits job to Slurm on Head Node
2. Slurm schedules job to a Compute Node
3. Compute Node pulls NIM container from registry (if not cached)
4. Apptainer starts the NIM microservice inside the container
5. The client script (inside the container or via forwarded port) sends inference requests to the NIM service
6. NIM processes input and returns output back to the client
c. Architectural patterns/design principles
• Microservice pattern: NIM as a standalone inference service inside a container
• Batch scheduling: Slurm decouples job submission from execution
• Infrastructure-as-code: job scripts and container definitions versioned in Git
d. Technologies & frameworks
• Slurm (job scheduler)
• Apptainer (container runtime on HPC)
• NVIDIA NIM (inference microservice)
• Bash/Python for client scripts
• Asciinema for demo recording
• Vue.js + VitePress (slides)