# acchapm1/nim-container-demo architecture

> Architecture diagram of the GitHub repository acchapm1/nim-container-demo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/acchapm1/nim-container-demo
- Repository: https://github.com/acchapm1/nim-container-demo

## Overview

##### Project type and purpose
- This is a demo repository for “AI Workflows on HPC with NVIDIA NIM.” It shows how to run NVIDIA NIM inference microservices inside Apptainer containers on an HPC cluster, using Slurm for scheduling and also supporting local testing.
- Key indicators:
- README.md describes running “NVIDIA NIM containers using Apptainer on an HPC cluster.”
- Directories like `examples/`, `asciinema/`, and `nim.md` indicate hands-on demos and documentation.
- A `slides/` folder hosts a static site (Vue components and markdown) for presentation.

##### File structure
- Top-level items:
- `asciinema/`: recorded terminal demos
- `examples/`: scripts and code to launch Llama/LOL-COW demos via Slurm or locally
- `nim.md`: deep-dive documentation on NIM usage
- `slides/`: Vue-based slide deck with Netlify/Vercel configs
- There is no “frontend/backend” MVC, but rather:
- Client interfaces (CLI scripts, Python examples)
- HPC infrastructure (Slurm scheduler, Apptainer container runtime)
- NVIDIA NIM microservice running inside containers
- Deployment/configuration files:
- `slides/netlify.toml`, `slides/vercel.json` (slides hosting)
- Slurm job scripts under `examples/llama3`

##### README for additional insights
- Architecture is not drawn but implied: client submits jobs, Apptainer runs NIM, NIM serves inference, results returned.
- Dependencies: Apptainer, Slurm, NVIDIA NIM containers, optional local Docker testing.

a. Main components
- Developer Machine (local testing)
- HPC Head Node (Slurm controller)
- Compute Nodes (running Apptainer)
- NVIDIA NIM Container (inference microservice)
- Client Interfaces (bash scripts, Python clients)
- Container Registry (source of NIM container images)
b. Relationships & interactions
1. User runs `runLlama.sh` or Slurm script → submits job to Slurm on Head Node
2. Slurm schedules job to a Compute Node
3. Compute Node pulls NIM container from registry (if not cached)
4. Apptainer starts the NIM microservice inside the container
5. The client script (inside the container or via forwarded port) sends inference requests to the NIM service
6. NIM processes input and returns output back to the client
c. Architectural patterns/design principles
- Microservice pattern: NIM as a standalone inference service inside a container
- Batch scheduling: Slurm decouples job submission from execution
- Infrastructure-as-code: job scripts and container definitions versioned in Git
d. Technologies & frameworks
- Slurm (job scheduler)
- Apptainer (container runtime on HPC)
- NVIDIA NIM (inference microservice)
- Bash/Python for client scripts
- Asciinema for demo recording
- Vue.js + VitePress (slides)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Developer Machine Local Testing
    subgraph "Developer Machine (Local Testing)" 
        direction TB
        CLI1["runLlama.sh"]:::infra
        CLI2["bashllmchat.sh"]:::infra
        CLI3["pyllmchat.py"]:::infra
        CLI4["llm-info.py"]:::infra
    end

    %% HPC Cluster
    subgraph "HPC Cluster" 
        direction TB
        HeadNode["Slurm Controller (Head Node)"]:::infra
        ComputeNode["Compute Node"]:::infra
        subgraph "On Compute Node" 
            direction TB
            Apptainer["Apptainer Runtime"]:::infra
            NIMService["NIM Inference Microservice"]:::service
        end
    end

    Registry["Container Registry"]:::registry

    %% Asciinema Demos
    subgraph "Asciinema Recorded Demos"
        direction TB
        Demo1["llama2-demo.cast"]:::docs
        Demo2["llama3-demo.cast"]:::docs
    end

    %% Slides Presentation
    subgraph "Slides Presentation (Vue.js + VitePress)"
        direction TB
        IG[".gitignore"]:::frontend
        NPMRC[".npmrc"]:::frontend
        README["README.md"]:::docs
        PKG["package.json"]:::frontend
        LOCK["pnpm-lock.yaml"]:::frontend
        NETLIFY["netlify.toml"]:::infra
        VERCEL["vercel.json"]:::infra
        CVUE["Counter.vue"]:::frontend
        IMPORTED["imported-slides.md"]:::docs
        SLIDESMD["slides.md"]:::docs
        EXTSNIP["external.ts"]:::frontend
    end

    %% Flows
    CLI1 -->|"submit job"| HeadNode
    CLI2 -->|"submit job"| HeadNode
    HeadNode -->|"dispatch job"| ComputeNode
    ComputeNode -->|"pull image"| Registry
    ComputeNode -->|"launch container"| Apptainer
    Apptainer -->|"start service"| NIMService
    CLI3 -->|"inference request"| NIMService
    CLI4 -->|"info request"| NIMService

    %% Asciinema demos references
    CLI1 -.-> Demo2
    CLI1 -.-> Demo1

    %% Styles
    classDef infra fill:#ccf,stroke:#3366cc,color:#000
    classDef service fill:#cfc,stroke:#339933,color:#000
    classDef registry fill:#fcf,stroke:#cc33cc,color:#000
    classDef frontend fill:#f9f,stroke:#c0392b,color:#000
    classDef docs fill:#ffc,stroke:#f39c12,color:#000

    %% Click Events for component mappings
    click CLI1 "https://github.com/acchapm1/nim-container-demo/blob/main/examples/llama3/runLlama.sh"
    click CLI2 "https://github.com/acchapm1/nim-container-demo/blob/main/examples/llama3/bashllmchat.sh"
    click CLI3 "https://github.com/acchapm1/nim-container-demo/blob/main/examples/llama3/pyllmchat.py"
    click CLI4 "https://github.com/acchapm1/nim-container-demo/blob/main/examples/llama3/llm-info.py"
    click HeadNode "https://github.com/acchapm1/nim-container-demo/blob/main/examples/llama3/slurmLlama.sh"
    click Apptainer "https://github.com/acchapm1/nim-container-demo/blob/main/examples/llama3/runLlama.sh"
    click NIMService "https://github.com/acchapm1/nim-container-demo/blob/main/nim.md"
    click Demo1 "https://github.com/acchapm1/nim-container-demo/blob/main/asciinema/llama2-demo.cast"
    click Demo2 "https://github.com/acchapm1/nim-container-demo/blob/main/asciinema/llama3-demo.cast"
    click IG "https://github.com/acchapm1/nim-container-demo/blob/main/slides/.gitignore"
    click NPMRC "https://github.com/acchapm1/nim-container-demo/blob/main/slides/.npmrc"
    click README "https://github.com/acchapm1/nim-container-demo/blob/main/slides/README.md"
    click PKG "https://github.com/acchapm1/nim-container-demo/blob/main/slides/package.json"
    click LOCK "https://github.com/acchapm1/nim-container-demo/blob/main/slides/pnpm-lock.yaml"
    click NETLIFY "https://github.com/acchapm1/nim-container-demo/blob/main/slides/netlify.toml"
    click VERCEL "https://github.com/acchapm1/nim-container-demo/blob/main/slides/vercel.json"
    click CVUE "https://github.com/acchapm1/nim-container-demo/blob/main/slides/components/Counter.vue"
    click IMPORTED "https://github.com/acchapm1/nim-container-demo/blob/main/slides/pages/imported-slides.md"
    click SLIDESMD "https://github.com/acchapm1/nim-container-demo/blob/main/slides/slides.md"
    click EXTSNIP "https://github.com/acchapm1/nim-container-demo/blob/main/slides/snippets/external.ts"
```
