How williamcaban/ollama-ubi works
Main Components
• Container Build & Image:
• The Containerfile/Containerfile.build are used to create the UBI-based image.
• build.sh and upload.sh handle the build and image publishing process (with the image ending up on quay.io).
• Kubernetes Deployment:
• Manifests directory contains YAML files for the persistent volume claim (PVC), deployment configuration, service, and routing (Ingress) for exposing the API.
• The deployment manifest defines the ollama server container (which handles pulling and running LLMs).
• Scripts/Dependencies:
• The scripts (e.g., scripts/rh_linux_deps.sh) are included to manage host dependencies that might be necessary on Red Hat Linux environments, ensuring the proper environment for image building or testing.
Relationships & Data Flow
• The build process creates the container image from the Containerfile and associated scripts.
• Once built and uploaded, the image is deployed via Kubernetes/OpenShift:
• The PVC ensures that any persistent data (if needed) is stored reliably.
• The Deployment resource runs one or more pods containing the ollama server.
• The Service (ollama-svc) directs traffic from the deployment pods.
• The Route (ollama-route) exposes the API externally through a unique URL.
• External users interact with the service’s API (default port 11434) via the route.
• Additionally, once the pod is up, manual commands via oc exec can download large language models (LLMs) into the container.
Important Architectural Patterns & Design Principles
• Containerization: Using UBI-based images ensures compatibility with OpenShift environments.
• Declarative Infrastructure: Kubernetes manifests (manifests/*.yaml) represent a declarative approach to deploying and managing the lifecycle of the service.
• Separation of Concerns: The build scripts, container definition, and Kubernetes deployment are cleanly separated, making it easier to update the application, scale deployment, or manage model downloads independently.
• Integration with External Services: The setup assumes the presence of the NVIDIA OpenShift Operator for GPU support, which is essential for handling compute-heavy LLM tasks.
Elements
• "Build Environment" or "CI/CD Pipeline" representing build.sh, Containerfile, and upload.sh.
• "Container Image" hosted on quay.io.
• "Persistent Volume (PVC)" used for storage.
• "Deployment" (illustrating pods running the ollama server).
• "Service (ollama-svc)" that balances traffic among pods.
• "Route (ollama-route)" that exposes the service externally.
• "External Client" that interacts with the ollama API.
• From Build Environment to Container Image (showing the build/push pipeline).
• From Container Image to Deployment (indicating that the image is used to run pods).
• From Persistent Volume to Deployment (showing storage linkage).
• From Route to Service and then to Deployment (illustrating request flow from external clients).