How virtualdisk/containers works
Project type and purpose
• Type: A monolithic GitHub-hosted repository implementing a CI/CD pipeline to build, test, and publish custom Docker container images.
• Purpose: Automatically produce and update a family of container images (apps/…) by layering custom configurations atop upstream images, then push them to GitHub Container Registry.
File structure
• Top-level files:
– README.md, LICENSE: documentation and licensing.
– Taskfile.yml: local CLI orchestration of builds.
– metadata.rules.cue: schema definitions for container metadata.
• .github directory:
– workflows/*.yaml: GitHub Actions pipelines (e.g., build-images.yaml, release-scheduled.yaml).
– scripts/*.py and templates: helper scripts to generate build matrices, render READMEs, convert JSON→YAML.
• apps/<image-name> directories:
– Dockerfile: build instructions for each image.
– ci/: Goss tests (goss.yaml) and helper scripts (latest.sh).
– metadata.yaml: image metadata consumed by scripts.
Extract insights from README
• The repo builds images tagged both “rolling” and version-specific.
• Consumers can pull from ghcr.io/VirtualDisk/<image>.
• Builds are triggered on code changes and on a scheduled cadence.
Main components and interactions
1. GitHub Repository (source code)
2. GitHub Actions Workflows
• build-images workflow
• get-changed-images workflow
• release-scheduled workflow
3. Build Orchestration Scripts
• matrix generators (json-to-yaml.py, prepare-matrices.py)
• README templating (render-readme.py)
4. Taskfile CLI (developer-invoked build)
5. apps/* containers
• Docker Build Context (Dockerfile + metadata.yaml)
• CI tests (goss)
6. CUE Validator (metadata.rules.cue ensures metadata.yml conformance)
7. Docker Engine (build executor)
8. GitHub Container Registry (ghcr.io/VirtualDisk)
9. Upstream Registries (Docker Hub base images referenced in each Dockerfile)
1. Developer commits → GitHub Repository
2. Push/Merge event → GitHub Actions Workflows
3. build-images workflow → invoke matrix generator scripts → emit job matrix
4. For each app in matrix:
– Checkout code → validate metadata with CUE → run Docker build via Taskfile or directly
– Run CI tests (goss) inside built container
– Tag image (rolling + version) → push to GHCR
5. release-scheduled workflow triggers periodic rebuilds
6. Consumers pull images from GHCR when deploying onto infrastructure
c. Architectural patterns and design principles
• Monorepo/CICD-first: All Dockerfiles and metadata live together, controlled via templated workflows.
• Build matrix generation: Dynamically discover apps, avoid manual job definitions.
• Schema-driven metadata: CUE enforces consistency across container metadata.
• GitOps style: Repo is single source of truth, and rebuilds happen automatically on schedule or code change.
d. Key technologies, frameworks, and libraries
• GitHub Actions for CI/CD
• Python scripts for matrix and README generation
• Taskfile (go-task) for local orchestration
• CUE for metadata validation
• Docker (buildx) for image builds
• Goss for container sanity tests
• GitHub Container Registry (GHCR) as image store
Final touches
• Consumer side to indicate where these images are eventually used, even if that’s external to this repo.