# virtualdisk/containers architecture

> Architecture diagram of the GitHub repository virtualdisk/containers, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/virtualdisk/containers
- Repository: https://github.com/virtualdisk/containers

## Overview

##### Project type and purpose
- Type: A monolithic GitHub-hosted repository implementing a CI/CD pipeline to build, test, and publish custom Docker container images.
- Purpose: Automatically produce and update a family of container images (apps/…) by layering custom configurations atop upstream images, then push them to GitHub Container Registry.

##### File structure
- Top-level files:
– README.md, LICENSE: documentation and licensing.
– Taskfile.yml: local CLI orchestration of builds.
– metadata.rules.cue: schema definitions for container metadata.
- .github directory:
– workflows/*.yaml: GitHub Actions pipelines (e.g., build-images.yaml, release-scheduled.yaml).
– scripts/*.py and templates: helper scripts to generate build matrices, render READMEs, convert JSON→YAML.
- apps/<image-name> directories:
– Dockerfile: build instructions for each image.
– ci/: Goss tests (goss.yaml) and helper scripts (latest.sh).
– metadata.yaml: image metadata consumed by scripts.

##### Extract insights from README
- The repo builds images tagged both “rolling” and version-specific.
- Consumers can pull from ghcr.io/VirtualDisk/<image>.
- Builds are triggered on code changes and on a scheduled cadence.

##### Main components and interactions
1. GitHub Repository (source code)
2. GitHub Actions Workflows
- build-images workflow
- get-changed-images workflow
- release-scheduled workflow
3. Build Orchestration Scripts
- matrix generators (json-to-yaml.py, prepare-matrices.py)
- README templating (render-readme.py)
4. Taskfile CLI (developer-invoked build)
5. apps/* containers
- Docker Build Context (Dockerfile + metadata.yaml)
- CI tests (goss)
6. CUE Validator (metadata.rules.cue ensures metadata.yml conformance)
7. Docker Engine (build executor)
8. GitHub Container Registry (ghcr.io/VirtualDisk)
9. Upstream Registries (Docker Hub base images referenced in each Dockerfile)

1. Developer commits → GitHub Repository
2. Push/Merge event → GitHub Actions Workflows
3. build-images workflow → invoke matrix generator scripts → emit job matrix
4. For each app in matrix:
– Checkout code → validate metadata with CUE → run Docker build via Taskfile or directly
– Run CI tests (goss) inside built container
– Tag image (rolling + version) → push to GHCR
5. release-scheduled workflow triggers periodic rebuilds
6. Consumers pull images from GHCR when deploying onto infrastructure

c. Architectural patterns and design principles
- Monorepo/CICD-first: All Dockerfiles and metadata live together, controlled via templated workflows.
- Build matrix generation: Dynamically discover apps, avoid manual job definitions.
- Schema-driven metadata: CUE enforces consistency across container metadata.
- GitOps style: Repo is single source of truth, and rebuilds happen automatically on schedule or code change.

d. Key technologies, frameworks, and libraries
- GitHub Actions for CI/CD
- Python scripts for matrix and README generation
- Taskfile (go-task) for local orchestration
- CUE for metadata validation
- Docker (buildx) for image builds
- Goss for container sanity tests
- GitHub Container Registry (GHCR) as image store

##### Final touches
- Consumer side to indicate where these images are eventually used, even if that’s external to this repo.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Source Artifacts
    subgraph "Source Artifacts"
        Repo["GitHub Repository"]:::source
        Taskfile["Taskfile.yml"]:::process
        CUE["metadata.rules.cue"]:::source
        README["README.md"]:::source
        LICENSE["LICENSE"]:::source
    end

    %% Application Definitions
    subgraph "Applications (apps/*)"
        DockerCtx["Dockerfile + metadata.yaml"]:::source
        CITests["ci/goss.yaml + latest.sh"]:::process
    end

    %% CI/CD Workflows and Scripts
    subgraph "CI/CD"
        subgraph "Workflows (.github/workflows)"
            BuildImages["build-images.yaml"]:::process
            GetChanged["get-changed-images.yaml"]:::process
            ReleaseSched["release-scheduled.yaml"]:::process
        end
        subgraph "Scripts (.github/scripts)"
            JSON2YAML["json-to-yaml.py"]:::process
            PrepareMat["prepare-matrices.py"]:::process
            RenderReadme["render-readme.py"]:::process
            Templates["templates (README.md.j2)"]:::process
        end
    end

    %% Infrastructure
    subgraph "Infrastructure"
        DockerEngine["Docker Engine (buildx)"]:::infra
        Upstream["Upstream Registries"]:::infra
        GHCR["GitHub Container Registry"]:::infra
    end

    Consumer["Consumers (pull images)"]:::consumer

    %% Flows
    Repo -->|push/merge| BuildImages
    Repo -->|push/merge| GetChanged
    Repo -->|schedule| ReleaseSched
    BuildImages --> JSON2YAML
    BuildImages --> PrepareMat
    BuildImages --> RenderReadme
    GetChanged --> PrepareMat
    ReleaseSched --> PrepareMat

    PrepareMat -->|"emit matrix"| DockerCtx
    DockerCtx -->|validate metadata| CUE
    CUE -->|metadata OK| DockerEngine
    DockerEngine -->|build image| DockerEngine
    DockerEngine -->|run tests| CITests
    DockerEngine -->|tag & push| GHCR

    Upstream -->|base images| DockerEngine
    GHCR -->|pull| Consumer

    Taskfile -->|local build| DockerCtx

    %% Click events
    click Repo "https://github.com/virtualdisk/containers/blob/main/."
    click BuildImages "https://github.com/virtualdisk/containers/blob/main/.github/workflows/build-images.yaml"
    click GetChanged "https://github.com/virtualdisk/containers/blob/main/.github/workflows/get-changed-images.yaml"
    click ReleaseSched "https://github.com/virtualdisk/containers/blob/main/.github/workflows/release-scheduled.yaml"
    click JSON2YAML "https://github.com/virtualdisk/containers/blob/main/.github/scripts/json-to-yaml.py"
    click PrepareMat "https://github.com/virtualdisk/containers/blob/main/.github/scripts/prepare-matrices.py"
    click RenderReadme "https://github.com/virtualdisk/containers/blob/main/.github/scripts/render-readme.py"
    click Templates "https://github.com/virtualdisk/containers/blob/main/.github/scripts/templates/README.md.j2"
    click Taskfile "https://github.com/virtualdisk/containers/blob/main/Taskfile.yml"
    click CUE "https://github.com/virtualdisk/containers/blob/main/metadata.rules.cue"
    click DockerCtx "https://github.com/virtualdisk/containers/tree/main/apps/<image-name>/Dockerfile"
    click DockerCtx "https://github.com/virtualdisk/containers/blob/main/apps/<image-name>/metadata.yaml"
    click CITests "https://github.com/virtualdisk/containers/blob/main/apps/<image-name>/ci/goss.yaml"
    click CITests "https://github.com/virtualdisk/containers/blob/main/apps/<image-name>/ci/latest.sh"
    click README "https://github.com/virtualdisk/containers/blob/main/README.md"
    click LICENSE "https://github.com/virtualdisk/containers/tree/main/LICENSE"

    %% Styles
    classDef source fill:#cce5ff,stroke:#004085,color:#004085;
    classDef process fill:#d4edda,stroke:#155724,color:#155724;
    classDef infra fill:#fff3cd,stroke:#856404,color:#856404;
    classDef consumer fill:#f5c6cb,stroke:#721c24,color:#721c24;
```
