How digicatapult/bridgeai-gitops-infra works
Main Components
• Source Repository: Represent the Git repository (this project) that holds all configuration manifests.
• Production Cluster (“bridgeai-prod”)
• Local “kind-cluster”
• Within each cluster, identify and represent the following sub-components:
• Base Configuration and Namespaces (under the “base” folder)
• Application deployments:
• Airflow (with associated ConfigMaps, certificates, volumes, ingress/nginx configurations)
• MLFlow (including its associated MinIO backend for artifacts)
• Prediction Service (with its certificate and ingress setup along with a REST API/Swagger interface)
• KServe for model serving (managed via CRDs)
• Cert-manager (for TLS and certificate management)
• Secrets Management: Represent secrets (e.g., credentials for Airflow, MLFlow, GitHub, DockerHub, etc.) that are stored and later applied via Flux.
Relationships and Interactions
• That changes committed to the Git repository trigger Flux to synchronize configuration changes to the clusters.
• That certificate management and secret synchronization are centrally controlled by their respective modules in Flux (e.g., cert-manager, secret-sync within the “base” folders).
Architectural Patterns and Design Principles
• GitOps Practice: Emphasize how the entire infrastructure is declaratively defined and managed through Git commits.
• Separation of Environments: Differentiate production (“bridgeai-prod”) and development (“kind-cluster”) environments.
• Microservice Configuration: Although the applications might be standalone components (e.g., Airflow and MLFlow), they are all configured for integration via Flux and Kubernetes resources.
• Security and Access Management: Certificate management (TLS via cert-manager), secrets, and ingress configurations are all highlighted and central to the deployment.
Relevant Technologies and Frameworks
• Kubernetes (with CRDs for KServe and flux-specific components)
• Flux (v2 – GitOps tool for continuous reconciliation)
• Docker, kind, kubectl (for cluster management and deployments)
• NGINX (as an ingress controller configured via Helm releases)
• MinIO (used both for MLFlow artifact storage and for DVC/Evidently data storage)
Specific Diagramic Elements
• Clear Labels: Each component should be clearly named (e.g., “Flux”, “bridgeai-prod Cluster”, “kind-cluster”, “Airflow”, “MLFlow”, “Prediction Service”, “KServe”, “Cert-manager”, “Secrets”).
• Annotations could differentiate production resources from development resources.
• Be very detailed. Even if it means breaking down the cluster into multiple subcomponents, capture elements such as namespaces, helm releases, configuration maps, and secrets.
• Special cases: how a KServe Inference Service is represented via CRDs or how external user access (via Swagger UI for Prediction Service or UI endpoints for Airflow/MLFlow) is routed through a common ingress (NGINX).
• Infrastructure flows that detail how external tools (kubectl, Docker, GitHub) interact with this system.
Using these guidelines, a principal software engineer can produce an accurate representation of the architecture, ensuring that the GitOps process, multi-cluster setups, and all critical deployment components are clearly visualized and understood.