# datum-cloud/infra-provider-gcp architecture

> Architecture diagram of the GitHub repository datum-cloud/infra-provider-gcp, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/datum-cloud/infra-provider-gcp
- Repository: https://github.com/datum-cloud/infra-provider-gcp

## Overview

##### Project Type and Purpose
- This is a Kubernetes Operator (built with Kubebuilder) written in Go that serves as a GCP Infrastructure Provider.
- It watches custom API types (from the workload-operator and network-services-operator), then uses the Google Config Connector to create and manage GCP resources (Compute instances/MIGs, networking, IAM, etc.).

##### Top-Level File/Directory Analysis
- cmd/main.go: operator’s entrypoint, sets up the controller-runtime Manager.
- internal/controller/…: one reconcile controller per API type (InstanceDiscovery, WorkloadDeployment, WorkloadGateway, NetworkContext, CloudInit, K8sConfigConnector).
- config/: kustomize overlays and CRD/Role/ServiceAccount manifests for deploying the operator & Config Connector CRDs.
- test/e2e/: end-to-end tests using a real or fake cluster.
- hack/, Makefile, Dockerfile: build, lint, test, and containerization scripts.

##### Key Technologies & Patterns
- Go + controller-runtime (Kubebuilder): manager, schemes, reconcile loops.
- Kubernetes API server as control plane; watches CRs.
- Kustomize for YAML overlays (config/…).
- GCP Config Connector CRDs as “adapter” to GCP APIs.
- End-to-end testing harness (test/e2e).

##### Main Components
a. Kubernetes Cluster
- API Server / etcd (central control plane)
- Config Connector controllers (managing GCP CRDs)
- Datum GCP Infrastructure Provider Pod (the operator manager + controllers)
- CRDs & Custom Resources:
– Workload CRs (from workload-operator)
– Network CRs (from network-services-operator)
– Config Connector CRDs (ComputeInstance, MIG, Firewall, IAM bindings, etc.)
b. External Systems
- GCP Control Plane (Compute Engine, Networking, IAM, etc.)
- Container registry (for IMG)
c. Developer Workstation / CI
- kubectl, make, docker build/push, GitHub Actions workflows

##### Component Interactions & Data Flow
1. Developer pushes code → GitHub Actions builds + tests → Docker image → Registry.
2. Operator Pod (using that image) is deployed via Kustomize (Makefile).
3. Operator’s Manager starts Controllers: each watches its target CR type.
4. When a Workload or Network custom resource is created/updated:
- Reconcile loop runs in the appropriate controller.
- Controller uses the Kubernetes client to create/update Config Connector CRs in the cluster.
5. Config Connector controllers see the new CRs → call the GCP REST API → provision or modify GCP resources.
6. Controller reads status from Config Connector CRs, updates status on the original custom resources.

##### Architectural Patterns & Principles
- Controller Pattern: each resource type has a dedicated reconcile loop.
- Declarative Infrastructure: uses Kubernetes CRs as the source of truth.
- Separation of Concerns:
- Datum controllers only bridge custom API → Config Connector CRs;
- Config Connector controllers handle GCP API specifics.
- Config-as-Code: all deployment manifests are managed via Kustomize.

##### Essential Elements Checklist
- Components: API Server, etcd, Datum Operator Pod (Manager & Controllers), Config Connector controllers, GCP.
- Labels: each controller, each CRD group, external systems.
- Deployment artifacts: mention Kustomize overlay, Docker image.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Developer/CI"
        Dev["Developer Workstation / CI"]:::ci
        Actions["GitHub Actions"]:::ci
        Dev -->|push code| Actions
        Actions -->|build & test| Image["Docker Image"]:::artifact
        click Actions "https://github.com/datum-cloud/infra-provider-gcp/blob/main/.github/workflows/test.yml"
        click Actions "https://github.com/datum-cloud/infra-provider-gcp/blob/main/.github/workflows/lint.yml"
        click Actions "https://github.com/datum-cloud/infra-provider-gcp/blob/main/.github/workflows/test-e2e.yml"
        click Actions "https://github.com/datum-cloud/infra-provider-gcp/blob/main/.github/workflows/publish.yaml"
        click Image "https://github.com/datum-cloud/infra-provider-gcp/tree/main/Dockerfile"
    end

    Image -->|push| Registry["Container Registry"]:::external

    Registry -->|pull image| Deploy["Kustomize Deploy"]:::infra
    click Deploy "https://github.com/datum-cloud/infra-provider-gcp/blob/main/config/default/kustomization.yaml"

    Deploy -->|apply manifests| API_Server

    subgraph "Kubernetes Cluster"
        subgraph "Control Plane"
            API_Server["API Server"]:::control
            ETCD["etcd"]:::control
        end

        subgraph "Datum Operator Pod"
            OPPod["Operator Entrypoint\n(cmd/main.go)"]:::datum
            click OPPod "https://github.com/datum-cloud/infra-provider-gcp/blob/main/cmd/main.go"

            subgraph "Controllers"
                ID["InstanceDiscovery Controller"]:::datum
                click ID "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/instancediscovery_controller.go"
                WD["WorkloadDeployment Controller"]:::datum
                click WD "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/workloaddeployment_controller.go"
                WG["WorkloadGateway Controller"]:::datum
                click WG "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/workloadgateway_controller.go"
                NC["NetworkContext Controller"]:::datum
                click NC "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/networkcontext_controller.go"
                CI["CloudInit Controller"]:::datum
                click CI "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/cloudinit/cloudconfig.go"
                SP["Secret Populator"]:::datum
                click SP "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/cloudinit/populate_secrets.py"
                K8C["K8s ConfigConnector Controller"]:::datum
                click K8C "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/k8sconfigconnector/conditions.go"
                SCU["Shared Controller Utilities"]:::datum
                click SCU "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/annotations.go"
                click SCU "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/controller/result.go"
                XCU["Cross-Cluster Utilities"]:::datum
                click XCU "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/crossclusterutil/controllerutil.go"
                click XCU "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/crossclusterutil/enqueue_upstream_owner.go"
                click XCU "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/crossclusterutil/labels.go"
                click XCU "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/crossclusterutil/namespace.go"
                LU["Location Utilities"]:::datum
                click LU "https://github.com/datum-cloud/infra-provider-gcp/blob/main/internal/locationutil/location.go"
            end
        end

        subgraph "CRDs & Resources"
            DatumCR["Datum Custom Resources\n(Workload/Network)"]:::custom
            CCCRD["Config Connector CRDs"]:::config
            click CCCRD "https://github.com/datum-cloud/infra-provider-gcp/blob/main/config/default/kustomization.yaml"
        end

        subgraph "Config Connector"
            CCCtrl["Config Connector Controllers\n(provision GCP)"]:::config
        end

        API_Server --> ETCD
        Deploy --> API_Server
        API_Server --> OPPod
        OPPod --> ID
        OPPod --> WD
        OPPod --> WG
        OPPod --> NC
        OPPod --> CI
        OPPod --> K8C

        Controllers -->|watch/create/update| DatumCR
        DatumCR -->|create/update| API_Server
        Controllers -->|create/update| CCCRD
        CCCRD -->|watch| CCCtrl
        CCCtrl -->|REST call to GCP| GCP
        GCP -->|status update| CCCtrl
        CCCtrl -->|update status| CCCRD
        CCCRD -->|status| Controllers
    end

    GCP["GCP Control Plane"]:::external

    classDef control fill:#e0f7fa,stroke:#006064,color:#006064;
    classDef datum fill:#e8f5e9,stroke:#2e7d32,color:#2e7d32;
    classDef config fill:#fff3e0,stroke:#ef6c00,color:#ef6c00;
    classDef external fill:#eceff1,stroke:#607d8b,color:#455a64;
    classDef ci fill:#f3e5f5,stroke:#6a1b9a,color:#4a148c;
    classDef artifact fill:#c5cae9,stroke:#303f9f,color:#1a237e;
    classDef infra fill:#e1f5fe,stroke:#0288d1,color:#01579b;
    classDef custom fill:#f1f8e9,stroke:#558b2f,color:#558b2f;
```
