# secretflow/kuscia architecture

> Architecture diagram of the GitHub repository secretflow/kuscia, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/secretflow/kuscia
- Repository: https://github.com/secretflow/kuscia

## Overview

##### High-level areas of the system
- Frontend/API Layer: Although not a full-fledged “frontend” in the traditional UI sense, the API components—implemented via the “kusciaapi”, “gateway”, and “transport” modules (with their GRPC/HTTP handlers found in proto definitions and related packages)—act as primary points of interaction.
- Orchestration Core/Controllers: The core “cmd/kuscia” binary and multiple “pkg/controllers” modules implement the orchestration logic. These components handle the lifecycle of secure computing tasks, schedule deployments, manage pods, and update CRD states.
- Agent and Runtime Components: The “pkg/agent” module and its various subpackages (e.g., container management and local/runtime implementations) provide the infrastructure for managing containerized tasks (including resource management, garbage collection, and probing).
- Custom Resource Definitions (CRDs): The “crds” directory clearly defines CRDs (for domains, datamesh, interconnection, tasks, etc.) that extend Kubernetes’ API and enable custom orchestration constructs.
- Configuration and Integration: “pkg/confmanager” and “pkg/datamesh” modules deal with configuration management and data service orchestration. They provide dynamic configuration handling for inter-domain scheduling, data access, and secure interconnection.
- Security & Interconnection: Components under “pkg/interconn”, “pkg/gateway”, and “pkg/kusciaapi” (which include certificate handling, domain route management, and secure handshake protocols) implement secure, multi-domain communication.
- Auxiliary Components: The scheduling (in “pkg/scheduler”), diagnostics (“pkg/diagnose”), logging/monitoring (“pkg/metricexporter” and “pkg/reporter”), and web interfaces (under “pkg/web”) provide supplementary infrastructure for ensuring health, metrics, and system integrity.

##### Relationships and interactions
- APIs (both HTTP and GRPC) expose endpoints for external systems to submit tasks, configurations, and queries. These endpoints communicate with the core orchestrator.
- The orchestrator (controllers) monitors and reconciles desired state using CRDs. It interacts with various agents deployed on each node.
- Agents manage the local container runtime (using container handling, scheduling, monitoring, and logging) and interact with the orchestrator via the CRDs and API calls.
- The configuration manager loads and propagates secure parameters and policies for tasks and clusters.
- Data-related tasks and datamesh components interlink with the interconnection modules to support cross-domain communications and privacy-preserving operations.
- Gateway components, along with proxy and interconnection modules, facilitate secure network communication and manage port reuse, thus ensuring that minimal public networking resources are required.

##### Important architectural patterns
- Modular Microservices: The project is split into multiple modules communicating through well-defined interfaces (HTTP/GRPC).
- Operator Pattern in Kubernetes: Use of CRDs along with controllers reflects the Kubernetes operator design pattern.
- Secure Inter-domain Communication: Security and privacy aspects have been emphasized with certificate management, handshake protocols, and gateway routing.
- Plugin/Extensibility: Middleware components (e.g., under “pkg/agent/middleware/plugins”) indicate that extensibility is baked into the system design for additional functionality (like image security, environment import, and config rendering).

##### Relevant technologies and frameworks
- Kubernetes (K3s) is the underlying orchestration platform.
- GRPC and HTTP for API endpoints.
- Bazel for build and dependency management (evidenced by BUILD.bazel, MODULE.bazel, and other build scripts).
- Go (with test files, extensive pkg structure, and a robust CLI under cmd/) as the language of implementation.
- CI/CD and deployment scripts (using CircleCI, GitHub actions, shell scripts).
- Dockerfiles and Kubernetes deployment templates are used for containerization and cluster deployment.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Client
    EC("External Client"):::external

    %% API Layer Subgraph
    subgraph "Frontend/API Layer"
        API("Kuscia API"):::api
        GW("Gateway"):::api
        TR("Transport"):::api
    end

    %% Orchestration/Controllers Subgraph
    subgraph "Orchestration Controllers"
        ORCH("Orchestrator/Controllers"):::orchestrator
    end

    %% Kubernetes CRDs Subgraph
    subgraph "Kubernetes CRDs"
        CRD("CRDs"):::k8s
    end

    %% Agent and Runtime Subgraph
    subgraph "Agent & Runtime Components"
        AGENT("Agent"):::agent
    end

    %% Configuration and Integration Subgraph
    subgraph "Configuration & Integration"
        CONF("ConfManager"):::config
        DM("Datamesh"):::config
    end

    %% Security & Interconnection Subgraph
    subgraph "Security & Interconnection"
        SEC("Interconnection"):::security
    end

    %% Auxiliary Components Subgraph
    subgraph "Auxiliary Components"
        SCH("Scheduler"):::auxiliary
        DIAG("Diagnostics"):::auxiliary
        MET("Metrics"):::auxiliary
        WEB("Web Interface"):::auxiliary
    end

    %% Data Flow Connections
    EC -->|"HTTP/GRPC"| API
    EC -->|"HTTP/GRPC"| GW
    EC -->|"HTTP/GRPC"| TR

    API -->|"calls"| ORCH
    GW -->|"calls"| ORCH
    TR -->|"calls"| ORCH

    ORCH -->|"updates"| CRD
    ORCH -->|"triggers"| AGENT
    AGENT -->|"reports"| CRD

    CONF -->|"configures"| ORCH
    CONF -->|"configures"| AGENT
    DM -->|"configures"| ORCH
    DM -->|"assists"| SEC

    GW -->|"secure_handshake"| SEC
    API -->|"secure_comm"| SEC

    ORCH -->|"schedules"| SCH
    ORCH -->|"monitors"| DIAG
    ORCH -->|"reports_metrics"| MET
    ORCH -->|"UI_feed"| WEB
    AGENT -->|"status_update"| MET

    %% Styling
    classDef external fill:#e0f7e9,stroke:#2e7d32,stroke-width:2px;
    classDef api fill:#ffecb3,stroke:#f57f17,stroke-width:2px;
    classDef orchestrator fill:#bbdefb,stroke:#0d47a1,stroke-width:2px;
    classDef agent fill:#f8bbd0,stroke:#880e4f,stroke-width:2px;
    classDef k8s fill:#c8e6c9,stroke:#1b5e20,stroke-width:2px;
    classDef config fill:#fff9c4,stroke:#f9a825,stroke-width:2px;
    classDef security fill:#ffcdd2,stroke:#b71c1c,stroke-width:2px;
    classDef auxiliary fill:#d1c4e9,stroke:#4a148c,stroke-width:2px;

    %% Click Events
    click API "https://github.com/secretflow/kuscia/tree/main/pkg/kusciaapi"
    click GW "https://github.com/secretflow/kuscia/tree/main/pkg/gateway"
    click TR "https://github.com/secretflow/kuscia/tree/main/pkg/transport"
    click ORCH "https://github.com/secretflow/kuscia/tree/main/cmd/kuscia"
    click AGENT "https://github.com/secretflow/kuscia/tree/main/pkg/agent"
    click CRD "https://github.com/secretflow/kuscia/tree/main/crds"
    click CONF "https://github.com/secretflow/kuscia/tree/main/pkg/confmanager"
    click DM "https://github.com/secretflow/kuscia/tree/main/pkg/datamesh"
    click SEC "https://github.com/secretflow/kuscia/tree/main/pkg/interconn"
    click SCH "https://github.com/secretflow/kuscia/tree/main/pkg/scheduler"
    click DIAG "https://github.com/secretflow/kuscia/tree/main/pkg/diagnose"
    click MET "https://github.com/secretflow/kuscia/tree/main/pkg/metricexporter"
    click WEB "https://github.com/secretflow/kuscia/tree/main/pkg/web"
```
