# federatedai/kubefate architecture

> Architecture diagram of the GitHub repository federatedai/kubefate, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-02-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/federatedai/kubefate
- Repository: https://github.com/federatedai/kubefate

## Overview

##### Project Type and Purpose
- The project is a deployment and management tool for federated learning environments, specifically supporting FATE. It is built to enable multi-party collaborative machine learning across diverse environments.
- The README emphasizes two primary deployment models: docker-deploy (for quick multi-party setups using Docker Compose) and k8s-deploy (for production-ready Kubernetes cluster deployments). It also integrates continuous integration (CI) pipelines and helm-charts for Kubernetes resources.

##### Analyzing the File Structure
- Look at high-level directories:
- "docker-deploy": Contains Docker Compose templates, environment configuration (*.env), scripts, and templates to deploy FATE on a single host.
- "k8s-deploy": Contains files, configurations, and Go source code for a binary that manages a full cluster lifecycle (deploys, queries, destroys clusters) via RESTful APIs.
- "helm-charts": Multiple subdirectories (such as FATE, FATE-Serving, FedML-Client/Server, UpgradeManager) which provide Helm charts to deploy various FATE components and other related services on Kubernetes.
- "build/ci", ".github/workflows", and others for CI/CD, testing, and pipeline automation.
- Notice the presence of documentation, proposals, Kubernetes resource YAMLs, and scripts. These indicate not only the automation of deployments but also support for continuous integration and enhanced operational procedures.

##### Insights from the README
- The README confirms the split between Docker Compose and Kubernetes deployment models.
- It clarifies that the primary operational focus is on enabling federated learning in different environments.
- It also highlights that the system includes both a CLI and service (written in Go) to handle complete cluster lifecycle managing tasks.

a. Main Components to Include:
– Deployment Modes:
- Docker Deploy: Represent the Docker Compose environment, showing the FATE components (e.g., serving-admin, serving-proxy, serving-server) deployed on a single host.
- Kubernetes Deploy: Represent Kubernetes clusters, Helm charts for FATE, FATE-Serving, FedML components, and the binary (CLI/service) that includes RESTful APIs.
– CI/CD Pipeline:
- CI workflows (Jenkins, GitHub Actions), Ansible automation scripts, and Docker build workflows.
– Core Services:
- API service and CLI (from k8s-deploy) that manage cluster operations.
- Underlying FATE components (e.g., Eggroll backends, Spark integrations) that support the federated learning process.
– External Dependencies:
- Docker registry (e.g., Docker Hub).
- Kubernetes cluster (public/private/hybrid).
- External messaging or load-balancing components (e.g., ingress controllers via Helm charts).

b. Relationships and Interactions:
– Show how Helm charts are used to deploy services on Kubernetes and how Docker Compose orchestrates FATE components on a single host.
– Include interactions between CI/CD pipelines (triggered via GitHub Workflows, Jenkins pipelines) and the codebase/repository to automate build and deployment processes.
– Illustrate data flow from user commands or REST API calls (management console or CLI) to the backend services and cluster resources.

c. Architectural Patterns and Design Principles:
– Separation between deployment options: Clearly demarcate the docker-deploy path versus the k8s-deploy path.
– Modular design: Components such as the CLI/service and Helm charts are self-contained modules that enable extensibility.
– Cloud-native patterns: The overall architecture leverages containerization (Docker), container orchestration (Kubernetes), and GitOps/CI pipelines.

d. Key Technologies, Frameworks, and Libraries:
– Docker and Docker Compose for container orchestration in development.
– Kubernetes for production deployments along with Helm charts for resource templating.
– Golang for the binary that underpins the CLI and service.
– CI/CD tools like Jenkins, GitHub Workflows, and Ansible for automation.
– Supporting configurations like YAML, Helm charts, and Kubernetes manifests.

## Diagram (Mermaid)

```mermaid
flowchart TD
    CI["CI/CD Pipelines and Automation"]:::cicd

    subgraph "Docker Deploy Environment"
        DCE["Docker Compose Environment"]:::deploy
        FS["FATE Serving Components"]:::fate
        FT["FATE Training Components"]:::fate
        DCE -->|"deploys"| FS
        DCE -->|"deploys"| FT
    end

    subgraph "Kubernetes Deploy Environment"
        CLI["CLI/API Cluster Management"]:::service
        HC["Helm Charts for Kubernetes Resources"]:::helm
        K8S["Kubernetes Cluster Deployment"]:::k8s
        CLI -->|"manages"| K8S
        HC -->|"deploys via"| K8S
    end

    CI -->|"triggers"| DCE
    CI -->|"triggers"| CLI
    CI -->|"triggers"| HC

    %% Click Events
    click DCE "https://github.com/federatedai/kubefate/tree/master/docker-deploy"
    click FS "https://github.com/federatedai/kubefate/tree/master/docker-deploy/serving_template/docker-serving"
    click FT "https://github.com/federatedai/kubefate/tree/master/docker-deploy/training_template/backends"
    click CLI "https://github.com/federatedai/kubefate/tree/master/k8s-deploy/pkg/cli"
    click CLI "https://github.com/federatedai/kubefate/blob/master/k8s-deploy/kubefate.go"
    click HC "https://github.com/federatedai/kubefate/tree/master/helm-charts"
    click K8S "https://github.com/federatedai/kubefate/tree/master/k8s-deploy"
    click CI "https://github.com/federatedai/kubefate/tree/master/.github/workflows"
    click CI "https://github.com/federatedai/kubefate/tree/master/build/ci"

    %% Styles
    classDef cicd fill:#f9c74f,stroke:#000,stroke-width:2px,color:#000;
    classDef deploy fill:#90be6d,stroke:#000,stroke-width:2px,color:#000;
    classDef fate fill:#f94144,stroke:#000,stroke-width:2px,color:#fff;
    classDef service fill:#577590,stroke:#000,stroke-width:2px,color:#fff;
    classDef helm fill:#43aa8b,stroke:#000,stroke-width:2px,color:#fff;
    classDef k8s fill:#277da1,stroke:#000,stroke-width:2px,color:#fff;
```
