# rancher/ecm-distro-tools architecture

> Architecture diagram of the GitHub repository rancher/ecm-distro-tools, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rancher/ecm-distro-tools
- Repository: https://github.com/rancher/ecm-distro-tools

## Overview

##### Main System Components
A. Source Code
1. Go Commands (cmd/*) – each subfolder under cmd (e.g., cmd/release, cmd/rke2_release, cmd/backport, etc.) is a standalone CLI tool that links against shared Go libraries.
2. Shell Scripts (bin/) – POSIX-compliant scripts and shell libraries that wrap or orchestrate other utilities.
3. Shared Libraries (pkg-style folders) – modules under http/, docker/, registry/, repository/, types/, exec/, mgmt/ provide reusable functionality (API clients, logging, reprository abstractions).
B. Build & Packaging
1. Makefile & cmd/*/Makefile – local build orchestration, multi-arch builds.
2. Dockerfile & .devcontainer – containerized build/test/development environment.
3. GitHub Actions workflows (.github/workflows/main.yml, release.yml) – CI/CD for building binaries, publishing container images, auto-releasing to GitHub.
4. action.yml – definition of the “Setup ecm-distro-tools” GitHub Action.
C. Installation & Distribution
1. install.sh – bootstrap script to fetch and install binaries + shell libs to /usr/local/bin/ecm-distro-tools.
2. GitHub Releases & Container Registry – artifacts published as GitHub release assets and Docker images (rancher/ecm-distro-tools).
D. QA & Infra
1. qa/aws-ec2-mgr & qa/infra – scripts and AMI definitions for spinning up EC2 instances for testing.
2. qa/workload-manifests – Kubernetes YAML to validate Rancher/K3s/RKE2 clusters.
E. Documentation & Reports
1. docs/ – markdown docs describing release processes, dependencies, testing.
2. reports/ – generated release notes and markdown summaries.

##### Relationships & Data Flows
- Developers commit code → GitHub triggers CI (main.yml):
- Build Go binaries (Makefile → go build)
- Build Docker image
- Run tests against http/, registry/, repository/ libraries
- On tagged release → release.yml:
- Publish binaries to GitHub Releases
- Push Docker image to container registry
- install.sh (end user) → Downloads from GitHub Releases → Installs binaries + shell libs (+ updates PATH)
- CLI tools & shell scripts → call into shared Go libs (for API calls to GitHub, Docker registry, Rancher APIs, Kubernetes clusters)
- QA scripts → provision EC2 instances → deploy workload-manifests → validate cluster using CLI tools

##### Architectural Patterns & Principles
- Modular CLI Architecture – each cmd/* folder is an independent command, minimizing cross-dependencies.
- Shared Library Layer – generic API clients and utilities centralized in http/, docker/, registry/, repository/, types/
- Polyglot Mono-repo – co-existence of Go binaries and POSIX shell scripts, with clearly separated directories.
- CI/CD as Code – GitHub Actions workflows define build & release pipelines.
- Infrastructure as Code – QA infra definitions (AMI JSON, Kubernetes YAML) for automated testing.

##### Relevant Technologies & Frameworks
- Go (modules, standard library)
- POSIX Shell + shellcheck for scripts
- Make & Docker for builds
- GitHub Actions for CI/CD
- GitHub API (via shared HTTP library)
- Docker Registry (via registry/ library)
- AWS EC2 (QA infra)
- Kubernetes manifests for cluster validation

##### Essential Elements
- Clear labels for each component (e.g., “cmd/release (Go CLI)”, “bin/libstd-ecm.sh (Shell library)”)
- code → CI/CD → artifacts → release channels → install
- runtime flows (CLI → shared libs → external APIs)
- QA provisioning and testing flows

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Source Code Layer
    subgraph "Source Code" 
        direction TB
        cmd_backport["cmd/backport (Go CLI)"]:::code
        cmd_gen_report["cmd/gen_release_report (Go CLI)"]:::code
        cmd_rancher["cmd/rancher_release (Go CLI)"]:::code
        cmd_release["cmd/release (Go CLI)"]:::code
        cmd_rke2["cmd/rke2_release (Go CLI)"]:::code
        cmd_semv["cmd/semv (Go CLI)"]:::code
        cmd_testcov["cmd/test_coverage (Go CLI)"]:::code
        cmd_upstream["cmd/upstream_go_version (Go CLI)"]:::code
        bin_scripts["bin/ (Shell scripts & libs)"]:::code

        subgraph "Shared Libraries"
            direction TB
            lib_http["http/ (Go HTTP library)"]:::code
            lib_docker["docker/ (Go Docker library)"]:::code
            lib_registry["registry/ (Go Registry library)"]:::code
            lib_repo["repository/ (Go Repository library)"]:::code
            lib_types["types/ (Go Types library)"]:::code
            lib_exec["exec/ (Go Exec utilities)"]:::code
            lib_mgmt["mgmt/ (Go Management utilities)"]:::code
        end
    end

    %% Build & Release Layer
    subgraph "Build & CI/CD" 
        direction TB
        make_root["Makefile"]:::pipeline
        dockerfile["Dockerfile"]:::pipeline
        devcontainer[".devcontainer/"]:::pipeline
        ci_build[".github/workflows/main.yml"]:::pipeline
        ci_release[".github/workflows/release.yml"]:::pipeline
        action_def["action.yml"]:::pipeline
    end

    %% Distribution Layer
    subgraph "Distribution" 
        direction TB
        install["install.sh"]:::pipeline
        gh_releases["GitHub Releases"]:::external
        docker_registry["Docker Registry"]:::external
    end

    %% Runtime Layer
    subgraph "Runtime (End User)" 
        direction TB
        cli_tools["CLI Tools & Shell Scripts"]:::code
        shared_libs["Shared Libraries"]:::code
    end

    %% QA & Infra Layer
    subgraph "QA & Infra" 
        direction TB
        qa_ec2["qa/aws-ec2-mgr/"]:::pipeline
        qa_infra["qa/infra/amis/"]:::pipeline
        qa_workloads["qa/workload-manifests/"]:::pipeline
    end

    %% External Services
    subgraph "External Services" 
        direction TB
        github_api["GitHub API"]:::external
        docker_api["Docker Registry API"]:::external
        aws_ec2["AWS EC2"]:::external
        k8s_api["Kubernetes API"]:::external
    end

    %% Relationships
    developers["Developers (git commit)"]:::external -->|push code| ci_build
    ci_build -->|run Makefile| make_root
    ci_build -->|build Docker image| dockerfile
    ci_build -->|use devcontainer| devcontainer
    ci_build -->|run tests| lib_http
    ci_build -->|run tests| lib_registry

    ci_release -->|publish binaries| gh_releases
    ci_release -->|push image| docker_registry

    install -->|download artifacts| gh_releases
    install -->|download images| docker_registry

    cli_tools -->|call| shared_libs
    shared_libs -->|API calls| github_api
    shared_libs -->|API calls| docker_api
    shared_libs -->|API calls| aws_ec2
    shared_libs -->|API calls| k8s_api

    qa_ec2 -->|provision| aws_ec2
    qa_ec2 -->|use AMIs| qa_infra
    qa_workloads -->|deploy workloads| k8s_api
    qa_workloads -->|validate with| cli_tools

    %% Click Events
    click cmd_backport "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/backport/"
    click cmd_gen_report "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/gen_release_report/"
    click cmd_rancher "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/rancher_release/"
    click cmd_release "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/release/"
    click cmd_rke2 "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/rke2_release/"
    click cmd_semv "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/semv/"
    click cmd_testcov "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/test_coverage/"
    click cmd_upstream "https://github.com/rancher/ecm-distro-tools/tree/master/cmd/upstream_go_version/"
    click lib_http "https://github.com/rancher/ecm-distro-tools/tree/master/http/"
    click lib_docker "https://github.com/rancher/ecm-distro-tools/tree/master/docker/"
    click lib_registry "https://github.com/rancher/ecm-distro-tools/tree/master/registry/"
    click lib_repo "https://github.com/rancher/ecm-distro-tools/tree/master/repository/"
    click lib_types "https://github.com/rancher/ecm-distro-tools/tree/master/types/"
    click lib_exec "https://github.com/rancher/ecm-distro-tools/tree/master/exec/"
    click lib_mgmt "https://github.com/rancher/ecm-distro-tools/tree/master/mgmt/"
    click bin_scripts "https://github.com/rancher/ecm-distro-tools/tree/master/bin/"
    click make_root "https://github.com/rancher/ecm-distro-tools/tree/master/Makefile"
    click dockerfile "https://github.com/rancher/ecm-distro-tools/tree/master/Dockerfile"
    click devcontainer "https://github.com/rancher/ecm-distro-tools/tree/master/.devcontainer/"
    click ci_build "https://github.com/rancher/ecm-distro-tools/blob/master/.github/workflows/main.yml"
    click ci_release "https://github.com/rancher/ecm-distro-tools/blob/master/.github/workflows/release.yml"
    click action_def "https://github.com/rancher/ecm-distro-tools/blob/master/action.yml"
    click install "https://github.com/rancher/ecm-distro-tools/blob/master/install.sh"
    click qa_ec2 "https://github.com/rancher/ecm-distro-tools/tree/master/qa/aws-ec2-mgr/"
    click qa_infra "https://github.com/rancher/ecm-distro-tools/tree/master/qa/infra/amis/"
    click qa_workloads "https://github.com/rancher/ecm-distro-tools/tree/master/qa/workload-manifests/"

    %% Styles
    classDef code fill:#cfe2f3,stroke:#036; 
    classDef pipeline fill:#d9ead3,stroke:#063; 
    classDef external fill:#fce5cd,stroke:#a65;
```
