# bazelbuild/bazel-buildfarm architecture

> Architecture diagram of the GitHub repository bazelbuild/bazel-buildfarm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bazelbuild/bazel-buildfarm
- Repository: https://github.com/bazelbuild/bazel-buildfarm

## Overview

##### Main Components
- Client Interface: Represent the Bazel (or similar) client which sends remote execution and caching requests. The README even provides an example of how to configure the bazelrc for remote execution.
- Buildfarm Server: This is the central orchestration point that processes build requests. The server handles scheduling, execution policies, action caches, and integration with logging/metrics. Its code is primarily found under directories like src/main/java/build/buildfarm/server, and common modules (e.g., configuration, logging, metrics) are scattered throughout.
- Buildfarm Worker (Shard Worker): These are the execution nodes that perform the build actions. The separate worker executable (and the persistent workers in “persistentworkers” and “worker/shard”) are represented as independent components. They are invoked with their own configuration and logging properties.
- Content Addressable Storage (CAS): This component deals with storing build artifacts. Look at the folders under src/main/java/build/buildfarm/cas and its CFC submodules that implement the CAS functionality.
- Redis Cache/State Service: The README provides instructions for running a Redis container and configuring it. This external dependency is crucial for managing ephemeral build states and queues.
- External Dependencies & Integration Points: These include third-party components (gRPC, protobuf, and others managed via rules_jvm_external), deployment artifacts (Dockerfiles, Helm charts in the kubernetes directory), and the Bazel build setup (WORKSPACE, MODULE.bazel).

##### Relationships and Interactions
- It forwards build actions to the Buildfarm Workers.
- It manages the action cache and controls build execution through the CAS.
- It connects to Redis (and possibly other external backing services) to retrieve or update state and caching information.
- Communication between the Server and its subcomponents for configuration, logging (using Java’s Logging framework), and metrics reporting (including Prometheus and log-based metrics).

##### Architectural Patterns and Design Principles
- Separation of Concerns: Clearly separate the orchestration layer (server) from the execution layer (worker). Each component has a distinct responsibility.
- Modularization: The file tree shows modular division using Bazel BUILD files and multiple packages (cas, backplane, common, worker, instance, etc.). Indicate this modular structure to clarify dependencies and build targets.
- Extensibility via Configuration: The project uses YAML and configuration files for server and worker parameters. Emphasize these aspects to show how the system is designed for flexible deployment and operational tuning.
- Use of gRPC and Remote APIs: Indicate gRPC interfaces for communication between the client, server, and workers as highlighted in the README and file structure.

##### Detail the Essential Architectural Elements
- Integration of third-party dependencies (like gRPC, protobuf, and external repository dependencies) which are managed via the Bazel WORKSPACE file and MODULE.bazel.
- Both the build and deployment aspects—highlight how the build targets (as seen in “BUILD” and Bazel files) translate into the production server, worker, and container images.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client Layer"
        Client["Bazel Client"]:::client
    end

    %% Server Layer
    subgraph "Server Layer"
        Server["Buildfarm Server"]:::server
        Persistence["Persistent Worker Management"]:::worker
        Proxy["Proxy Services"]:::proxy
    end

    %% Execution Layer
    subgraph "Execution Layer"
        Worker["Buildfarm Worker (Shard Worker)"]:::worker
    end

    %% Storage & Caching
    subgraph "Storage & Caching"
        CAS["Content Addressable Storage (CAS)"]:::storage
        Redis["Redis Cache/State Service"]:::cache
    end

    %% External & Build Setup
    subgraph "External & Build Setup"
        External["External Dependencies & Build Setup"]:::external
    end

    %% Connections
    Client -->|"gRPCRemoteExecutionAPI"| Server
    Server -->|"TaskDispatch"| Worker
    Worker -->|"ReportingMetrics"| Server
    Server -->|"CASIntegration"| CAS
    Server -->|"StateManagement"| Redis
    Server -->|"PersistentConfig"| Persistence
    Persistence -->|"WorkerManagement"| Worker
    Server -->|"HTTPProxyCalls"| Proxy
    External -->|"BuildConfiguration"| Server

    %% Click Events
    click Server "https://github.com/bazelbuild/bazel-buildfarm/blob/main/src/main/java/build/buildfarm/server/BuildFarmServer.java"
    click Worker "https://github.com/bazelbuild/bazel-buildfarm/tree/main/src/main/java/build/buildfarm/worker/"
    click CAS "https://github.com/bazelbuild/bazel-buildfarm/tree/main/src/main/java/build/buildfarm/cas/"
    click Persistence "https://github.com/bazelbuild/bazel-buildfarm/tree/main/persistentworkers/"
    click Proxy "https://github.com/bazelbuild/bazel-buildfarm/tree/main/src/main/java/build/buildfarm/proxy/http/"
    click Redis "https://github.com/bazelbuild/bazel-buildfarm/tree/main/src/main/java/build/buildfarm/common/redis/"
    click External "https://github.com/bazelbuild/bazel-buildfarm/tree/main/WORKSPACE"

    %% Styles
    classDef client fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef server fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef worker fill:#FFFFCC,stroke:#000,stroke-width:2px;
    classDef storage fill:#FFC0CB,stroke:#000,stroke-width:2px;
    classDef cache fill:#FFDAB9,stroke:#000,stroke-width:2px;
    classDef proxy fill:#FFA500,stroke:#000,stroke-width:2px;
    classDef external fill:#D3D3D3,stroke:#000,stroke-width:2px;
```
