# buildbarn/bb-remote-execution architecture

> Architecture diagram of the GitHub repository buildbarn/bb-remote-execution, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/buildbarn/bb-remote-execution
- Repository: https://github.com/buildbarn/bb-remote-execution

## Overview

##### Main Components

a. Core Services:
- bb_scheduler: Central scheduling service
- bb_worker: Build action orchestrator
- bb_runner: Command execution service
- bb_storage (external dependency): Storage daemon

b. Supporting Components:
- Build Queue
- CAS (Content Addressable Storage)
- File System Handlers
- Authentication/Authorization Layer
- Protocol Buffers Interface Layer

##### Component Relationships

**Client Tools (Bazel/BuildStream/recc)**
- ↓
**[bb_storage] ←→ [bb_scheduler]**
- ↓
- [bb_worker] ←→ [CAS/Blob Storage]
- ↓
- [bb_runner]
- ↓
- [Build Environment]

##### Key Architectural Elements

a. Service Boundaries:
- Clear separation between privileged and unprivileged components
- Network boundaries between components

b. Data Flow:
- Build request flow from client to execution
- File/artifact movement between storage and workers
- Results propagation back to clients

##### Important Subsystems to Highlight

a. Worker Components:
- Input file caching system
- Output file handling
- Build action orchestration
- Resource management

b. Runner Components:
- Execution environment isolation
- Command execution
- Resource constraints
- File system access

##### Integration Points

- GRPC interfaces between components
- Filesystem mounting points
- Authentication boundaries
- Monitoring/metrics collection points

##### Specific Implementation Details

- Volume mounting points for bb_runner_installer
- Relationship with external container images
- Points where caching occurs

##### Notes

Annotations for:
- Scalability points
- Security considerations
- Performance optimization points
- Critical paths

Remember to highlight the pluggable nature of the runner component and the ability to use different execution environments through container images.

This system design should clearly communicate both the high-level architecture and the important implementation details that make this remote execution system robust and secure.

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Client["Client Layer"]
        C1["Bazel"]
        C2["BuildStream"]
        C3["recc"]
    end

    subgraph Storage["Storage Layer"]
        BS["bb_storage"]
        CAS["Content Addressable Storage"]:::cas
    end

    subgraph Scheduler["Scheduler Layer"]
        SCH["BB Scheduler"]:::scheduler
        BQ["Build Queue"]:::scheduler
    end

    subgraph Worker["Worker Layer"]
        WRK["BB Worker"]:::worker
        FSH["File System Handlers"]:::worker
    end

    subgraph Runner["Runner Layer"]
        RUN["BB Runner"]:::runner
        BE["Build Environment"]:::runner
    end

    subgraph Auth["Security Layer"]
        AUTH["Authentication/Authorization"]:::security
        PROT["Protocol Buffers Interface"]:::security
    end

    C1 & C2 & C3 --> BS
    BS <--> SCH
    BS <--> CAS
    SCH --> BQ
    BQ --> WRK
    WRK <--> CAS
    WRK --> FSH
    WRK --> RUN
    RUN --> BE
    AUTH -.-> SCH & WRK & RUN
    PROT -.-> SCH & WRK & RUN

    classDef scheduler fill:#f9f,stroke:#333
    classDef worker fill:#bbf,stroke:#333
    classDef runner fill:#bfb,stroke:#333
    classDef security fill:#fbb,stroke:#333
    classDef cas fill:#ddd,stroke:#333

    click SCH "https://github.com/buildbarn/bb-remote-execution/tree/master/cmd/bb_scheduler"
    click WRK "https://github.com/buildbarn/bb-remote-execution/tree/master/cmd/bb_worker"
    click RUN "https://github.com/buildbarn/bb-remote-execution/tree/master/cmd/bb_runner"
    click CAS "https://github.com/buildbarn/bb-remote-execution/tree/master/pkg/cas"
    click FSH "https://github.com/buildbarn/bb-remote-execution/tree/master/pkg/filesystem"
    click AUTH "https://github.com/buildbarn/bb-remote-execution/tree/master/pkg/credentials"
    click PROT "https://github.com/buildbarn/bb-remote-execution/tree/master/pkg/proto"
    click BQ "https://github.com/buildbarn/bb-remote-execution/blob/master/pkg/scheduler/in_memory_build_queue.go"

    subgraph Legend["Legend"]
        L1["Scheduler Components"]:::scheduler
        L2["Worker Components"]:::worker
        L3["Runner Components"]:::runner
        L4["Security Components"]:::security
        L5["Storage Components"]:::cas
        L6["--> Data Flow"]
        L7["-.-> Security Boundary"]
    end
```
