# antmicro/buildgrid architecture

> Architecture diagram of the GitHub repository antmicro/buildgrid, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/antmicro/buildgrid
- Repository: https://github.com/antmicro/buildgrid

## Overview

BuildGrid is a remote build-execution service implementing the Remote Execution and Remote Workers APIs. A build client submits an action; the execution service hands it to the execution instance and scheduler, which coordinate worker bots and expose operation updates. The client uses CAS to supply inputs and retrieve outputs, while the action cache can reuse prior results. The tree also shows capability discovery, ByteStream, and reference storage. The controller wiring and several service handlers are sampled; other subsystem links reflect documented product scope and are kept conservative where implementation excerpts are unavailable.

Main entry points: [`buildgrid/server/scheduler.py`](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/scheduler.py) (Scheduler), [`buildgrid/server/controller.py`](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/controller.py) (Execution controller), [`buildgrid/server/execution/instance.py`](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/execution/instance.py) (Execution instance), [`buildgrid/server/operations/instance.py`](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/operations/instance.py) (Operations instance)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_execution["Execution and scheduling"]
  node_controller["Execution controller<br/>[controller.py]"]
  node_exec_instance["Execution instance<br/>[instance.py]"]
  node_scheduler["Scheduler<br/>[scheduler.py]"]
  node_ops_instance["Operations instance<br/>[instance.py]"]
end

subgraph group_workers["Worker coordination"]
  node_bots_interface["Bots interface<br/>[instance.py]"]
  node_tenant_manager["Lease execution<br/>[tenantmanager.py]"]
end

subgraph group_data["Content and results"]
  node_cas_instance["CAS instance<br/>[instance.py]"]
  node_cas_storage["CAS storage<br/>[storage_abc.py]"]
  node_action_instance["Action cache<br/>[instance.py]"]
  node_action_s3["S3 action cache<br/>[s3storage.py]"]
  node_reference_storage["Reference cache<br/>[storage.py]"]
end

subgraph group_api["Client-facing APIs"]
  node_exec_service["Execution service<br/>[service.py]"]
  node_ops_service["Operations service<br/>[service.py]"]
  node_bots_service["Bots service<br/>[service.py]"]
  node_cas_service["CAS and ByteStream<br/>[service.py]"]
  node_action_service["Action cache service<br/>[service.py]"]
  node_reference_service["Reference service<br/>[service.py]"]
  node_capability_service["Capabilities service<br/>[service.py]"]
  node_capability_instance["Capabilities instance<br/>[instance.py]"]
end

subgraph group_state["Persistence"]
  node_persistence[("Persistent state<br/>[interface.py]")]
end

node_client(("Build client"))
node_worker(("Worker bot"))
node_external_storage[("External S3 storage")]

node_client -->|"submits actions"| node_exec_service
node_exec_service -->|"executes request"| node_exec_instance
node_controller -->|"constructs"| node_scheduler
node_controller -->|"constructs"| node_exec_instance
node_controller -->|"constructs"| node_bots_interface
node_controller -->|"constructs"| node_ops_instance
node_exec_instance -->|"schedules jobs"| node_scheduler
node_controller -->|"registers instances"| node_exec_service
node_client -->|"queries operations"| node_ops_service
node_ops_service -->|"handles requests"| node_ops_instance
node_ops_instance -->|"tracks jobs"| node_scheduler
node_worker -->|"reports sessions"| node_bots_service
node_bots_service -->|"dispatches requests"| node_bots_interface
node_bots_interface -->|"shares scheduler"| node_scheduler
node_client -->|"transfers blobs"| node_cas_service
node_cas_service -->|"handles requests"| node_cas_instance
node_cas_instance -->|"reads and writes"| node_cas_storage
node_client -->|"queries results"| node_action_service
node_action_service -->|"handles requests"| node_action_instance
node_scheduler -->|"checks cached results"| node_action_instance
node_action_instance -->|"stores results"| node_cas_storage
node_action_s3 -.->|"persists cache keys"| node_external_storage
node_client -->|"uses references"| node_reference_service
node_reference_service -->|"handles requests"| node_reference_storage
node_reference_storage -->|"checks blobs"| node_cas_storage
node_client -->|"discovers capabilities"| node_capability_service
node_capability_service -->|"returns capabilities"| node_capability_instance
node_scheduler -.->|"stores scheduler state"| node_persistence

click node_exec_service "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/execution/service.py"
click node_controller "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/controller.py"
click node_exec_instance "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/execution/instance.py"
click node_scheduler "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/scheduler.py"
click node_ops_service "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/operations/service.py"
click node_ops_instance "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/operations/instance.py"
click node_bots_service "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/bots/service.py"
click node_bots_interface "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/bots/instance.py"
click node_tenant_manager "https://github.com/antmicro/buildgrid/blob/master/buildgrid/bot/tenantmanager.py"
click node_cas_service "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/cas/service.py"
click node_cas_instance "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/cas/instance.py"
click node_cas_storage "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/cas/storage/storage_abc.py"
click node_action_service "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/actioncache/service.py"
click node_action_instance "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/actioncache/instance.py"
click node_action_s3 "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/actioncache/s3storage.py"
click node_reference_service "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/referencestorage/service.py"
click node_reference_storage "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/referencestorage/storage.py"
click node_capability_service "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/capabilities/service.py"
click node_capability_instance "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/capabilities/instance.py"
click node_persistence "https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/persistence/interface.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_controller,node_exec_instance,node_scheduler,node_ops_instance,node_client toneBlue
class node_bots_interface,node_tenant_manager,node_external_storage toneAmber
class node_cas_instance,node_cas_storage,node_action_instance,node_action_s3,node_reference_storage toneMint
class node_exec_service,node_ops_service,node_bots_service,node_cas_service,node_action_service,node_reference_service,node_capability_service,node_capability_instance,node_worker toneRose
class node_persistence toneIndigo
```

## Components

### Execution and scheduling

- [Execution controller](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/controller.py): `buildgrid/server/controller.py`
- [Execution instance](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/execution/instance.py): `buildgrid/server/execution/instance.py`
- [Scheduler](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/scheduler.py): `buildgrid/server/scheduler.py`
- [Operations instance](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/operations/instance.py): `buildgrid/server/operations/instance.py`

### Worker coordination

- [Bots interface](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/bots/instance.py): `buildgrid/server/bots/instance.py`
- [Lease execution](https://github.com/antmicro/buildgrid/blob/master/buildgrid/bot/tenantmanager.py): `buildgrid/bot/tenantmanager.py`

### Content and results

- [CAS instance](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/cas/instance.py): `buildgrid/server/cas/instance.py`
- [CAS storage](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/cas/storage/storage_abc.py): `buildgrid/server/cas/storage/storage_abc.py`
- [Action cache](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/actioncache/instance.py): `buildgrid/server/actioncache/instance.py`
- [S3 action cache](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/actioncache/s3storage.py): `buildgrid/server/actioncache/s3storage.py`
- [Reference cache](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/referencestorage/storage.py): `buildgrid/server/referencestorage/storage.py`

### Client-facing APIs

- [Execution service](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/execution/service.py): `buildgrid/server/execution/service.py`
- [Operations service](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/operations/service.py): `buildgrid/server/operations/service.py`
- [Bots service](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/bots/service.py): `buildgrid/server/bots/service.py`
- [CAS and ByteStream](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/cas/service.py): `buildgrid/server/cas/service.py`
- [Action cache service](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/actioncache/service.py): `buildgrid/server/actioncache/service.py`
- [Reference service](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/referencestorage/service.py): `buildgrid/server/referencestorage/service.py`
- [Capabilities service](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/capabilities/service.py): `buildgrid/server/capabilities/service.py`
- [Capabilities instance](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/capabilities/instance.py): `buildgrid/server/capabilities/instance.py`

### Persistence

- [Persistent state](https://github.com/antmicro/buildgrid/blob/master/buildgrid/server/persistence/interface.py): `buildgrid/server/persistence/interface.py`

### Other components

- **Build client**
- **Worker bot**
- **External S3 storage**

## Connections

- Build client → Execution service: submits actions
- Execution service → Execution instance: executes request
- Execution controller → Scheduler: constructs
- Execution controller → Execution instance: constructs
- Execution controller → Bots interface: constructs
- Execution controller → Operations instance: constructs
- Execution instance → Scheduler: schedules jobs
- Execution controller → Execution service: registers instances
- Build client → Operations service: queries operations
- Operations service → Operations instance: handles requests
- Operations instance → Scheduler: tracks jobs
- Worker bot → Bots service: reports sessions
- Bots service → Bots interface: dispatches requests
- Bots interface → Scheduler: shares scheduler
- Build client → CAS and ByteStream: transfers blobs
- CAS and ByteStream → CAS instance: handles requests
- CAS instance → CAS storage: reads and writes
- Build client → Action cache service: queries results
- Action cache service → Action cache: handles requests
- Scheduler → Action cache: checks cached results
- Action cache → CAS storage: stores results
- S3 action cache → External S3 storage: persists cache keys
- Build client → Reference service: uses references
- Reference service → Reference cache: handles requests
- Reference cache → CAS storage: checks blobs
- Build client → Capabilities service: discovers capabilities
- Capabilities service → Capabilities instance: returns capabilities
- Scheduler → Persistent state: stores scheduler state