# grafana/tempo architecture

> Architecture diagram of the GitHub repository grafana/tempo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/grafana/tempo
- Repository: https://github.com/grafana/tempo

## Overview

##### Project type and purpose
- This is Grafana Tempo, an open-source, distributed tracing backend.
- The `tempodb` directory is Tempo’s pluggable storage engine: it handles write-ahead logging, block encoding/decoding, compaction, indexing, and interfaces to object stores (S3, GCS, Azure, local FS).

##### `tempodb` file structure
Top-level under `tempodb/`:
- backend/ – implementations of object storage backends (azure, gcs, s3, local)
- encoding/ – block encoding schemas (common, v2, vparquet2/3/4)
- wal/ – write-ahead log for writes before blocks are built
- blocklist, blockselector – choose which blocks to query or compact
- compactor.go, retention.go – block merging and data retention
- tenant index – maps tenants → block metadata for fast lookups
- pool/ – resource pooling for concurrent operations
Configuration and tests accompany each sub-package.

##### README insights
- Tempo writes incoming traces through buffering (WAL) → encoding into Parquet blocks → stores blocks in object storage.
- Queries use a tenant index + block selector to find relevant blocks, retrieve & decode them, then reconstruct trace data.

##### Be as detailed as possible
- WAL → Encoder → Storage Adapter chain step by step
- Compactor polling object store, selecting blocks, merging, writing back
- How the Tenant Index is updated on writes/compactions and read on queries
- Role of the Blocklist Poller that keeps track of available blocks in each tenant

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Entry Point
    Entry["Tempodb Engine"]:::support
    click Entry "https://github.com/grafana/tempo/blob/main/tempodb/tempodb.go"

    %% Write Path
    subgraph "Write Path"
        direction TB
        WALWriter["WAL Writer"]:::write
        click WALWriter "https://github.com/grafana/tempo/blob/main/tempodb/wal/wal.go"
        EncoderEngine["Encoding Engine"]:::write
        BlockStoreAdapter["Block Store Adapter"]:::write
        click BlockStoreAdapter "https://github.com/grafana/tempo/blob/main/tempodb/backend/backend.go"
    end

    %% Encoding Engine sub-components
    subgraph "Encoders"
        direction TB
        EncoderCommon["Common Core"]:::write
        click EncoderCommon "https://github.com/grafana/tempo/tree/main/tempodb/encoding/common"
        EncV2["Parquet v2"]:::write
        click EncV2 "https://github.com/grafana/tempo/tree/main/tempodb/encoding/v2"
        EncPV2["vParquet2"]:::write
        click EncPV2 "https://github.com/grafana/tempo/tree/main/tempodb/encoding/vparquet2"
        EncPV3["vParquet3"]:::write
        click EncPV3 "https://github.com/grafana/tempo/tree/main/tempodb/encoding/vparquet3"
        EncPV4["vParquet4"]:::write
        click EncPV4 "https://github.com/grafana/tempo/tree/main/tempodb/encoding/vparquet4"
    end

    %% Storage Backends
    subgraph "Storage Backends"
        direction TB
        AzureStorage[(Azure Blob Storage)]:::ext
        click AzureStorage "https://github.com/grafana/tempo/blob/main/tempodb/backend/azure/azure.go"
        GCSStorage[(GCS Storage)]:::ext
        click GCSStorage "https://github.com/grafana/tempo/blob/main/tempodb/backend/gcs/gcs.go"
        S3Storage[(S3 Storage)]:::ext
        click S3Storage "https://github.com/grafana/tempo/blob/main/tempodb/backend/s3/s3.go"
        LocalStorage[(Local FS)]:::ext
        click LocalStorage "https://github.com/grafana/tempo/blob/main/tempodb/backend/local/local.go"
    end

    %% Compaction & Retention
    subgraph "Compaction & Retention"
        direction TB
        Compactor["Compactor Service"]:::comp
        click Compactor "https://github.com/grafana/tempo/blob/main/tempodb/compactor.go"
        Retention["Retention Manager"]:::comp
        click Retention "https://github.com/grafana/tempo/blob/main/tempodb/retention.go"
    end

    %% Read Path
    subgraph "Read Path"
        direction TB
        QueryReq["Query Request"]:::support
        TenantIndex[(Tenant Index)]:::read
        click TenantIndex "https://github.com/grafana/tempo/blob/main/tempodb/backend/tenantindex.go"
        BlockSelector["Block Selector"]:::read
        click BlockSelector "https://github.com/grafana/tempo/blob/main/tempodb/blockselector/compaction_block_selector.go"
        StorageAdapterRead["Storage Adapter"]:::read
        Decoder["Decoder"]:::read
        QueryResponse["Query Response"]:::read
    end

    %% Supporting Services
    subgraph "Supporting Services"
        direction TB
        Blocklist["Blocklist Poller"]:::support
        click Blocklist "https://github.com/grafana/tempo/blob/main/tempodb/blocklist/poller.go"
        Pool["Pool Manager"]:::support
        click Pool "https://github.com/grafana/tempo/blob/main/tempodb/pool/pool.go"
    end

    %% Connections Write Path
    Entry --> WALWriter
    WALWriter --> EncoderEngine
    EncoderEngine --> BlockStoreAdapter
    EncoderEngine --> EncoderCommon
    EncoderCommon --> BlockStoreAdapter
    EncoderEngine --> EncV2 --> BlockStoreAdapter
    EncoderEngine --> EncPV2 --> BlockStoreAdapter
    EncoderEngine --> EncPV3 --> BlockStoreAdapter
    EncoderEngine --> EncPV4 --> BlockStoreAdapter
    BlockStoreAdapter --> AzureStorage
    BlockStoreAdapter --> GCSStorage
    BlockStoreAdapter --> S3Storage
    BlockStoreAdapter --> LocalStorage

    %% Compaction Flow
    AzureStorage --> Compactor
    GCSStorage --> Compactor
    S3Storage --> Compactor
    LocalStorage --> Compactor
    Compactor --> BlockStoreAdapter
    Compactor --> TenantIndex

    %% Retention Flow
    Retention --> BlockStoreAdapter
    Retention --> TenantIndex

    %% Read Flow
    QueryReq --> TenantIndex --> BlockSelector --> StorageAdapterRead --> Decoder --> QueryResponse
    StorageAdapterRead --> AzureStorage
    StorageAdapterRead --> GCSStorage
    StorageAdapterRead --> S3Storage
    StorageAdapterRead --> LocalStorage

    %% Supporting Services Connections
    Blocklist --> TenantIndex
    Blocklist --> BlockStoreAdapter
    Pool --> WALWriter
    Pool --> EncoderEngine
    Pool --> Compactor
    Pool --> StorageAdapterRead

    %% Styles
    classDef write fill:#ADD8E6,stroke:#333,stroke-width:1px
    classDef read fill:#D8BFD8,stroke:#333,stroke-width:1px
    classDef comp fill:#FFA500,stroke:#333,stroke-width:1px
    classDef ext fill:#90EE90,stroke:#333,stroke-width:1px
    classDef support fill:#D3D3D3,stroke:#333,stroke-width:1px
```
