# cortexproject/cortex architecture

> Architecture diagram of the GitHub repository cortexproject/cortex, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cortexproject/cortex
- Repository: https://github.com/cortexproject/cortex

## Overview

##### Project Type and Purpose
- Cortex is a cloud-native, distributed, full-backend system written in Go. It provides horizontally scalable, highly available, multi-tenant, long-term storage and querying for Prometheus and OpenTelemetry metrics.
- Key indicators: a “cmd/” directory with multiple microservice binaries (cortex, alertmanager, query-tee, test-exporter, thanosconvert), a rich “pkg/” layer broken into service domains (distributor, ingester, querier, ruler, compactor, storegateway, etc.), and a README describing Prometheus remote_write ingestion, object-store backends (S3/GCS/Swift/Azure), and a ring-based sharding/gossip design.

##### File-Structure Analysis
- Top-level “cmd/” houses the entrypoints for each service component.
- “pkg/” decomposes the core services and libraries:
- ingestion path: distributor → ingester → chunk or block storage (pkg/storage, pkg/chunk)
- query path: query-frontend → querier → store-gateway → object store
- background / operational services: compactor, ruler, alertmanager, scheduler
- service-discovery & sharding: pkg/ring, pkg/ha (gossip ring), pkg/distributed_execution
- External-facing API in pkg/api for health-checks and query endpoints.
- Configuration / runtime in pkg/configs, pkg/cortex, plus Dockerfiles, Makefiles, GitHub workflows, integration tests, docs under “docs/architecture.md”.

##### Key Architectural Insights from README
- Multi-tenant isolation, shuffle sharding, consistent hashing ring, gossip or KV (Consul/Etcd/DynamoDB) for membership and ring coordination.
- Object-store backed chunk and new blocks storage.
- Separate write path (remote_write → distributor → ingester → storage) and read path (query_frontend → querier → storegateway → storage).

a. Main Components to include:
- Clients (Prometheus or OTEL exporters) →
- Distributor (HTTP/gRPC ingress) →
- Ingester cluster (in-memory TSDB, replication, gRPC) →
- Object storage (S3/GCS/Swift/Azure) & Index storage
- Store-gateway (serving historical blocks)
- Query-frontend (dedupe, fan-out, rate-limit)
- Querier (executes PromQL, fan-in streams)
- Compactor (block compaction, retention enforcement)
- Ruler (recording rules evaluation)
- Alertmanager (alert routing & dedup)
- Ring/KV store (Consul/Etcd/DynamoDB) & Gossip RPC
- Metrics & Tracing pipelines (OpenTelemetry collector, Grafana Agent)
- Optional tooling: query-tee, test-exporter, thanosconvert

b. Relationships & Data Flows:
1. Prometheus → Distributor (remote_write API)
2. Distributor → Ingester(s) (gRPC, ring-based sharding)
3. Ingester → Object Storage + Index in parallel
4. Queries: User (Grafana) → Query-Frontend → Querier → Store-Gateway → Object Storage
5. Ruler periodically queries / PromQL via Query-Frontend → stores results via Alertmanager API → Alertmanager
6. Compactor watches completed blocks in Object Storage → writes down-sampled or compacted blocks
7. All services join a ring via KV or Gossip for discovery and sharding
8. Service health & metrics scraped by Prometheus

c. Architectural Patterns & Principles:
- Microservices: each binary in “cmd/” fulfills a single responsibility.
- Ring-based consistent hashing for sharding and replication.
- Side-car/gossip/RPC for service discovery and high availability.
- Fan-out/fan-in for query scalability.
- Asynchronous, append-only writes to object store; eventual consistency.
- Multi-tenant isolation via label-prefixing and tenant-ID metadata.

d. Key Technologies & Libraries:
- Go, gRPC (protobuf in pkg/*pb), HTTP APIs (pkg/api), PromQL engine (pkg/querier), TSDB storage libraries, Cortex protobuf codecs.
- Object store SDKs (pkg/storage/bucket), KV clients (Consul, Etcd, DynamoDB).
- Docker + Kubernetes manifests (not shown), Makefile automation, GitHub Actions.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Clients
    Clients["Prometheus / OTEL Exporters"]:::external

    %% Service Discovery
    Ring["Consistent-Hashing Ring & Gossip (KV)"]:::infra
    click Ring "https://github.com/cortexproject/cortex/blob/master/pkg/ring/ring.go"
    RingKV["KV Store Clients"]:::infra
    click RingKV "https://github.com/cortexproject/cortex/tree/master/pkg/ring/kv"

    %% Ingestion Layer
    subgraph "Ingestion" 
        direction TB
        Distributor["Distributor (cmd/distributor)"]:::ingestion
        click Distributor "https://github.com/cortexproject/cortex/blob/master/pkg/distributor/distributor.go"
        Ingester["Ingester (cmd/ingester)"]:::ingestion
        click Ingester "https://github.com/cortexproject/cortex/blob/master/pkg/ingester/ingester.go"
    end

    %% Storage Layer
    subgraph "Storage" 
        direction TB
        ObjectStorage["Object Storage (S3/GCS/Swift/Azure)"]:::external
        click ObjectStorage "https://github.com/cortexproject/cortex/tree/master/pkg/storage/bucket"
        TSDBStorage["TSDB Block Storage"]:::storage
        click TSDBStorage "https://github.com/cortexproject/cortex/tree/master/pkg/storage/tsdb"
        StoreGateway["Store-Gateway (cmd/store-gateway)"]:::storage
        click StoreGateway "https://github.com/cortexproject/cortex/blob/master/pkg/storegateway/gateway.go"
    end

    %% Query Layer
    subgraph "Query" 
        direction TB
        QueryFrontend["Query-Frontend (cmd/query-frontend)"]:::query
        click QueryFrontend "https://github.com/cortexproject/cortex/blob/master/pkg/frontend/frontend.go"
        Querier["Querier (cmd/querier)"]:::query
        click Querier "https://github.com/cortexproject/cortex/blob/master/pkg/querier/querier.go"
    end

    %% Background Services
    subgraph "Background Jobs"
        direction TB
        Compactor["Compactor (cmd/compactor)"]:::background
        click Compactor "https://github.com/cortexproject/cortex/blob/master/pkg/compactor/compactor.go"
        Ruler["Ruler (cmd/ruler)"]:::background
        click Ruler "https://github.com/cortexproject/cortex/blob/master/pkg/ruler/ruler.go"
        Alertmanager["Alertmanager (cmd/alertmanager)"]:::background
        click Alertmanager "https://github.com/cortexproject/cortex/blob/master/pkg/alertmanager/alertmanager.go"
    end

    %% Tools and Utilities
    subgraph "Tools"
        direction TB
        CortexBinary["Cortex Binary (cmd/cortex)"]:::tooling
        click CortexBinary "https://github.com/cortexproject/cortex/blob/master/cmd/cortex/main.go"
        QueryTee["Query-Tee (cmd/query-tee)"]:::tooling
        click QueryTee "https://github.com/cortexproject/cortex/blob/master/cmd/query-tee/main.go"
        TestExporter["Test-Exporter (cmd/test-exporter)"]:::tooling
        click TestExporter "https://github.com/cortexproject/cortex/blob/master/cmd/test-exporter/main.go"
        ThanosConvert["ThanosConvert (cmd/thanosconvert)"]:::tooling
        click ThanosConvert "https://github.com/cortexproject/cortex/blob/master/cmd/thanosconvert/main.go"
    end

    %% Config & API & Tracing
    Config["Configuration / Init"]:::infra
    click Config "https://github.com/cortexproject/cortex/blob/master/pkg/configs/config.go"
    CortexInit["Cortex Init"]:::infra
    click CortexInit "https://github.com/cortexproject/cortex/blob/master/pkg/cortex/cortex.go"
    API["External API (health & query)"]:::infra
    click API "https://github.com/cortexproject/cortex/blob/master/pkg/api/api.go"
    Tracing["Tracing Integration"]:::infra
    click Tracing "https://github.com/cortexproject/cortex/blob/master/pkg/tracing/tracing.go"

    %% Data Flows
    Clients -->|"remote_write (HTTP/gRPC)"| Distributor
    Distributor -->|"gRPC (shuffle sharding)"| Ingester
    Distributor -->|joins ring| Ring
    Ingester -->|writes blocks| ObjectStorage
    Ingester -->|writes TSDB| TSDBStorage
    Ingester -->|joins ring| Ring
    QueryFrontend -->|joins ring| Ring
    Querier -->|joins ring| Ring
    StoreGateway -->|joins ring| Ring
    Ruler -->|joins ring| Ring
    Compactor -->|joins ring| Ring
    Alertmanager -->|joins ring| Ring

    %% Write path
    Clients --> Distributor
    Distributor --> Ingester
    Ingester --> ObjectStorage
    Ingester --> TSDBStorage

    %% Read path
    User["Grafana / API Clients"]:::external
    User -->|"query_range (HTTP)"| QueryFrontend
    QueryFrontend -->|"PromQL fan-out"| Querier
    Querier -->|"fetch blocks"| StoreGateway
    StoreGateway --> ObjectStorage
    StoreGateway --> TSDBStorage

    %% Background flows
    Compactor -->|"compaction"| ObjectStorage
    Ruler -->|"PromQL via frontend"| QueryFrontend
    Ruler -->|alert write| Alertmanager
    Alertmanager -->|notification| User

    %% Infrastructure
    RingKV --> Ring
    ObjectStorage -->|backend| ObjectStorage

    %% Metrics & Tracing
    PrometheusMetrics["Prometheus Scraper"]:::infra
    PrometheusMetrics --> Distributor
    PrometheusMetrics --> Ingester
    PrometheusMetrics --> Querier
    PrometheusMetrics --> StoreGateway
    PrometheusMetrics --> Ruler
    PrometheusMetrics --> Compactor
    PrometheusMetrics --> Alertmanager
    Tracing --> Distributor
    Tracing --> Ingester
    Tracing --> Querier

    %% Config and Init usage
    Config --> Distributor
    Config --> Ingester
    Config --> Querier
    Config --> StoreGateway
    CortexInit --> Distributor
    CortexInit --> Ingester
    CortexInit --> Querier
    CortexInit --> StoreGateway

    %% Styles
    classDef ingestion fill:#cce5ff,stroke:#3399ff,color:#003366;
    classDef storage fill:#e6ffcc,stroke:#66cc33,color:#336600;
    classDef query fill:#d9f2ff,stroke:#00aaff,color:#005566;
    classDef background fill:#ffe6cc,stroke:#ff9933,color:#663300;
    classDef external fill:#f2f2f2,stroke:#999999,color:#333333;
    classDef infra fill:#f0e6ff,stroke:#9966cc,color:#330066;
    classDef tooling fill:#fff2cc,stroke:#ffcc33,color:#665500;
    class Clients,User,PrometheusMetrics external
    class Ring,RingKV,Config,CortexInit,API,Tracing infra
    class Distributor,Ingester ingestion
    class ObjectStorage,TSDBStorage,StoreGateway storage
    class QueryFrontend,Querier query
    class Compactor,Ruler,Alertmanager background
    class CortexBinary,QueryTee,TestExporter,ThanosConvert tooling
```
