# partnerstack/celery-prometheus-exporter architecture

> Architecture diagram of the GitHub repository partnerstack/celery-prometheus-exporter, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/partnerstack/celery-prometheus-exporter
- Repository: https://github.com/partnerstack/celery-prometheus-exporter

## Overview

celery-prometheus-exporter monitors Celery over Redis and exposes task, worker, runtime, and configured queue-length metrics for Prometheus. The main workflow configures the exporter, receives Celery events into task state, periodically samples workers and queue lengths, updates metric collectors, and serves them over HTTP. The repository also contains a configured Celery demo workload. Relationships below are based on the README and excerpted source; unsampled integrations are not inferred.

Main entry points: [`exporter/__init__.py`](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py) (Periodic collector), [`exporter/metrics.py`](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/metrics.py) (Metric definitions), [`app/config.py`](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/app/config.py) (Demo task config), [`app/__init__.py`](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/app/__init__.py) (Demo Celery tasks)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_setup["Exporter Setup"]
  node_cli["Exporter startup<br/>[__init__.py]"]
  node_celery_client["Celery client<br/>[__init__.py]"]
  node_demo_config["Demo task config<br/>[config.py]"]
  node_demo_tasks["Demo Celery tasks<br/>[__init__.py]"]
end

subgraph group_collection["Metric Collection"]
  node_event_receiver["Event receiver<br/>[__init__.py]"]
  node_task_state["Task state<br/>[__init__.py]"]
  node_collector["Periodic collector<br/>[__init__.py]"]
end

subgraph group_exposure["Metrics Exposure"]
  node_metric_defs["Metric definitions<br/>[metrics.py]"]
  node_registry["Prometheus registry<br/>[metrics.py]"]
  node_http_endpoint["HTTP metrics endpoint<br/>[__init__.py]"]
end

node_operator(("Operator"))
node_redis[("Redis broker")]
node_prometheus(("Prometheus"))

node_operator -->|"configures"| node_cli
node_cli -->|"configures"| node_celery_client
node_celery_client -->|"connects to"| node_redis
node_celery_client -->|"starts"| node_event_receiver
node_event_receiver -->|"updates"| node_task_state
node_event_receiver -->|"observes runtime"| node_metric_defs
node_cli -->|"starts"| node_collector
node_collector -->|"pings workers"| node_celery_client
node_collector -->|"reads queue lengths"| node_redis
node_collector -->|"counts tasks"| node_task_state
node_collector -->|"updates gauges"| node_metric_defs
node_metric_defs -->|"registers metrics"| node_registry
node_cli -->|"starts"| node_http_endpoint
node_http_endpoint -->|"serves"| node_registry
node_prometheus -.->|"scrapes"| node_http_endpoint
node_demo_config -->|"configures broker"| node_redis
node_demo_tasks -->|"uses settings"| node_demo_config

click node_cli "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py"
click node_celery_client "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py"
click node_event_receiver "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py"
click node_task_state "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py"
click node_collector "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py"
click node_metric_defs "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/metrics.py"
click node_registry "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/metrics.py"
click node_http_endpoint "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py"
click node_demo_config "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/app/config.py"
click node_demo_tasks "https://github.com/partnerstack/celery-prometheus-exporter/blob/master/app/__init__.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_cli,node_celery_client,node_demo_config,node_demo_tasks toneBlue
class node_event_receiver,node_task_state,node_collector,node_redis toneAmber
class node_metric_defs,node_registry,node_http_endpoint toneMint
class node_operator,node_prometheus toneIndigo
```

## Components

### Exporter Setup

- [Exporter startup](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py): `exporter/__init__.py`
- [Celery client](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py): `exporter/__init__.py`
- [Demo task config](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/app/config.py): `app/config.py`
- [Demo Celery tasks](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/app/__init__.py): `app/__init__.py`

### Metric Collection

- [Event receiver](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py): `exporter/__init__.py`
- [Task state](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py): `exporter/__init__.py`
- [Periodic collector](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py): `exporter/__init__.py`

### Metrics Exposure

- [Metric definitions](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/metrics.py): `exporter/metrics.py`
- [Prometheus registry](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/metrics.py): `exporter/metrics.py`
- [HTTP metrics endpoint](https://github.com/partnerstack/celery-prometheus-exporter/blob/master/exporter/__init__.py): `exporter/__init__.py`

### Other components

- **Operator**
- **Redis broker**
- **Prometheus**

## Connections

- Operator → Exporter startup: configures (evidence: `README.md`)
- Exporter startup → Celery client: configures (evidence: `exporter/__init__.py`)
- Celery client → Redis broker: connects to (evidence: `exporter/__init__.py`)
- Celery client → Event receiver: starts (evidence: `exporter/__init__.py`)
- Event receiver → Task state: updates (evidence: `exporter/__init__.py`)
- Event receiver → Metric definitions: observes runtime (evidence: `exporter/__init__.py`)
- Exporter startup → Periodic collector: starts (evidence: `exporter/__init__.py`)
- Periodic collector → Celery client: pings workers (evidence: `exporter/__init__.py`)
- Periodic collector → Redis broker: reads queue lengths (evidence: `exporter/__init__.py`)
- Periodic collector → Task state: counts tasks (evidence: `exporter/__init__.py`)
- Periodic collector → Metric definitions: updates gauges (evidence: `exporter/__init__.py`)
- Metric definitions → Prometheus registry: registers metrics (evidence: `exporter/metrics.py`)
- Exporter startup → HTTP metrics endpoint: starts (evidence: `exporter/__init__.py`)
- HTTP metrics endpoint → Prometheus registry: serves (evidence: `exporter/__init__.py`)
- Prometheus → HTTP metrics endpoint: scrapes (evidence: `README.md`)
- Demo task config → Redis broker: configures broker (evidence: `app/config.py`)
- Demo Celery tasks → Demo task config: uses settings (evidence: `app/__init__.py`)
