How prometheus-community/stackdriver_exporter works
Project Type and Purpose
• This is an open-source Go-based “exporter” that bridges Prometheus and Google Stackdriver Monitoring.
• It implements a proxy: on each Prometheus scrape it queries the Stackdriver API for time-series data, adapts them into Prometheus metrics, and exposes them over HTTP.
High-Level Architecture
A. CLI & Configuration Layer
• Parses flags (project IDs, metric prefixes, filters, web.listen-address, TLS/auth config).
• Bootstraps internal components and HTTP server.
B. HTTP Server / Web Layer
• Exposes two endpoints:
– /metrics for Prometheus to scrape converted Stackdriver metrics
– /stackdriver (or configured) for internal exporter telemetry
• Uses [exporter-toolkit](https://github.com/prometheus/exporter-toolkit) for TLS/basic-auth (optional).
C. Metrics Registry
• A Prometheus Registry that holds collectors and their returned metric families.
D. Collector Components (in collectors/ and delta/)
1. MonitoringCollector
– For each configured metric prefix:
• Builds a Stackdriver API filter (project + prefix + any user-provided filters).
• Calls the Stackdriver Monitoring API via Google’s Go client.
– Converts GAUGE → Prometheus Gauge, CUMULATIVE → Counter, DISTRIBUTION → Histogram.
2. Delta Aggregator (optional, --monitoring.aggregate-deltas)
– Tracks DELTA-kind metrics in-memory as counters.
– Uses TTL and jitter/backoff settings to handle slow moving metrics and API retries.
3. Cache/Descriptor Store (collectors/cache.go)
– Caches metric descriptors and metadata to reduce repeated API calls.
E. Utility Layer (utils/, hash/)
• Normalizes metric names and labels to Prometheus spec.
• Computes consistent hashes for time-series ordering or caching keys.
External Services & Integrations
• Google Stackdriver Monitoring API (REST/gRPC)
• Prometheus server (HTTP scrape client)
• Optional deployment targets: Docker, Kubernetes (Helm chart), Cloud Foundry, BOSH
Data Flow & Interactions
1. Prometheus → HTTP GET /metrics
2. HTTP Server → invokes registered collectors in Registry
3. For each collector:
a. Build Stackdriver query (with filter, interval, offset)
b. (Optional) Check in-memory cache or delta store
c. Call Google API → receive time-series
d. Convert to Prometheus metric families
e. Store / update delta counters if configured
4. Registry → collate all metric families → render text protobuf
5. Prometheus ingest metrics
Architectural Patterns & Principles
• Plugin-style collectors: each metric prefix/type can be added or filtered dynamically.
• Separation of concerns: HTTP serving, metric collection, delta aggregation, caching, and CLI parsing are modularized.
• In-memory state for DELTA aggregation, with TTL to bound resource usage.
• Retry/backoff patterns for robust Stackdriver API calls.
• Core data path (Prometheus scrape → collector → API → conversion → expose).
• Extensibility: where one would “plug in” a new collector or filter.