How webdevops/azure-metrics-exporter works
Project Type and Purpose
• This is a backend service: a Go-based Prometheus exporter (“azure-metrics-exporter”) that scrapes Azure Monitor metrics and exposes them in Prometheus format.
• It runs as a single Docker container, listens on HTTP endpoints (/metrics, /probe/..., /query), uses the Azure SDK to fetch data, and employs an in-memory cache to reduce Azure API calls.
File Structure
• Top-level:
• main.go – application entry point (HTTP server init, router setup).
• common.*.go – shared utilities (logging, system info).
• config/ – environment‐variable and CLI flag definitions (opts.go, consts.go).
• metrics/ – Azure‐specific metric clients (policy, insights, subscription, resourcegraph, etc.) and general metric helpers.
• Probe‐level functions (probe_metrics_*.go) – implement each /probe/metrics* endpoint.
• templates/query.html – minimal web UI for interactive query testing.
• Docker, CI/CD, and build scripts support containerization and automated releases.
README Insights
• Exposes multiple HTTP endpoints for different scraping modes: subscription-wide, single resource, list via service discovery, ResourceGraph queries, etc.
• Leverages the Azure SDK for Go, configurable entirely via Prometheus scrape config and environment variables.
• Features in-memory caching of service discovery and fetched metrics, templating engine for metric names, and full dimension support.
Main Components & Their Relationships
a. HTTP Server Layer
• Listens on configurable port (default :8080).
• Routes requests to endpoint handlers:
• /metrics – built-in Go exporter metrics.
• /probe/metrics* – sharded probes for Azure Monitor data.
• /query – serves query.html for dev web UI.
b. Configuration Loader
• Reads CLI flags and environment variables (config/opts.go).
• Provides settings to all modules: timeouts, caching durations, template strings, Azure environment.
c. Logging & System Info
• common.logger.go configures log format (JSON or text, levels).
• common.system.go captures runtime metadata.
d. Service Discovery Module
• Uses Azure ResourceManager API and Azure ResourceGraph API to list resources based on filters or tags.
• Caches discovery results (metrics/servicediscovery.go) for configurable duration.
e. Metrics Collector (Probe) Module
• Implements logic in metrics/*.go for each Azure service (Insights, Policy, subscription metrics).
• Batches requests per resource or subscription, supports multiple aggregations and dimensions.
• Applies the template engine to generate Prometheus metric names and help strings.
f. In-Memory Cache
• Two layers:
1. Service Discovery cache
2. Metric fetch cache
• Reduces Azure API calls; cache durations are user-configurable.
g. Azure SDK Client
• Underlying HTTP client for Azure REST calls (monitored by AzureTracing metrics).
• Auth via environment variables or Managed Identity.
h. Template Engine
• Renders metric names and help texts using Go templates, supports placeholders ({name}, {metric}, {aggregation}, etc.).
i. Query Web UI
• Single‐page HTML served under /query to experiment with endpoints and auto-generate Prometheus scrape configs.
j. External Systems
• Prometheus (scrapes exporter endpoints).
• Azure Monitor & ARM & ResourceGraph APIs.
• Docker Registry & CI pipelines for build, test, and release.
Architectural Patterns & Design Principles
• Layered architecture: HTTP handling ➔ configuration ➔ discovery ➔ data collection ➔ templating ➔ response.
• Separation of concerns: probes isolated per endpoint and service type.
• Caching as a decorator around service discovery and metric fetches.
• Plugin-style modules: each Azure service lives in its own file and can be individually maintained.