# webdevops/azure-metrics-exporter architecture

> Architecture diagram of the GitHub repository webdevops/azure-metrics-exporter, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/webdevops/azure-metrics-exporter
- Repository: https://github.com/webdevops/azure-metrics-exporter

## Overview

##### Project Type and Purpose
- This is a backend service: a Go-based Prometheus exporter (“azure-metrics-exporter”) that scrapes Azure Monitor metrics and exposes them in Prometheus format.
- It runs as a single Docker container, listens on HTTP endpoints (`/metrics`, `/probe/...`, `/query`), uses the Azure SDK to fetch data, and employs an in-memory cache to reduce Azure API calls.

##### File Structure
- Top-level:
- `main.go` – application entry point (HTTP server init, router setup).
- `common.*.go` – shared utilities (logging, system info).
- `config/` – environment‐variable and CLI flag definitions (`opts.go`, `consts.go`).
- `metrics/` – Azure‐specific metric clients (policy, insights, subscription, resourcegraph, etc.) and general metric helpers.
- Probe‐level functions (`probe_metrics_*.go`) – implement each `/probe/metrics*` endpoint.
- `templates/query.html` – minimal web UI for interactive query testing.
- Docker, CI/CD, and build scripts support containerization and automated releases.

##### README Insights
- Exposes multiple HTTP endpoints for different scraping modes: subscription-wide, single resource, list via service discovery, ResourceGraph queries, etc.
- Leverages the Azure SDK for Go, configurable entirely via Prometheus scrape config and environment variables.
- Features in-memory caching of service discovery and fetched metrics, templating engine for metric names, and full dimension support.

##### Main Components & Their Relationships
a. HTTP Server Layer
- Listens on configurable port (default :8080).
- Routes requests to endpoint handlers:
- `/metrics` – built-in Go exporter metrics.
- `/probe/metrics*` – sharded probes for Azure Monitor data.
- `/query` – serves `query.html` for dev web UI.
b. Configuration Loader
- Reads CLI flags and environment variables (`config/opts.go`).
- Provides settings to all modules: timeouts, caching durations, template strings, Azure environment.
c. Logging & System Info
- `common.logger.go` configures log format (JSON or text, levels).
- `common.system.go` captures runtime metadata.
d. Service Discovery Module
- Uses Azure ResourceManager API and Azure ResourceGraph API to list resources based on filters or tags.
- Caches discovery results (`metrics/servicediscovery.go`) for configurable duration.
e. Metrics Collector (Probe) Module
- Implements logic in `metrics/*.go` for each Azure service (Insights, Policy, subscription metrics).
- Batches requests per resource or subscription, supports multiple aggregations and dimensions.
- Applies the template engine to generate Prometheus metric names and help strings.
f. In-Memory Cache
- Two layers:
1. Service Discovery cache
2. Metric fetch cache
- Reduces Azure API calls; cache durations are user-configurable.
g. Azure SDK Client
- Underlying HTTP client for Azure REST calls (monitored by AzureTracing metrics).
- Auth via environment variables or Managed Identity.
h. Template Engine
- Renders metric names and help texts using Go templates, supports placeholders (`{name}`, `{metric}`, `{aggregation}`, etc.).
i. Query Web UI
- Single‐page HTML served under `/query` to experiment with endpoints and auto-generate Prometheus scrape configs.
j. External Systems
- Prometheus (scrapes exporter endpoints).
- Azure Monitor & ARM & ResourceGraph APIs.
- Docker Registry & CI pipelines for build, test, and release.

##### Architectural Patterns & Design Principles
- Layered architecture: HTTP handling ➔ configuration ➔ discovery ➔ data collection ➔ templating ➔ response.
- Separation of concerns: probes isolated per endpoint and service type.
- Caching as a decorator around service discovery and metric fetches.
- Plugin-style modules: each Azure service lives in its own file and can be individually maintained.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Systems
    Prometheus["Prometheus"]:::external
    AzureMonitor["Azure Monitor API"]:::azure
    AzureResourceGraph["Azure ResourceGraph API"]:::azure
    AzureResourceManager["Azure ResourceManager API"]:::azure
    EnvVars(("Environment Variables")):::external

    %% Docker Container Boundary
    subgraph "azure-metrics-exporter (Docker Container)" 
        direction TB
        HTTPServer["HTTP Server Layer\n(listens on /metrics, /probe/*, /query)"]:::internal
        ConfigLoader["Configuration Loader"]:::internal
        Logger["Logging Module"]:::internal
        SystemInfo["System Info Module"]:::internal

        subgraph Caches
            direction TB
            SDCache["ServiceDiscovery Cache"]:::cache
            MetricsCache["Metrics Fetch Cache"]:::cache
        end

        ServiceDiscovery["Service Discovery Module"]:::internal
        MetricsCollectorCore["Metrics Collector (Probe) Core"]:::internal

        subgraph "Probe Endpoints"
            direction TB
            ProbeSub["/probe/metrics Subscription"]:::internal
            ProbeRes["/probe/metrics Resource"]:::internal
            ProbeList["/probe/metrics List"]:::internal
            ProbeRG["/probe/metrics ResourceGraph"]:::internal
            ProbeScrape["/probe/metrics Scrape"]:::internal
        end

        TemplateEngine["Template Engine"]:::internal
        QueryUI["Query Web UI\n(templates/query.html)"]:::ui

        %% Internal Flows
        EnvVars -.-> ConfigLoader
        ConfigLoader --> HTTPServer
        Logger -.-> HTTPServer
        SystemInfo -.-> HTTPServer

        Prometheus --> HTTPServer
        HTTPServer --> ProbeSub
        HTTPServer --> ProbeRes
        HTTPServer --> ProbeList
        HTTPServer --> ProbeRG
        HTTPServer --> ProbeScrape
        HTTPServer -->|"serves /query"| QueryUI

        ProbeSub --> SDCache
        ProbeRes --> SDCache
        ProbeList --> SDCache
        ProbeRG --> SDCache
        ProbeScrape --> SDCache

        SDCache --> ServiceDiscovery
        ServiceDiscovery --> AzureResourceManager
        ServiceDiscovery --> AzureResourceGraph

        ProbeSub --> MetricsCollectorCore
        ProbeRes --> MetricsCollectorCore
        ProbeList --> MetricsCollectorCore
        ProbeRG --> MetricsCollectorCore
        ProbeScrape --> MetricsCollectorCore

        MetricsCollectorCore --> AzureMonitor
        AzureMonitor --> MetricsCollectorCore

        MetricsCollectorCore --> MetricsCache
        MetricsCache --> TemplateEngine

        TemplateEngine --> HTTPServer
        HTTPServer --> Prometheus
    end

    %% Click Events
    click HTTPServer "https://github.com/webdevops/azure-metrics-exporter/blob/main/main.go"
    click ConfigLoader "https://github.com/webdevops/azure-metrics-exporter/blob/main/config/opts.go"
    click ConfigLoader "https://github.com/webdevops/azure-metrics-exporter/blob/main/config/consts.go"
    click Logger "https://github.com/webdevops/azure-metrics-exporter/blob/main/common.logger.go"
    click SystemInfo "https://github.com/webdevops/azure-metrics-exporter/blob/main/common.system.go"
    click SDCache "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/servicediscovery.go"
    click ServiceDiscovery "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/servicediscovery.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/prober.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/metrics.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/settings.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/misc.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/azure.policy.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/insights.general.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/insights.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/insights.subscription.go"
    click MetricsCollectorCore "https://github.com/webdevops/azure-metrics-exporter/blob/main/metrics/insights.target.go"
    click ProbeSub "https://github.com/webdevops/azure-metrics-exporter/blob/main/probe_metrics_subscription.go"
    click ProbeRes "https://github.com/webdevops/azure-metrics-exporter/blob/main/probe_metrics_resource.go"
    click ProbeList "https://github.com/webdevops/azure-metrics-exporter/blob/main/probe_metrics_list.go"
    click ProbeRG "https://github.com/webdevops/azure-metrics-exporter/blob/main/probe_metrics_resourcegraph.go"
    click ProbeScrape "https://github.com/webdevops/azure-metrics-exporter/blob/main/probe_metrics_scrape.go"
    click QueryUI "https://github.com/webdevops/azure-metrics-exporter/blob/main/templates/query.html"
    click misc "https://github.com/webdevops/azure-metrics-exporter/blob/main/misc.go"
    click Dockerfile "https://github.com/webdevops/azure-metrics-exporter/tree/main/Dockerfile"
    click Makefile "https://github.com/webdevops/azure-metrics-exporter/tree/main/Makefile"
    click CI "https://github.com/webdevops/azure-metrics-exporter/blob/main/.github/workflows/*.yaml"

    %% Styles
    classDef internal fill:#D6E9F8,stroke:#4A90E2,stroke-width:1px
    classDef cache fill:#FFF2CC,stroke:#E6B800,stroke-width:1px,shape:cylinder
    classDef external fill:#EEEEEE,stroke:#BBBBBB,stroke-dasharray: 5 5
    classDef azure fill:#E0F7FA,stroke:#00838F,stroke-width:1px
    classDef ui fill:#E8F5E9,stroke:#43A047,stroke-width:1px
```
