How prometheus/alertmanager works
Project type and purpose
• Alertmanager is a full-stack, stateless (but clustered) alert‐routing and notification service written in Go.
• It ingests alerts (typically from Prometheus), applies grouping, silencing, and inhibition rules, and dispatches notifications via various receivers (email, Slack, PagerDuty, webhooks, etc.).
• It exposes a REST/OpenAPI v2 API, a web UI (Elm+React), a command‐line client (amtool), and a metrics endpoint.
File structure
• cmd/alertmanager → the main HTTP server binary
• cmd/amtool → the CLI tool
• api/… → auto‐generated HTTP handlers and client code for the API v2
• cluster/… → gossip-based peer communication and replication
• config/… → parsing the YAML routing, inhibition, silences, and receiver configs
• store/…, silence/…, dispatch/… → in-memory stores, dispatch logic (grouping & routing), silence engine
• notify/… → individual notifier adapters (email, Slack, PagerDuty, webhook, etc.)
• template/… → Go template rendering for notification payloads
• ui/app (Elm) + ui/react-app (React) → two separate web UIs
• asset/ & embed → static asset bundling into the Go binary
• test/… & examples/… → acceptance tests and sample configs
README for additional insights
• README describes alert flow, clustering flags, high-availability requirements, and the OpenAPI contract
– HTTP ingress → API → core services → dispatch → notifiers
– Cluster messaging between peers
– CLI & web UI calling into the API
Components & interactions
A. Ingress Layer
1. HTTP Server (cmd/alertmanager)
– Exposes endpoints:
• /api/v2 (OpenAPI handlers in api/v2/restapi)
• /metrics (Prometheus metrics via api/metrics)
• Web UI assets (embedded Elm and React static files)
2. CLI Client (amtool)
– Calls the same HTTP API (/api/v2/*) over HTTP
B. Core Processing
1. Config Loader (config/)
– Parses YAML into Route Tree, Inhibition Rules, Receivers
2. Store
– Alerts Store (in‐memory, store/)
– Silences Store (in‐memory, silence/)
3. Dispatch Engine (dispatch/)
– Applies group_by and routes to build AlertGroups
– Checks silences & inhibition rules
4. Template Renderer (template/)
– Renders notification payloads (email HTML, JSON, Slack markdown)
C. Notification Layer
– Notify Adapters (notify/email, notify/slack, notify/pagerduty, etc.)
– Each adapter implements a common interface (notify/notify.go)
D. Cluster Subsystem (cluster/)
– Gossip protocol over UDP/TCP
– Connection pooling, TLS transport, delegate messages
– Replicates state (alerts, silences, etc.) across peers
E. Web UIs
1. Elm-based UI (ui/app) served as static assets
2. React-based UI (ui/react-app) also embedded and served
F. External Integrations
– Prometheus or any alerting client POSTs to /api/v2/alerts
– Prometheus scrapes /metrics
– External notifiers (email servers, Slack endpoints, PagerDuty API)
Architectural patterns & principles
• Layered architecture: Ingress → Core → Dispatch → Notifier
• Modular adapters for every notifier in notify/
• In-memory state replication via gossip for HA (distributed state)
• OpenAPI‐first API generation (api/v2/openapi.yaml + Go-Swagger)
• Embedded static assets (asset/ + Promu + go:embed)
• Separation of concerns: config parsing, business logic, transport
Be very detailed
• Inside “Core”, show sub-modules: Config Parser, Grouping Engine, Silence Engine, Inhibition Engine
• Static asset embedding (asset/ and ui/*) flowing into the HTTP server binary