How kubesphere/alert works
Project type and purpose
• This is a production-grade, distributed alerting system built in Go. It exposes gRPC/HTTP endpoints (API Gateway), manages alert policies (Manager), executes checks and notifications (Executor), and monitors service health and cluster state (Watcher). It persists state in MySQL and coordinates distributed tasks/locks via etcd.
File structure
• Top-level:
• cmd/alert: main entry point for the alert binary
• api/: protobuf definitions for all gRPC/HTTP APIs (with vendor’ed grpc-gateway swagger tooling)
• pkg/: reusable modules organized by component (apigateway, client SDK, db, etcd, manager, executor, watcher, notification, models, services, etc.)
• deploy/: k8s YAMLs for each service (manager, executor, watcher, API Gateway, etcd, MySQL)
• Dockerfile(.dev), Makefile: build and dev workflows
• pkg/db holds schema migrations, initialization scripts, and DDL.
Extract architectural insights from README/doc
• README calls out four logical services (API Gateway, Manager, Executor, Watcher) plus external dependencies (MySQL, etcd).
Main components and interactions
a. Clients / External Users
• HTTP/JSON or gRPC SDK clients (pkg/client)
• kubectl/CLI/webhook integrations
b. API Gateway
• grpc-gateway layer translating REST ↔ gRPC
• Swagger UI / OpenAPI spec for external consumption
c. Manager Service
• Loads alert policies, handles create/read/update/delete of alerts (uses pkg/services/manager)
• Places check tasks into a queue or etcd lock for Executors
• Reads/writes metadata to MySQL (pkg/db, pkg/models)
d. Executor Service
• Picks up scheduled checks (runner)
• Performs health checks and metric queries (pkg/services/executor)
• Reports results back to Manager or directly triggers notifications (pkg/notification)
e. Watcher Service
• Monitors liveness of Managers & Executors (health check loop)
• Maintains cluster state, triggers recovery or re-assignment upon failures
f. Infrastructure
• MySQL: stores schema under pkg/db/schema and migrations
• etcd: distributed lock manager, leader election, coordination (pkg/etcd)
• Kubernetes: all services run as Pods controlled by deploy/*.yaml
• Clients → API Gateway (REST/gRPC)
• API Gateway → Manager (gRPC)
• Manager → MySQL (SQL)
• Manager ↔ etcd (lock, election)
• Manager → Executor (task dispatch)
• Executor → Manager/Notification (results, notifications)
• Watcher → Manager/Executor (health probes)
• Watcher → etcd (coordination)
Architectural patterns and principles
• Microservices: clear separation of Gateway, Manager, Executor, Watcher
• Event-driven / task queue: Manager dispatches, Executor consumes
• Distributed coordination: etcd for leader election and locks
• Idempotent database migrations (pkg/db/schema)
• Protocol-first API design: protobuf + grpc-gateway + swagger
Relevant technologies & frameworks
• Go modules, gRPC, Protobuf
• grpc-gateway → REST fallback + OpenAPI/Swagger
• MySQL with migration scripts (Flyway-style naming)
• etcd client for distributed locking
• Kubernetes + Docker