How platformbuilds/mirador-core works
Project Type & Purpose
• This is a Go-based backend microservice (“mirador-core”) in a larger observability platform. It exposes a unified REST+WebSocket API (Gin) for metrics, logs, traces, AI-driven analysis (predict/RCA/alerts), schema definitions, and real-time streams.
• It orchestrates between clients (UI, Prometheus, CLI), external backends (VictoriaMetrics ecosystem, search engines, AI engines), and caches/sessions (Valkey/Redis cluster).
File Structure Analysis
• Top-level:
• cmd/server & cmd/schemactl – entrypoints for API server & CLI tool
• api/ – OpenAPI/Swagger definitions & Postman collection
• internal/ – application code organized by domain:
– api (handlers, middleware, server)
– config, discovery, grpc (clients + protobufs)
– services (business logic for metrics, logs, traces, AI, schema store, notifications)
– repo/storage (Weaviate for schema persistence)
– utils, metrics, monitoring, models
• pkg/ – reusable packages (auth, caching, logging)
• deployments/ & chart/ – Helm chart, Kubernetes yamls, local-dev Docker Compose
• Makefile, Dockerfile, CI workflows – build/test/deploy automation
• This layered layout reveals a clear separation:
• HTTP/gRPC ingress → middleware → handlers → services → backends (HTTP+gRPC) + cache + schema store.
README Insights
• Confirms main components: REST API, WebSocket streams, gRPC AI engines (Predict, RCA, Alert), VictoriaMetrics (metrics, logs, traces), optional search engines (Lucene & Bleve), Valkey cluster for caching/sessions, Weaviate for schema definitions, enterprise auth (LDAP/AD, OAuth 2.0, RBAC), external notifications (Slack/Teams/SMTP).
• Clients:
• Browser-based UI (mirador-ui)
• Prometheus scraper
• CLI (schemactl) / engineers via CURL/Postman
• Ingress Layer:
• Load Balancer / API Gateway / Ingress Controller (TLS termination)
• mirador-core Service:
• Gin HTTP Server + WebSocket endpoint
• Middleware (Auth, RBAC, Rate Limiter, Request Logger, Metrics)
• Handlers (MetricsQL, LogsQL, Traces, Predict, RCA, Alerts, Schema, Sessions)
• Internal Services Layer
– VictoriaMetricsServices (metrics/logs/traces)
– gRPC Clients (Predict/RCA/Alert engines)
– NotificationService
• Caching & Sessions:
• Valkey cluster (Redis-based) for query result caching, sessions, rate limiting counters
• Persistence & Backends:
• VictoriaMetrics ecosystem: VM-Select (metrics), VL-Select (logs), VT-Select (traces/Jaeger)
• Search Engines: Lucene service or embedded Bleve engine
• Weaviate for schema definitions
• Identity/Authentication: LDAP/AD or OAuth/OIDC provider
• External Notification endpoints: Slack, MS Teams, SMTP
• Clients → Ingress/LB → mirador-core HTTP
• mirador-core Handlers → Internal Services
• Services →
• VM/VL/VT HTTP endpoints
• gRPC to Predict/RCA/Alert engines
• Weaviate HTTP API
• Valkey cluster (Redis)
• External notifier webhooks
• mirador-core → WebSocket streams back to UI
• Prometheus → mirador-core /metrics
c. Architectural Patterns & Principles:
• Layered architecture (HTTP API → middleware → handlers → services → backends)
• Dependency injection via configuration (Viper)
• Stateless service design for horizontal scaling
• Cache-aside pattern with Valkey cluster for performance
• Façade over multiple metric/log/trace sources – fan-out + aggregation
• Pluggable search engine strategy (strategy pattern for Lucene vs. Bleve)
• Bulk upload safety controls (in-memory CSV handling)
d. Key Technologies & Libraries:
• Go / Gin HTTP framework / WebSocket
• gRPC + Protocol Buffers (AI engines)
• VictoriaMetrics (PromQL/MetricsQL), VictoriaLogs (LogsQL), VictoriaTraces (Jaeger API)
• Bleve & Lucene query translation
• Valkey cluster (Redis) caching
• Weaviate client (Go)
• LDAP/AD & OAuth 2.0 (enterprise auth), JWT sessions
• Helm, Kubernetes, Docker, Makefile for deployment
Final Checklist
• Essential components and interactions without over-splitting trivial modules.
• Keep it at a single page/slide level; use callouts or expanded inset for complex sub-flows (e.g., AI analysis pipeline).
• Validate with stakeholders (DevOps, AI team) that ports, protocols, and dependencies are accurate.