How axnasim/realtime-build-telemetry-dashboard works
What this project is (type + purpose)
This is a small full-stack “single service + static UI” system that provides a real-time build/test telemetry dashboard:
• Producers (CI agents/build runners) POST build events to a FastAPI backend (/metrics).
• The backend persists events (currently SQLite) and broadcasts updates over WebSockets to connected browsers.
• A static HTML/JS dashboard (Chart.js) renders real-time pass/fail counts and average duration.
• There are clear “next-step” integration points for Azure Event Hubs, Application Insights, and potentially Azure Data Explorer.
Runtime application (core)
• src/app_with_otel.py
FastAPI app entrypoint with REST + WebSocket and (as the name implies) OpenTelemetry instrumentation.
• src/models.py
• src/storage.py
Persistence & queries (SQLite insert and summary queries). This is the boundary between service logic and the DB.
• src/static/index.html
Client UI (Chart.js), connects via WebSocket and possibly fetches initial summaries via REST.
Data / examples
• data/sample_events.jsonl
CI / automation / telemetry
• .github/workflows/ci-with-telemetry.yml + ci-telemetry.json + ci_telemetry_client.py
• tests/*
Confirms API endpoints, DB behavior, integration (important for identifying boundaries: API ↔ storage ↔ websocket).
Deployment / packaging
• Dockerfile, docker-compose.yml, .env.example
These define how to run the service and any dependencies. Even if compose only runs one container, show the container boundary and config injection via environment variables.
Telemetry Producers
• “CI Telemetry Client” (ci_telemetry_client.py)
• “GitHub Actions Workflow” (.github/workflows/ci-with-telemetry.yml)
• Optionally “Developer curl / local scripts”
FastAPI Backend Service
• REST endpoint: POST /metrics
• Static file hosting for / → src/static/index.html
SQLite Database
• File-based DB (can be shown as “SQLite file volume” if containerized)
Web Dashboard (Browser)
• Loads static HTML/JS (Chart.js)
• Maintains WebSocket connection for live updates
Observability (OpenTelemetry)
• If exporter target isn’t defined, label it generically: “OTel Collector / Observability Backend (future: Application Insights)”.
• Azure Event Hubs (ingestion source)
• Application Insights (traces/metrics/logs)
• Azure Data Explorer (ADX) (storage + analytics)
• API Router / REST Handler
• Accepts JSON telemetry event (Pydantic validation via models.py)
• Calls storage layer
• Triggers broadcast
• Storage Layer (storage.py)
• Insert event
• Query aggregates (pass/fail counts, avg duration)
• WebSocket Manager / Broadcaster
• Tracks connected clients
• Pushes updated summary/events
• Static Content Server
• Serves index.html + JS assets
• OTel Middleware/Instrumentation
• Captures request spans, WS spans (if enabled), DB spans (if instrumented)
This gives you the “as many components as possible” decomposition without guessing beyond the repo.
Producer → FastAPI REST
• Payload: {agent_id, build_id, status, duration_ms, ...}
• “Pydantic validation (models.py)”
Browser → FastAPI
• Optionally “HTTP GET / (static index.html)”
CI pipeline → FastAPI
• Ingestion path: Producer → API → DB
• Fan-out path: API → WebSocket → UI
Important architectural patterns to explicitly show
• Event ingestion + broadcast pattern
REST for ingestion, WebSocket for real-time fan-out.
• Layered architecture inside the service
API layer → validation/models → storage abstraction → DB.
• Single-writer / multi-reader
Many producers write events; many dashboards read via WebSocket.
• Pluggable backends (evolution path)
SQLite today; ADX tomorrow; Event Hubs as alternate ingestion; Application Insights for observability.
System Context (1 slide)
Producers + Browser + Telemetry Dashboard Service + Storage + (Optional Azure).
Browser, FastAPI container, SQLite, OTel exporter, CI producer.
REST /metrics handler → storage → broadcast manager → WS clients; static UI serving.