# axnasim/realtime-build-telemetry-dashboard architecture

> Architecture diagram of the GitHub repository axnasim/realtime-build-telemetry-dashboard, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/axnasim/realtime-build-telemetry-dashboard
- Repository: https://github.com/axnasim/realtime-build-telemetry-dashboard

## Overview

##### What this project is (type + purpose)
This is a **small full-stack “single service + static UI” system** that provides a **real-time build/test telemetry dashboard**:

- **Producers (CI agents/build runners)** POST build events to a **FastAPI** backend (`/metrics`).
- The backend **persists** events (currently **SQLite**) and **broadcasts** updates over **WebSockets** to connected browsers.
- A **static HTML/JS dashboard** (Chart.js) renders real-time pass/fail counts and average duration.
- There are clear “next-step” integration points for **Azure Event Hubs**, **Application Insights**, and potentially **Azure Data Explorer**.

##### Runtime application (core)
- `src/app_with_otel.py`
FastAPI app entrypoint with **REST + WebSocket** and (as the name implies) **OpenTelemetry instrumentation**.
- `src/models.py`
- `src/storage.py`
**Persistence & queries** (SQLite insert and summary queries). This is the boundary between service logic and the DB.
- `src/static/index.html`
**Client UI** (Chart.js), connects via WebSocket and possibly fetches initial summaries via REST.

##### Data / examples
- `data/sample_events.jsonl`

##### CI / automation / telemetry
- `.github/workflows/ci-with-telemetry.yml` + `ci-telemetry.json` + `ci_telemetry_client.py`
- `tests/*`
Confirms API endpoints, DB behavior, integration (important for identifying boundaries: API ↔ storage ↔ websocket).

##### Deployment / packaging
- `Dockerfile`, `docker-compose.yml`, `.env.example`
These define how to run the service and any dependencies. Even if compose only runs one container, show the **container boundary** and config injection via environment variables.

##### Telemetry Producers
- “CI Telemetry Client” (`ci_telemetry_client.py`)
- “GitHub Actions Workflow” (`.github/workflows/ci-with-telemetry.yml`)
- Optionally “Developer curl / local scripts”
##### FastAPI Backend Service
- REST endpoint: `POST /metrics`
- Static file hosting for `/` → `src/static/index.html`
##### SQLite Database
- File-based DB (can be shown as “SQLite file volume” if containerized)
##### Web Dashboard (Browser)
- Loads static HTML/JS (Chart.js)
- Maintains WebSocket connection for live updates
##### Observability (OpenTelemetry)
- If exporter target isn’t defined, label it generically: “OTel Collector / Observability Backend (future: Application Insights)”.
- **Azure Event Hubs** (ingestion source)
- **Application Insights** (traces/metrics/logs)
- **Azure Data Explorer (ADX)** (storage + analytics)

- **API Router / REST Handler**
- Accepts JSON telemetry event (Pydantic validation via `models.py`)
- Calls storage layer
- Triggers broadcast
- **Storage Layer (`storage.py`)**
- Insert event
- Query aggregates (pass/fail counts, avg duration)
- **WebSocket Manager / Broadcaster**
- Tracks connected clients
- Pushes updated summary/events
- **Static Content Server**
- Serves `index.html` + JS assets
- **OTel Middleware/Instrumentation**
- Captures request spans, WS spans (if enabled), DB spans (if instrumented)

This gives you the “as many components as possible” decomposition without guessing beyond the repo.

##### Producer → FastAPI REST
- Payload: `{agent_id, build_id, status, duration_ms, ...}`
- “Pydantic validation (models.py)”
##### Browser → FastAPI
- Optionally “HTTP GET / (static index.html)”
##### CI pipeline → FastAPI

- **Ingestion path**: Producer → API → DB
- **Fan-out path**: API → WebSocket → UI

##### Important architectural patterns to explicitly show

- **Event ingestion + broadcast** pattern
REST for ingestion, WebSocket for real-time fan-out.
- **Layered architecture inside the service**
API layer → validation/models → storage abstraction → DB.
- **Single-writer / multi-reader**
Many producers write events; many dashboards read via WebSocket.
- **Pluggable backends (evolution path)**
SQLite today; ADX tomorrow; Event Hubs as alternate ingestion; Application Insights for observability.

##### System Context (1 slide)
Producers + Browser + Telemetry Dashboard Service + Storage + (Optional Azure).
Browser, FastAPI container, SQLite, OTel exporter, CI producer.
REST `/metrics` handler → storage → broadcast manager → WS clients; static UI serving.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% =========================
    %% System Context
    %% =========================
    subgraph "System Context (Real-time CI build/test telemetry dashboard)"
        direction TB

        subgraph "Telemetry Producers (External)"
            direction TB
            GH["GitHub Actions Workflow\n(CI Producer Pipeline)"]:::external
            CLICONF["CI Telemetry Config/Spec\n(event fields & target)"]:::artifact
            CLICLI["CI Telemetry Client\n(posts build/test events)"]:::external
            SAMPLE["Sample Telemetry Events\n(JSONL examples)"]:::artifact
            DEV["Developer Scripts / curl\n(Optional manual producer)"]:::external
        end

        subgraph "Consumers (External)"
            direction TB
            BROWSER["Web Dashboard (Browser)\n(Chart.js UI)"]:::external
        end

        subgraph "Telemetry Dashboard Service"
            direction TB
            FASTAPI["FastAPI API Server\n(REST + WebSocket + Static Hosting)\n(OTel-enabled variant)"]:::compute
            SQLITE[("SQLite Database\n(file-based persistence)")]:::datastore
            OTELDEST["OTel Collector / Observability Backend\n(future: Application Insights)"]:::observability
        end

        subgraph "Future / Optional Azure Integrations (Pluggable)"
            direction TB
            EH["Azure Event Hubs\n(alt ingestion source)"]:::future
            AI["Application Insights\n(traces/metrics/logs)"]:::future
            ADX["Azure Data Explorer (ADX)\n(storage + analytics)"]:::future
        end
    end

    %% Flows (context)
    GH -->|"runs"| CLICLI
    CLICONF -->|"configures"| CLICLI
    SAMPLE -->|"example_input"| CLICLI

    CLICLI -->|"HTTP POST /metrics\n(build_event JSON)"| FASTAPI
    DEV -->|"HTTP POST /metrics"| FASTAPI

    BROWSER -->|"HTTP GET /\n(static index.html)"| FASTAPI
    BROWSER -->|"WebSocket connect\nWS /ws"| FASTAPI
    FASTAPI -->|"WebSocket push\n(updated metrics/events)"| BROWSER

    FASTAPI -->|"insert_event + query_aggregates"| SQLITE
    FASTAPI -->|"OTel export\n(traces/metrics/logs)"| OTELDEST

    %% Future-state overlay (dashed relationships)
    EH -.->|"future_ingestion_path"| FASTAPI
    FASTAPI -.->|"future_export_target"| AI
    SQLITE -.->|"future_replace_storage"| ADX

    %% =========================
    %% Containerization / Runtime Wiring
    %% =========================
    subgraph "Packaging / Deployment"
        direction TB
        DOCKER["Docker Image Build\n(FastAPI service)"]:::packaging
        COMPOSE["docker-compose\n(local wiring)"]:::packaging
        ENV["Environment Variables\n(.env template)"]:::artifact
    end

    ENV -->|"injects_config"| COMPOSE
    COMPOSE -->|"runs"| DOCKER
    DOCKER -->|"hosts"| FASTAPI
    DOCKER -->|"mounts DB file/volume"| SQLITE

    %% =========================
    %% Component Diagram (Inside FastAPI Service)
    %% =========================
    subgraph "FastAPI Service (Component View)"
        direction TB

        subgraph "HTTP/WS Interface"
            direction TB
            REST["REST Handler\nPOST /metrics\n(telemetry ingestion)"]:::api
            WS["WebSocket Endpoint\nWS /ws\n(broadcast channel)"]:::ws
            STATIC["Static Content Server\n/ -> index.html + assets"]:::frontend
        end

        subgraph "Service Internals"
            direction TB
            MODELS["Schema / Validation\n(Pydantic models)"]:::contract
            STORAGE["Storage Layer\n(persistence + summary queries)"]:::service
            BROADCAST["WebSocket Manager / Broadcaster\n(tracks clients, pushes updates)"]:::service
            OTEL["OpenTelemetry Instrumentation\n(middleware/spans/exporter hooks)"]:::observability
        end

        REST -->|"validate_payload"| MODELS
        REST -->|"insert_event"| STORAGE
        STORAGE -->|"write/read"| SQLITE
        REST -->|"trigger_broadcast"| BROADCAST
        BROADCAST -->|"push updates"| WS

        STATIC -->|"serves UI"| BROWSER
        WS -->|"connected_clients"| BROWSER

        REST -->|"capture_spans"| OTEL
        WS -->|"capture_spans"| OTEL
        STORAGE -->|"DB spans (if enabled)"| OTEL
        OTEL -->|"export"| OTELDEST
    end

    %% =========================
    %% Tests / Documentation (Quality & Contracts)
    %% =========================
    subgraph "Quality & Documentation"
        direction TB
        APIDOC["API Documentation\n(endpoints/contracts)"]:::docs
        ARCHDOC["Architecture Documentation"]:::docs
        SETUPDOC["Setup / Run Documentation"]:::docs

        TESTAPI["Tests: API boundaries\n(REST/WS)"]:::tests
        TESTDB["Tests: DB behavior"]:::tests
        TESTINT["Tests: End-to-end integration"]:::tests
        TESTFIX["Test Fixtures / app setup"]:::tests
    end

    APIDOC -->|"defines"| REST
    ARCHDOC -->|"describes"| FASTAPI
    SETUPDOC -->|"instructions for"| COMPOSE

    TESTFIX -->|"configures"| FASTAPI
    TESTAPI -->|"verifies"| REST
    TESTAPI -->|"verifies"| WS
    TESTDB -->|"verifies"| STORAGE
    TESTINT -->|"verifies_system"| FASTAPI

    %% =========================
    %% Click Events (from component_mapping)
    %% =========================
    click FASTAPI "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/app_with_otel.py"
    click REST "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/app_with_otel.py"
    click WS "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/app_with_otel.py"
    click OTEL "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/app_with_otel.py"
    click MODELS "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/models.py"
    click STORAGE "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/storage.py"
    click SQLITE "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/storage.py"
    click STATIC "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/src/static/index.html"
    click CLICLI "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/ci_telemetry_client.py"
    click GH "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/.github/workflows/ci-with-telemetry.yml"
    click CLICONF "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/ci-telemetry.json"
    click SAMPLE "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/data/sample_events.jsonl"
    click DOCKER "https://github.com/axnasim/realtime-build-telemetry-dashboard/tree/main/Dockerfile"
    click COMPOSE "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/docker-compose.yml"
    click ENV "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/.env.example"
    click APIDOC "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/docs/API.md"
    click ARCHDOC "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/docs/ARCHITECTURE.md"
    click SETUPDOC "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/docs/SETUP.md"
    click TESTAPI "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/tests/test_api.py"
    click TESTDB "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/tests/test_database.py"
    click TESTINT "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/tests/test_integration.py"
    click TESTFIX "https://github.com/axnasim/realtime-build-telemetry-dashboard/blob/main/tests/conftest.py"

    %% =========================
    %% Styles
    %% =========================
    classDef external fill:#E5E7EB,stroke:#374151,stroke-width:1px,color:#111827,rx:8,ry:8
    classDef compute fill:#DBEAFE,stroke:#1D4ED8,stroke-width:1.5px,color:#0B1F5E
    classDef api fill:#BFDBFE,stroke:#1D4ED8,stroke-width:1.5px,color:#0B1F5E
    classDef ws fill:#C7D2FE,stroke:#4338CA,stroke-width:1.5px,color:#111827
    classDef frontend fill:#DCFCE7,stroke:#15803D,stroke-width:1.5px,color:#052E16
    classDef service fill:#E0F2FE,stroke:#0369A1,stroke-width:1.5px,color:#0C4A6E
    classDef datastore fill:#FEF3C7,stroke:#B45309,stroke-width:1.5px,color:#451A03
    classDef observability fill:#FCE7F3,stroke:#BE185D,stroke-width:1.5px,color:#500724
    classDef future fill:#F3F4F6,stroke:#6B7280,stroke-width:1.5px,color:#111827,stroke-dasharray: 6 4
    classDef packaging fill:#FFEDD5,stroke:#C2410C,stroke-width:1.5px,color:#431407
    classDef docs fill:#EDE9FE,stroke:#6D28D9,stroke-width:1.5px,color:#2E1065
    classDef tests fill:#CCFBF1,stroke:#0F766E,stroke-width:1.5px,color:#042F2E
    classDef contract fill:#F1F5F9,stroke:#334155,stroke-width:1.5px,color:#0F172A
    classDef artifact fill:#F8FAFC,stroke:#64748B,stroke-width:1.2px,color:#0F172A,rx:6,ry:6
```
