# carverauto/serviceradar architecture

> Architecture diagram of the GitHub repository carverauto/serviceradar, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/carverauto/serviceradar
- Repository: https://github.com/carverauto/serviceradar
- Video tour (about a minute): https://gitdiagram.com/carverauto/serviceradar/video

## Overview

##### Project Type and Purpose
- ServiceRadar is a full-stack, distributed network and service monitoring system.
- It comprises edge components (Agents, Checkers, Pollers), a centralized Core service (API, rule engine, alerting, stream processing), and a Web UI.
- It uses Go for most backend microservices, Rust for high-performance rule engines and checkers, Next.js/TypeScript for the dashboard, NATS for messaging, gRPC/HTTP for RPC, and Timeplus Proton/ClickHouse for stream storage.

##### Main Components
a. Edge Components
- Agent (cmd/agent, pkg/agent)
- Poller (cmd/poller, cmd/poller-ng, pkg/poller)
- Specialized Checkers (cmd/checkers/{dusk, rperf-client/server, snmp, sysmon})
b. Messaging & Consumers
- NATS broker (packaging/nats, pkg/natsutil)
- Consumers (cmd/consumers/{db-event-writer, netflow, zen}, pkg/consumers)
- Log forwarder Flowgger (cmd/flowgger)
- OpenTelemetry collector (cmd/otel, pkg/logger, pkg/http, proto/opentelemetry)
c. Core Service
- API & Rule Engine (cmd/core, pkg/core)
- Database layer (pkg/db, migrations)
- Model definitions (pkg/models)
- SRQL parser & translator (pkg/srql)
- Metrics store (pkg/metricstore), metrics manager (pkg/metrics)
- Alerts & notifications (Discord, Webhooks)
d. Web UI
- Next.js app (web/src/app, web/src/components) served behind Nginx (packaging/web/nginx.conf)
- Communicates over HTTP to Core API
e. Packaging & Deployment
- Dockerfiles (cmd/*, docker/, packaging/*)
- Kubernetes manifests (k8s/demo)
- install-serviceradar.sh, Makefile, Goreleaser config

##### Component Interactions
- Agent exposes gRPC endpoints for local service status.
- Poller connects to Agents via mTLS-secured gRPC, invokes Checkers for SNMP, ICMP, Sysmon, RPerf, Dusk, then publishes results to NATS or directly to Core.
- Core subscribes to NATS topics or accepts gRPC calls from Pollers, processes events through the Rust-based rule engine, writes metrics/logs to Timeplus Proton (ClickHouse), and persists device/poller state in a relational DB.
- Consumers (db-event-writer, netflow, zen) subscribe to NATS topics, transform or enrich events, then write to Core’s database or external sinks.
- Flowgger ingests logs via file/UDP/TCP/Redis, encodes them (Cap’n Proto, GELF, etc.), and publishes to NATS or Core.
- OpenTelemetry agent collects traces/logs/metrics from microservices, forwards to Core or third-party backends.
- Web UI issues REST calls to Core’s HTTP API (behind Nginx). Authentication via API key or JWT.

##### Architectural Patterns & Principles
- Microservices: each major function (agent, poller, checker, core, consumer) is a separate service.
- Event-driven / Publish-Subscribe: NATS is the central message bus.
- Stream Processing: Timeplus Proton (Rust) for high-throughput OLAP, ClickHouse as storage.
- Layered Architecture: edge layer (polling, data collection), service layer (rule engine, API), data layer (stream store, relational DB).
- Polyglot: Go for orchestration and API, Rust for performance-critical components, TypeScript for UI.
- Security: mTLS between services, API key/JWT for UI, TLS for NATS.

##### Essential Architectural Elements to Capture
- Security boundary showing mTLS and API authentication.
- Message bus as central backbone for decoupling.
- Storage tier with both OLAP (Proton/ClickHouse) and OLTP (Core DB).
- UI layer with reverse proxy (Nginx) and API gateway.
- Extensible plugin/checker architecture: illustrate how a new checker module fits.
- Deployment descriptors (Docker/K8s) should appear as annotations.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Edge Layer
    subgraph "Edge Layer"
        direction TB
        Agent["Agent (gRPC)"]:::goService
        Poller["Poller"]:::goService
        subgraph "Checkers"
            direction TB
            Dusk["Dusk Checker"]:::goService
            SNMP["SNMP Checker"]:::goService
            Sysmon["Sysmon Checker"]:::rustService
            RPerf["RPerf Checker"]:::rustService
        end
        Flowgger["Flowgger (Log Forwarder)"]:::rustService
        OTel["OpenTelemetry Collector"]:::rustService
    end

    %% Messaging Layer
    subgraph "Messaging"
        direction TB
        NATS["NATS Broker"]:::message
    end

    %% Core Layer
    subgraph "Core Layer"
        direction TB
        CoreAPI["Core API & Rule Engine"]:::goService
        subgraph "Consumers"
            direction TB
            DBWriter["DB Event Writer"]:::goService
            Netflow["Netflow Consumer"]:::goService
            Zen["Zen Consumer"]:::rustService
        end
        subgraph "Storage"
            direction TB
            OLTP["Core DB (Relational)"]:::db
            OLAP["Proton/ClickHouse"]:::db
        end
    end

    %% Frontend Layer
    subgraph "Frontend Layer"
        direction TB
        Nginx["Nginx (Reverse Proxy)"]:::external
        WebUI["Web UI (Next.js)"]:::ui
    end

    %% Connections
    Agent -->|"mTLS gRPC"| Poller
    Poller -->|"invoke"| Dusk
    Poller -->|"invoke"| SNMP
    Poller -->|"invoke"| Sysmon
    Poller -->|"invoke"| RPerf
    Poller -->|"publish"| NATS
    Flowgger -->|"publish"| NATS
    OTel -->|"publish"| NATS
    NATS -->|"subscribe"| CoreAPI
    NATS -->|"subscribe"| DBWriter
    NATS -->|"subscribe"| Netflow
    NATS -->|"subscribe"| Zen
    CoreAPI -->|"write"| OLTP
    CoreAPI -->|"write metrics"| OLAP
    DBWriter -->|"write"| OLTP
    Netflow -->|"write"| OLTP
    Zen -->|"write"| OLTP
    WebUI -->|"HTTP/HTTPS"| Nginx
    Nginx -->|"proxy"| CoreAPI

    %% Click Events - Edge
    click Agent "https://github.com/carverauto/serviceradar/blob/main/cmd/agent/main.go"
    click Agent "https://github.com/carverauto/serviceradar/tree/main/pkg/agent"
    click Poller "https://github.com/carverauto/serviceradar/blob/main/cmd/poller/main.go"
    click Poller "https://github.com/carverauto/serviceradar/tree/main/cmd/poller-ng/src"
    click Poller "https://github.com/carverauto/serviceradar/tree/main/pkg/poller"
    click Dusk "https://github.com/carverauto/serviceradar/blob/main/cmd/checkers/dusk/main.go"
    click Dusk "https://github.com/carverauto/serviceradar/tree/main/pkg/checker/dusk"
    click SNMP "https://github.com/carverauto/serviceradar/blob/main/cmd/checkers/snmp/main.go"
    click SNMP "https://github.com/carverauto/serviceradar/tree/main/pkg/checker/snmp"
    click Sysmon "https://github.com/carverauto/serviceradar/tree/main/cmd/checkers/sysmon"
    click RPerf "https://github.com/carverauto/serviceradar/tree/main/cmd/checkers/rperf-client"
    click RPerf "https://github.com/carverauto/serviceradar/tree/main/cmd/checkers/rperf-server"
    click NATS "https://github.com/carverauto/serviceradar/tree/main/packaging/nats"
    click NATS "https://github.com/carverauto/serviceradar/blob/main/pkg/natsutil/events.go"
    click Flowgger "https://github.com/carverauto/serviceradar/tree/main/cmd/flowgger"
    click OTel "https://github.com/carverauto/serviceradar/tree/main/cmd/otel"
    click OTel "https://github.com/carverauto/serviceradar/tree/main/pkg/logger"
    click OTel "https://github.com/carverauto/serviceradar/tree/main/pkg/http"
    click OTel "https://github.com/carverauto/serviceradar/tree/main/proto/opentelemetry"

    %% Click Events - Core
    click CoreAPI "https://github.com/carverauto/serviceradar/blob/main/cmd/core/main.go"
    click CoreAPI "https://github.com/carverauto/serviceradar/tree/main/pkg/core"
    click OLTP "https://github.com/carverauto/serviceradar/tree/main/pkg/db"
    click OLTP "https://github.com/carverauto/serviceradar/tree/main/pkg/db/migrations"
    click OLTP "https://github.com/carverauto/serviceradar/tree/main/pkg/models"
    click CoreAPI "https://github.com/carverauto/serviceradar/tree/main/pkg/srql"
    click CoreAPI "https://github.com/carverauto/serviceradar/tree/main/pkg/metricstore"
    click CoreAPI "https://github.com/carverauto/serviceradar/tree/main/pkg/metrics"

    %% Click Events - Consumers
    click DBWriter "https://github.com/carverauto/serviceradar/blob/main/cmd/consumers/db-event-writer/main.go"
    click DBWriter "https://github.com/carverauto/serviceradar/tree/main/pkg/consumers/db-event-writer"
    click Netflow "https://github.com/carverauto/serviceradar/blob/main/cmd/consumers/netflow/main.go"
    click Netflow "https://github.com/carverauto/serviceradar/tree/main/pkg/consumers/netflow"
    click Zen "https://github.com/carverauto/serviceradar/tree/main/cmd/consumers/zen"
    click Zen "https://github.com/carverauto/serviceradar/tree/main/pkg/consumers"

    %% Click Events - Frontend
    click WebUI "https://github.com/carverauto/serviceradar/tree/main/web/src/app"
    click WebUI "https://github.com/carverauto/serviceradar/tree/main/web/src/components"
    click Nginx "https://github.com/carverauto/serviceradar/blob/main/packaging/web/config/nginx.conf"

    %% Styles
    classDef goService fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef rustService fill:#FFA500,stroke:#000,stroke-width:1px
    classDef ui fill:#90EE90,stroke:#000,stroke-width:1px
    classDef message fill:#D8BFD8,stroke:#000,stroke-width:1px
    classDef db fill:#D3D3D3,stroke:#000,stroke-width:1px
    classDef external fill:#E0E0E0,stroke:#000,stroke-width:1px
```
