niklasfrick/spark-dashboard

Purpose and Entry Points

Spark Dashboard is a Linux/NVIDIA monitoring service for host hardware and vLLM inference engines. The Rust binary is the production entry point; it serves the embedded React dashboard, REST configuration API, and WebSockets. Key paths: src/main.rs, src/server.rs, src/cli/service.rs.

Backend Runtime and Orchestration

main parses CLI/environment configuration, initializes shared state, and starts independent Tokio collector tasks plus Axum. Collectors publish snapshots through a bounded broadcast channel so slow/disconnected browser clients do not control collection. Key paths: src/main.rs, src/metrics/mod.rs, src/engines/mod.rs.

Host Metrics Collection

The hardware collector polls at a configurable interval (default 1 second), assembling CPU, memory, disk, network, and multi-GPU snapshots. GPU collection relies on NVIDIA/NVML; CPU/system counters use Linux-facing sources such as sysinfo/procfs. It also supports GPU simulation for development and unified-memory detection. Key paths: src/metrics/gpu.rs, src/metrics/cpu.rs, src/metrics/memory.rs.

Engine Discovery and vLLM Monitoring

Engine collection is separate from hardware collection. It discovers vLLM processes and, when Docker socket access exists, vLLM containers; explicit engine URLs can override discovery. The vLLM adapter polls Prometheus-format metrics and tracks engine lifecycle/state, model metadata, latency, throughput, requests, and cache metrics. Key paths: src/engines/detector.rs, src/engines/vllm.rs, src/engines/prometheus.rs.

Delivery, API, and Persistence Boundaries

Axum exposes /ws for metric streams and /api/dashboard for a shared dashboard document. The backend treats configuration as opaque, size-limited bytes; config_store handles atomic persistence and read-only-state behavior. Optional /ws/logs uses Docker APIs to fan out logs only for tracked container engines. Production static assets are embedded; unmatched API routes remain API 404s rather than falling through to the SPA. Key paths: src/server.rs, src/ws.rs, src/config_store.rs.

React Dashboard Data Flow

The Vite/React client opens the WebSocket, reconnects on failure, batches received snapshots, and retains rolling history in circular buffers. Dashboard renders host metrics, multi-engine aggregate/per-engine tabs, charts, gauges, controls, and optional logs. Dashboard layout/configuration is loaded and saved through the REST API, with schema/migration/preset logic entirely client-side. Key paths: frontend/src/main.tsx, frontend/src/hooks/useMetrics.ts, frontend/src/components/views/Dashboard.tsx.

Deployment and External Dependencies

Production supports a systemd-managed host binary and a GPU-enabled Docker image. The systemd service provisions a restricted service user and persistent state directory; Docker requires NVIDIA Container Toolkit and optionally mounts /var/run/docker.sock for container discovery/logs. External integrations are NVIDIA drivers/NVML, Docker Engine API via Bollard, vLLM HTTP/Prometheus endpoints, and optional authenticated engine APIs. Key paths: deploy/host/systemd/spark-dashboard.service, deploy/docker/Dockerfile, deploy/docker/docker-compose.yml.