How porta0922/monitorv3 works
1. System shape
- This repo is a distributed monitoring platform with three runtime pieces: a Rust endpoint agent, a Rust backend server, and a React/TypeScript dashboard.
- The core architectural pattern is: capture on endpoint → publish to RabbitMQ → persist in PostgreSQL/TimescaleDB → query via HTTP/WebSocket from the dashboard.
2. Endpoint agent boundary
- The agent/ crate is responsible for local data collection on Windows, Linux, and macOS.
- It tracks active application/use state, keyboard and mouse activity, USB events, Wi‑Fi events, installed software inventory, and process/security signals.
- The Windows design supports a hybrid mode: a system service plus a user-session component so it can capture both background and foreground activity.
3. Agent internal subsystems
- The agent is split into focused modules: input_tracking, keystroke_tracker, usb_detection, usb_file_copy_detection, wifi_detection, inventory, monitoring, osquery_runner, process_protection, offline_cache, rabbitmq_publisher, and device_id.
- device_id provides stable endpoint identity; process_protection keeps the agent harder to terminate; offline_cache stores events locally in SQLite when the network or broker is unavailable.
- osquery_runner is the security-scan engine, but it is optional and silent if osquery is absent.
4. Event transport layer
- The agent publishes telemetry to RabbitMQ using the monitoring topic exchange.
- The backend consumes domain-specific queues for activity, heartbeat, USB, inventory, Wi‑Fi, and security data.
- This decouples endpoint collection from server persistence and allows the agent to keep running even when the backend is temporarily unreachable.
5. Backend server boundary
- The server/ crate is the central integration layer: it exposes HTTP APIs, consumes RabbitMQ, and writes to the database.
- api.rs and ws.rs serve client traffic; rabbitmq_consumer.rs ingests broker messages; postgres_db.rs and db.rs handle persistence access.
- auth.rs, whitelist.rs, and config.rs define the control plane around access, allowed clients/devices, and runtime settings.
6. Persistence layer
- PostgreSQL/TimescaleDB is the primary durable store, with schema created by the SQL migrations under migrations/.
- The database holds the device catalog, logs, hourly aggregates, USB/Wi‑Fi history, and the security event tables introduced by later migrations.
- TimescaleDB matters here because the workload is largely time-series telemetry and historical rollups.
7. Security analytics flow
- Security findings come from two places: osquery-based detections mapped to MITRE ATT&CK and USB-copy detection based on recently written files on removable drives.
- These findings are published as security events, deduplicated, persisted, and then exposed through /api/security and /api/security/summary.
- The agent can also fetch an osquery policy from the server, so security scan frequency can be centrally controlled per fleet.
8. Real-time delivery to the UI
- The server’s WebSocket layer provides live activity delivery, which is what powers the dashboard’s streaming views.
- The dashboard’s useActivityStream hook and live tables are the main consumers of this path, complementing the REST API for historical data.
9. Dashboard architecture
- dashboard/ is a Vite + React + TypeScript SPA with feature pages for dashboard, activity, alerts, devices, inventory, metrics, security, heatmaps, and USB.
- It uses a small API client plus feature-specific hooks (useOverview, useTopApps, useAuth, useActivityStream) instead of direct server coupling in components.
- This makes the UI a thin presentation layer over the server’s REST and WebSocket interfaces.
10. Deployment and runtime environments
- docker-compose.yml provisions the local infrastructure stack, including RabbitMQ and PostgreSQL/TimescaleDB, so the agent/server separation can be run consistently.
- The repo also includes OS-specific install scripts in deploy/ and Dockerfiles for the server and dashboard, reflecting that the agent is meant to be deployed onto endpoints, not just containers.
- Windows install scripts support service-based installation, user-task installation, or the hybrid mode described in the README.
11. Cross-cutting boundary: offline resilience
- The agent’s SQLite offline cache is the main resilience mechanism: it buffers telemetry when broker connectivity is lost and later replays it.
- This means the architecture tolerates intermittent connectivity at the edge without losing local endpoint data, while still preserving the server as the system of record.
12. Common model alignment
- The root-level shared_types.rs suggests shared DTOs or event schemas used to keep agent, server, and dashboard aligned on payload structure.
- That shared contract is important because the system moves typed telemetry across process, network, and storage boundaries.
Main entry points: src/main.rs Agent entry · rabbitmq_publisher.rs Broker publish · src/main.rs Server entry · rabbitmq_consumer.rs Broker consume
Components 25
Endpoint agent
Rust agent that collects local telemetry and security signals on managed devices.
- Agent entry · Initializes endpoint collection, resilience, and publishing on each device.
agent/src/main.rs - Device ID · Generates stable endpoint identity for telemetry and device catalog matching.
agent/src/device_id.rs - Input tracking · Captures active use state and keyboard/mouse activity.
agent/src/input_tracking.rs - Inventory · Collects installed software and endpoint inventory data.
agent/src/inventory.rs - USB events · Detects removable media and USB-related activity.
agent/src/usb_detection.rs - USB copy · Flags suspicious file writes to removable drives.
agent/src/usb_file_copy_detection.rs - Wi-Fi events · Tracks wireless network changes and connectivity events.
agent/src/wifi_detection.rs - Process guard · Hardens the agent against termination and disruption.
agent/src/process_protection.rs - Offline cache · Stores telemetry locally when RabbitMQ is unavailable and replays later.
agent/src/offline_cache.rs - Osquery scan · Optional policy-driven osquery execution for endpoint security findings.
agent/src/osquery_runner.rs - Broker publish · Publishes endpoint telemetry to the monitoring exchange on RabbitMQ.
agent/src/rabbitmq_publisher.rs
Backend server
Rust service that consumes broker events, persists them, and serves client APIs.
- Server entry · Starts HTTP, WebSocket, broker consumption, and persistence services.
server/src/main.rs - HTTP API · Serves historical data, control-plane endpoints, and security queries.
server/src/api.rs - WebSocket · Streams live activity updates to dashboard clients.
server/src/ws.rs - Broker consume · Consumes domain queues for activity, inventory, USB, Wi-Fi, heartbeat, and security events.
server/src/rabbitmq_consumer.rs - Postgres store · Database access layer for durable telemetry, aggregates, and security records.
server/src/postgres_db.rs - Auth & config · Handles authentication, whitelisting, and runtime settings for the server.
server/src/auth.rs
Dashboard
React/TypeScript SPA for operators and live monitoring.
- Dashboard app · React application shell and page composition for the operator UI.
dashboard/src/App.tsx - API client · Thin client wrapper for REST calls into the server.
dashboard/src/api/client.ts - Live stream · Consumes WebSocket activity updates for streaming views.
dashboard/src/hooks/useActivityStream.ts - Feature pages · Pages for dashboard, activity, alerts, devices, inventory, metrics, security, heatmaps, and USB views.
dashboard/src/pages/DashboardPage.tsx
Persistence
Database schema and migrations for time-series telemetry and security history.
- Migrations · Creates and evolves the PostgreSQL/TimescaleDB schema.
migrations/001_init_schema.sql
Deployment
Local runtime and install artifacts for running the stack across environments.
- Compose stack · Local infrastructure for RabbitMQ and PostgreSQL/TimescaleDB.
docker-compose.yml - Install scripts · OS-specific installation and diagnostic scripts for endpoint rollout.
deploy/install-windows.bat
Shared contract
Common DTOs and event schemas used across the repo.
- Shared types · Common DTOs and event schemas shared across agent, server, and dashboard.
shared_types.rs
Connections 35
- Agent entry → Device ID: identity · Endpoint startup binds a stable device identity.
- Agent entry → Input tracking: collects · Main agent orchestrates local telemetry capture.
- Agent entry → Inventory: collects · Main agent orchestrates inventory discovery.
- Agent entry → USB events: collects · Main agent orchestrates USB monitoring.
- Agent entry → Wi-Fi events: collects · Main agent orchestrates Wi-Fi monitoring.
- Agent entry → Process guard: hardens · Startup enables process survivability measures.
- Agent entry → Offline cache: buffers · Telemetry can be staged locally when the broker is unavailable.
- Agent entry → Osquery scan: optional scan · Security scans run only when osquery is available.
- Input tracking → Broker publish: events · Captured activity is forwarded for publication.
- Inventory → Broker publish: events · Inventory snapshots are forwarded for publication.
- USB events → USB copy: feeds · USB activity informs suspicious copy detection.
- USB copy → Broker publish: security events · Removable-drive write findings are published as security telemetry.
- Wi-Fi events → Broker publish: events · Wi-Fi changes are forwarded for publication.
- Offline cache → Broker publish: replay · Buffered telemetry is replayed after connectivity returns.
- Broker publish → Broker consume: monitoring exchange · Endpoint telemetry crosses the broker boundary into server ingestion.
- Broker consume → Postgres store: persists · Consumed events are written to the database.
- Server entry → HTTP API: serves · Server entry exposes HTTP endpoints.
- Server entry → WebSocket: streams · Server entry exposes live updates over WebSocket.
- Server entry → Broker consume: ingests · Server starts broker consumers for domain queues.
- Server entry → Auth & config: controls · Server loads auth, whitelist, and runtime settings.
- HTTP API → Postgres store: queries · REST endpoints read historical and summary data.
- WebSocket → Broker consume: live feed · WebSocket delivery is backed by incoming activity events.
- Migrations → Postgres store: schema · Migrations define the durable tables used by the server.
- Shared types → Agent entry: contracts · Shared schemas keep telemetry payloads aligned.
- Shared types → Server entry: contracts · Shared schemas keep server ingestion and APIs aligned.
- Shared types → Dashboard app: contracts · Shared types keep the UI aligned with server payloads.
- Dashboard app → API client: REST · Pages and components call through the API client.
- Dashboard app → Live stream: realtime · Shell composes live activity views.
- Dashboard app → Feature pages: routes · The SPA is organized into feature pages.
- Live stream → WebSocket: subscribes · The hook consumes server WebSocket activity.
- API client → HTTP API: requests · Frontend REST calls target the HTTP API.
- Feature pages → API client: data · Feature pages load historical data through the client.
- Compose stack → Broker consume: infra · Local stack brings up broker-backed ingestion.
- Compose stack → Postgres store: infra · Local stack brings up the database used by the server.
- Install scripts → Agent entry: install · Deployment scripts install the endpoint agent on managed devices.