# ductrongx/log-monitoring-system architecture

> Architecture diagram of the GitHub repository ductrongx/log-monitoring-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ductrongx/log-monitoring-system
- Repository: https://github.com/ductrongx/log-monitoring-system

## Overview

Log Monitoring System centralizes application logs for real-time processing, search, monitoring, and response. Applications submit logs through the ingestion API; ingestion validates access, sanitizes and publishes raw-log events, which workers normalize and store while emitting live, critical-alert, and anomaly signals. Detection and alerting lead to notifications and optional AI-assisted incident investigation; engineers use APIs and the dashboard for search, analytics, identity, and retention management. Paths for unsampled subsystems identify existing module directories; their internal wiring is shown only where documented.

Main entry points: [`apps/backend/src/main/java/com/vdt/log_monitoring/modules/alerting/internal`](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/alerting/internal) (Alert evaluation), [`apps/backend/src/main/java/com/vdt/log_monitoring/modules/processing/internal`](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/processing/internal) (Processing worker), [`apps/backend/src/main/java/com/vdt/log_monitoring/modules/anomaly/internal`](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/anomaly/internal) (Anomaly detection), [`apps/frontend/src/features/auth/auth-storage.ts`](https://github.com/ductrongx/log-monitoring-system/blob/main/apps/frontend/src/features/auth/auth-storage.ts) (Monitoring dashboard)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_intake["Ingestion and processing"]
  node_ingestion_api["Log ingestion API"]
  node_ingestion["Ingestion service"]
  node_raw_queue["Raw log queue"]
  node_processing["Processing worker"]
  node_clickhouse[("ClickHouse logs")]
  node_live_queue["Live log events"]
end

subgraph group_detection["Detection and response"]
  node_critical_queue["Critical alert events"]
  node_signal_queue["Anomaly signals"]
  node_anomaly["Anomaly detection"]
  node_anomaly_db[("Anomaly reports")]
  node_anomaly_events["Detected events"]
  node_alerting["Alert evaluation"]
  node_alert_db[("Alert state")]
  node_incident["Incident investigation"]
  node_ai["Incident AI service"]
end

subgraph group_experience["Query and operations"]
  node_websocket["WebSocket delivery"]
  node_dashboard["Monitoring dashboard<br/>[auth-storage.ts]"]
  node_analytics["Search and analytics"]
  node_retention["Retention policies"]
end

subgraph group_identity["Identity and access"]
  node_identity["Identity and access API"]
  node_identity_module["Identity services"]
  node_prom_api["Prometheus target API"]
end

node_apps(("Application sources"))
node_prometheus["Prometheus metrics"]
node_redis[("Redis state")]
node_telegram["Telegram"]
node_postgres[("PostgreSQL")]

node_apps -->|"submit logs"| node_ingestion_api
node_ingestion_api -->|"invoke facade"| node_ingestion
node_ingestion -->|"verify API key"| node_identity_module
node_ingestion -->|"publish raw logs"| node_raw_queue
node_raw_queue -->|"consume logs"| node_processing
node_processing -->|"store processed logs"| node_clickhouse
node_processing -->|"publish live events"| node_live_queue
node_processing -->|"publish critical events"| node_critical_queue
node_processing -->|"publish anomaly signals"| node_signal_queue
node_signal_queue -->|"consume signals"| node_anomaly
node_anomaly -->|"query metrics"| node_prometheus
node_anomaly -->|"store reports"| node_anomaly_db
node_anomaly -->|"publish detections"| node_anomaly_events
node_anomaly_events -->|"consume detections"| node_alerting
node_critical_queue -->|"consume alerts"| node_alerting
node_alerting -->|"store occurrences"| node_alert_db
node_alerting -->|"deduplicate alerts"| node_redis
node_alerting -.->|"send notifications"| node_telegram
node_alerting -->|"publish alert events"| node_websocket
node_live_queue -->|"deliver live logs"| node_websocket
node_websocket -->|"push events"| node_dashboard
node_dashboard -->|"search logs"| node_analytics
node_analytics -->|"query logs"| node_clickhouse
node_dashboard -->|"management requests"| node_identity
node_identity -->|"invoke services"| node_identity_module
node_identity_module -->|"persist identity"| node_postgres
node_prom_api -->|"provide scrape targets"| node_prometheus
node_anomaly_events -.->|"trigger investigation"| node_incident
node_incident -->|"request analysis"| node_ai
node_ai -->|"return analysis"| node_incident
node_dashboard -->|"manage retention"| node_retention
node_retention -->|"read policies and runs"| node_postgres
node_retention -->|"delete expired logs"| node_clickhouse

click node_ingestion_api "https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/ingestion/LogIngestionController.java"
click node_ingestion "https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/ingestion/internal/LogIngestionService.java"
click node_processing "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/processing/internal"
click node_anomaly "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/anomaly/internal"
click node_alerting "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/alerting/internal"
click node_websocket "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/realtime"
click node_dashboard "https://github.com/ductrongx/log-monitoring-system/blob/main/apps/frontend/src/features/auth/auth-storage.ts"
click node_analytics "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/analytics/internal"
click node_incident "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/incident/internal"
click node_ai "https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/incident/internal/ai/IncidentAnomalyReportAiService.java"
click node_identity "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/identity"
click node_identity_module "https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/identity/internal"
click node_retention "https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/retention/internal/RetentionService.java"
click node_prom_api "https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/identity/PrometheusTargetController.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingestion_api,node_ingestion,node_raw_queue,node_processing,node_clickhouse,node_live_queue toneBlue
class node_critical_queue,node_signal_queue,node_anomaly,node_anomaly_db,node_anomaly_events,node_alerting,node_alert_db,node_incident,node_ai,node_redis,node_postgres toneAmber
class node_websocket,node_dashboard,node_analytics,node_retention toneMint
class node_identity,node_identity_module,node_prom_api toneRose
class node_apps,node_prometheus,node_telegram toneIndigo
```

## Components

### Ingestion and processing

- [Log ingestion API](https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/ingestion/LogIngestionController.java): `apps/backend/src/main/java/com/vdt/log_monitoring/api/ingestion/LogIngestionController.java`
- [Ingestion service](https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/ingestion/internal/LogIngestionService.java): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/ingestion/internal/LogIngestionService.java`
- **Raw log queue**
- [Processing worker](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/processing/internal): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/processing/internal`
- **ClickHouse logs**
- **Live log events**

### Detection and response

- **Critical alert events**
- **Anomaly signals**
- [Anomaly detection](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/anomaly/internal): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/anomaly/internal`
- **Anomaly reports**
- **Detected events**
- [Alert evaluation](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/alerting/internal): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/alerting/internal`
- **Alert state**
- [Incident investigation](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/incident/internal): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/incident/internal`
- [Incident AI service](https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/incident/internal/ai/IncidentAnomalyReportAiService.java): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/incident/internal/ai/IncidentAnomalyReportAiService.java`

### Query and operations

- [WebSocket delivery](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/realtime): `apps/backend/src/main/java/com/vdt/log_monitoring/api/realtime`
- [Monitoring dashboard](https://github.com/ductrongx/log-monitoring-system/blob/main/apps/frontend/src/features/auth/auth-storage.ts): `apps/frontend/src/features/auth/auth-storage.ts`
- [Search and analytics](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/analytics/internal): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/analytics/internal`
- [Retention policies](https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/retention/internal/RetentionService.java): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/retention/internal/RetentionService.java`

### Identity and access

- [Identity and access API](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/identity): `apps/backend/src/main/java/com/vdt/log_monitoring/api/identity`
- [Identity services](https://github.com/ductrongx/log-monitoring-system/tree/main/apps/backend/src/main/java/com/vdt/log_monitoring/modules/identity/internal): `apps/backend/src/main/java/com/vdt/log_monitoring/modules/identity/internal`
- [Prometheus target API](https://github.com/ductrongx/log-monitoring-system/blob/main/apps/backend/src/main/java/com/vdt/log_monitoring/api/identity/PrometheusTargetController.java): `apps/backend/src/main/java/com/vdt/log_monitoring/api/identity/PrometheusTargetController.java`

### Other components

- **Application sources**
- **Prometheus metrics**
- **Redis state**
- **Telegram**
- **PostgreSQL**

## Connections

- Application sources → Log ingestion API: submit logs
- Log ingestion API → Ingestion service: invoke facade
- Ingestion service → Identity services: verify API key
- Ingestion service → Raw log queue: publish raw logs
- Raw log queue → Processing worker: consume logs
- Processing worker → ClickHouse logs: store processed logs
- Processing worker → Live log events: publish live events
- Processing worker → Critical alert events: publish critical events
- Processing worker → Anomaly signals: publish anomaly signals
- Anomaly signals → Anomaly detection: consume signals
- Anomaly detection → Prometheus metrics: query metrics
- Anomaly detection → Anomaly reports: store reports
- Anomaly detection → Detected events: publish detections
- Detected events → Alert evaluation: consume detections
- Critical alert events → Alert evaluation: consume alerts
- Alert evaluation → Alert state: store occurrences
- Alert evaluation → Redis state: deduplicate alerts
- Alert evaluation → Telegram: send notifications
- Alert evaluation → WebSocket delivery: publish alert events
- Live log events → WebSocket delivery: deliver live logs
- WebSocket delivery → Monitoring dashboard: push events
- Monitoring dashboard → Search and analytics: search logs
- Search and analytics → ClickHouse logs: query logs
- Monitoring dashboard → Identity and access API: management requests
- Identity and access API → Identity services: invoke services
- Identity services → PostgreSQL: persist identity
- Prometheus target API → Prometheus metrics: provide scrape targets
- Detected events → Incident investigation: trigger investigation
- Incident investigation → Incident AI service: request analysis
- Incident AI service → Incident investigation: return analysis
- Monitoring dashboard → Retention policies: manage retention
- Retention policies → PostgreSQL: read policies and runs
- Retention policies → ClickHouse logs: delete expired logs