# monish-k-s/log_analytics_platform architecture

> Architecture diagram of the GitHub repository monish-k-s/log_analytics_platform, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/monish-k-s/log_analytics_platform
- Repository: https://github.com/monish-k-s/log_analytics_platform

## Overview

This is a real-time log observability platform. A scheduled producer generates log events, publishes them to Kafka, and the processing engine consumes them for in-memory analytics, PostgreSQL persistence, and Lucene indexing. REST endpoints expose log queries, analytics, and search. The README also documents Grafana dashboards; their precise data connection is not established by the sampled source, so no runtime link is asserted. Processing and producer paths are sampled; other integrations reflect documented capabilities where noted.

Main entry points: [`processing-engine/src/main/java/com/monish/processing/analytics/LogAnalyticsService.java`](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/analytics/LogAnalyticsService.java) (Analytics service), [`processing-engine/src/main/java/com/monish/processing/controller/LogController.java`](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/controller/LogController.java) (REST controller), [`processing-engine/src/main/java/com/monish/processing/repository/LogRepository.java`](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/repository/LogRepository.java) (Log repository), [`producer-service/src/main/java/com/monish/producer/generator/LogGenerator.java`](https://github.com/monish-k-s/log_analytics_platform/blob/main/producer-service/src/main/java/com/monish/producer/generator/LogGenerator.java) (Log generator)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Log ingestion"]
  node_producer["Log generator<br/>[LogGenerator.java]"]
  node_event["Log event<br/>[LogEvent.java]"]
  node_publisher["Kafka publisher"]
  node_kafka["Kafka topic"]
end

subgraph group_processing["Processing and storage"]
  node_consumer["Log consumer<br/>[LogConsumer.java]"]
  node_analytics["Analytics service"]
  node_entity["Log entity<br/>[LogEntity.java]"]
  node_repository["Log repository<br/>[LogRepository.java]"]
  node_postgres[("PostgreSQL")]
  node_lucene[("Lucene index")]
end

subgraph group_api["Queries and analytics"]
  node_controller["REST controller<br/>[LogController.java]"]
end

subgraph group_observability["Observability"]
  node_grafana["Grafana dashboards"]
end

node_user(("API client"))

node_producer -->|"creates"| node_event
node_producer -->|"publishes via"| node_publisher
node_publisher -->|"sends events"| node_kafka
node_kafka -->|"delivers events"| node_consumer
node_consumer -->|"processes event"| node_analytics
node_analytics -->|"maps event"| node_entity
node_analytics -->|"persists log"| node_repository
node_repository -->|"reads and writes"| node_postgres
node_analytics -->|"indexes log"| node_lucene
node_user -->|"requests"| node_controller
node_controller -->|"queries logs"| node_repository
node_repository -->|"returns logs"| node_controller
node_controller -->|"requests metrics"| node_analytics
node_controller -->|"searches index"| node_lucene

click node_producer "https://github.com/monish-k-s/log_analytics_platform/blob/main/producer-service/src/main/java/com/monish/producer/generator/LogGenerator.java"
click node_event "https://github.com/monish-k-s/log_analytics_platform/blob/main/producer-service/src/main/java/com/monish/common/model/LogEvent.java"
click node_publisher "https://github.com/monish-k-s/log_analytics_platform/blob/main/producer-service/src/main/java/com/monish/producer/service/KafkaProducerService.java"
click node_consumer "https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/consumer/LogConsumer.java"
click node_analytics "https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/analytics/LogAnalyticsService.java"
click node_entity "https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/entity/LogEntity.java"
click node_repository "https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/repository/LogRepository.java"
click node_lucene "https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/service/LuceneIndexService.java"
click node_controller "https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/controller/LogController.java"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_producer,node_event,node_publisher,node_kafka,node_user toneBlue
class node_consumer,node_analytics,node_entity,node_repository,node_postgres,node_lucene toneAmber
class node_controller toneMint
class node_grafana toneRose
```

## Components

### Log ingestion

- [Log generator](https://github.com/monish-k-s/log_analytics_platform/blob/main/producer-service/src/main/java/com/monish/producer/generator/LogGenerator.java): `producer-service/src/main/java/com/monish/producer/generator/LogGenerator.java`
- [Log event](https://github.com/monish-k-s/log_analytics_platform/blob/main/producer-service/src/main/java/com/monish/common/model/LogEvent.java): `producer-service/src/main/java/com/monish/common/model/LogEvent.java`
- [Kafka publisher](https://github.com/monish-k-s/log_analytics_platform/blob/main/producer-service/src/main/java/com/monish/producer/service/KafkaProducerService.java): `producer-service/src/main/java/com/monish/producer/service/KafkaProducerService.java`
- **Kafka topic**

### Processing and storage

- [Log consumer](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/consumer/LogConsumer.java): `processing-engine/src/main/java/com/monish/processing/consumer/LogConsumer.java`
- [Analytics service](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/analytics/LogAnalyticsService.java): `processing-engine/src/main/java/com/monish/processing/analytics/LogAnalyticsService.java`
- [Log entity](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/entity/LogEntity.java): `processing-engine/src/main/java/com/monish/processing/entity/LogEntity.java`
- [Log repository](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/repository/LogRepository.java): `processing-engine/src/main/java/com/monish/processing/repository/LogRepository.java`
- **PostgreSQL**
- [Lucene index](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/service/LuceneIndexService.java): `processing-engine/src/main/java/com/monish/processing/service/LuceneIndexService.java`

### Queries and analytics

- [REST controller](https://github.com/monish-k-s/log_analytics_platform/blob/main/processing-engine/src/main/java/com/monish/processing/controller/LogController.java): `processing-engine/src/main/java/com/monish/processing/controller/LogController.java`

### Observability

- **Grafana dashboards**

### Other components

- **API client**

## Connections

- Log generator → Log event: creates
- Log generator → Kafka publisher: publishes via
- Kafka publisher → Kafka topic: sends events
- Kafka topic → Log consumer: delivers events
- Log consumer → Analytics service: processes event
- Analytics service → Log entity: maps event
- Analytics service → Log repository: persists log
- Log repository → PostgreSQL: reads and writes
- Analytics service → Lucene index: indexes log
- API client → REST controller: requests
- REST controller → Log repository: queries logs
- Log repository → REST controller: returns logs
- REST controller → Analytics service: requests metrics
- REST controller → Lucene index: searches index