How mohammed-hatem/weather_monitoring_station works
This is a distributed weather-monitoring product: station simulators and an optional Open-Meteo adapter publish status events to Kafka; the central service consumes them, archives every event to partitioned Parquet, and keeps the latest non-drop reading in BitCask for REST queries. A separate Kafka Streams detector emits rain alerts. README also documents Parquet shipping to Elasticsearch/Kibana. The excerpts show the station, ingestion, storage, REST, and Open-Meteo paths; shipper and detector links follow README descriptions, and infrastructure wiring is not inferred.
Main entry points: KafkaWeatherConsumer.java Weather Consumer · IntegrationConfig.java Open-Meteo Adapter · BitCask.java BitCask Store · WeatherStationApp.java Station Simulator
Components 19
Weather Producers
- Station Simulator
weather-station/src/main/java/com/ddia/weatherstation/WeatherStationApp.java - Station Messages
weather-station/src/main/java/com/ddia/weatherstation/model/WeatherStatusMessage.java - Open-Meteo Adapter
open-meteo-integration/src/main/java/com/ddia/openmeteo/config/IntegrationConfig.java - Virtual Station Models
open-meteo-integration/src/main/java/com/ddia/openmeteo/model/WeatherStatusMessage.java
Kafka Streams
- Rain Detector
kafka-processor/src/main/java/com/ddia/kafkaprocessor/RainDetectorProcessor.java
Central Station
- Weather Consumer
central-station/src/main/java/com/ddia/centralstation/consumer/KafkaWeatherConsumer.java - Weather Status Model
central-station/src/main/java/com/ddia/centralstation/model/WeatherStatusMessage.java - BitCask Store
central-station/src/main/java/com/ddia/centralstation/bitcask/BitCask.java - Key Directory Entry
central-station/src/main/java/com/ddia/centralstation/bitcask/KeyDirEntry.java - Parquet Archiver
central-station/src/main/java/com/ddia/centralstation/archiver/ParquetArchiver.java - BitCask REST API
central-station/src/main/java/com/ddia/centralstation/controller/BitCaskController.java
Analytics
- Parquet Shipper
parquet-shipper/shipper.py
Other components
- Kafka Topics
- Open-Meteo API
- Rain Alerts
- Partitioned Parquet
- BitCask Client
bitcask_client.sh - Elasticsearch
- Kibana
Connections 18
- Station Simulator → Station Messages: builds messages
weather-station/src/main/java/com/ddia/weatherstation/WeatherStationApp.java - Station Simulator → Kafka Topics: publishes status
weather-station/src/main/java/com/ddia/weatherstation/WeatherStationApp.java - Open-Meteo API → Open-Meteo Adapter: provides weather
open-meteo-integration/src/main/java/com/ddia/openmeteo/config/IntegrationConfig.java - Open-Meteo Adapter → Virtual Station Models: transforms response
open-meteo-integration/src/main/java/com/ddia/openmeteo/config/IntegrationConfig.java - Open-Meteo Adapter → Kafka Topics: publishes status
open-meteo-integration/src/main/java/com/ddia/openmeteo/config/IntegrationConfig.java - Kafka Topics → Weather Consumer: delivers status
central-station/src/main/java/com/ddia/centralstation/consumer/KafkaWeatherConsumer.java - Weather Consumer → Weather Status Model: deserializes
central-station/src/main/java/com/ddia/centralstation/consumer/KafkaWeatherConsumer.java - Weather Consumer → BitCask Store: writes latest
central-station/src/main/java/com/ddia/centralstation/consumer/KafkaWeatherConsumer.java - BitCask Store → Key Directory Entry: indexes locations
central-station/src/main/java/com/ddia/centralstation/bitcask/BitCask.java - Weather Consumer → Parquet Archiver: archives events
central-station/src/main/java/com/ddia/centralstation/consumer/KafkaWeatherConsumer.java - Parquet Archiver → Partitioned Parquet: writes batches
central-station/src/main/java/com/ddia/centralstation/archiver/ParquetArchiver.java - BitCask REST API → BitCask Store: queries and compacts
central-station/src/main/java/com/ddia/centralstation/controller/BitCaskController.java - BitCask Client → BitCask REST API: calls endpoints
central-station/src/main/java/com/ddia/centralstation/controller/BitCaskController.java - Kafka Topics → Rain Detector: streams status
README.md - Rain Detector → Rain Alerts: emits rain alerts
README.md - Partitioned Parquet → Parquet Shipper: ships files
README.md - Parquet Shipper → Elasticsearch: indexes batches
README.md - Elasticsearch → Kibana: visualizes data
README.md