How bmd1905/customer-purchase-prediction-ml-system works
Project Type and Purpose
• This project is a full-scale, production-grade MLOps pipeline built around real-time e-commerce behavior data processing.
• It handles data ingestion from Kafka and PostgreSQL CDC, schema validation through Flink, transformation using Spark Streaming, feature storage (both online in Redis and offline in PostgreSQL/DWH), distributed training via Ray and MLflow integration, and modern model serving with Ray Serve and a FastAPI-based feature service. In parallel, observability tools (SigNoz, Prometheus, Grafana, Superset, etc.) and an NGINX proxy provide monitoring, alerting, and secure access.
Establishing Relationships and Data Flow
• Data ingestion flows from Kafka producers and CDC (originating from PostgreSQL changes) into respective Kafka topics.
• Flink’s schema validation subscribes to these topics, then re-publishes to “validated” and “invalid” topics.
• Validated data then flows into two branches: one going to the Data Lake (MinIO) and eventually the Data Warehouse via the Airflow-managed data_pipeline.
• Simultaneously, Spark Streaming jobs read from the “validated” topic, process the data, and push features to both the offline store (PostgreSQL) and online store (Redis).
• From the warehouse/offline store, data is pulled by the Airflow-managed training pipeline, which funnels data into distributed training (Ray Cluster) and logs results into MLflow.
• Post training, the model is loaded via Ray Serve to power the prediction service, while the feature service separately exposes real-time features via FastAPI.
• Observability tools (SigNoz, Grafana, Prometheus) wrap around these pipelines, monitoring metrics, logs, and even Ray Dashboard metrics.
• Integration points like model versioning (MLflow and Model Registry) and the orchestrated deployment (via Docker Compose and Airflow).
Architectural Patterns and Design Principles to Highlight
• Microservices Architecture: Demonstrate how each module (e.g., CDC, feature store, model registry, serving API) operates as an independently containerized service.
• Streaming and Batch Data Processing: Clearly separate the real-time streaming side (Kafka, Flink, Spark Streaming) from the batch/analytical side (Data Warehouse and offline store).
• Distributed and Scalable Training/Serving: Indicate the use of Ray for both distributed training, hyperparameter tuning, and scalable model serving.
• Containerization and Orchestration: Emphasize that all components are containerized, with individual Docker Compose files deployed across a common network.