How decodingml/personalized-recommender-course works
And Label Main Components
• Data / Raw Data Source: Represent the raw input data (H&M dataset) referenced in "recsys/raw_data_sources" and the external Kaggle dataset.
• Feature Engineering Layer: Show the “recsys/features” module (articles, customers, embeddings, interactions, ranking, transactions) where the key data transformation and feature computations occur.
• Inference / Serving Module: Illustrate “recsys/inference” that contains prediction components (e.g., ranking_predictor, query_transformer) along with the Hopsworks integration endpoints in “recsys/hopsworks_integration” (which include dedicated serving scripts such as two_tower_serving.py, ranking_serving.py, and LLM ranking components). This layer is responsible for the live, online recommendations.
• User Interface (UI): Include the “recsys/ui” module and the standalone “streamlit_app.py” entry point for the web-based interface where users interact with the system.
• Configuration and Deployment: Add the configuration file (“recsys/config.py”) as the central settings hub, plus the deployment scripts and automation elements (Makefile, GitHub Actions in “.github/workflows” and “.github/actions”, and Docker/Devcontainer configurations) that help with environment setup and CI/CD.
Interactions and Data Flow
• From the feature layer, show the processed data flowing into the training module where the two-tower and ranking models get trained.
Architectural Patterns / Design Principles
• Modular Design: Clearly separate “features,” “training,” “inference,” and “UI” into different modules, showcasing a clean separation of concerns.
• MLOps Best Practices: Mention the presence of CI/CD pipelines (GitHub Actions, Makefile) and configuration management (.env.example, pyproject.toml) that support scalable deployment.
Relevant Technologies
• That the system primarily uses Python, with a package-based project layout.
• Use of Streamlit for the UI, Polars (or similar tools) for feature engineering, and possibly Kubernetes/KServe for serving inference pipelines.
• That GitHub Actions is used for automating ML pipelines and deployments.
Happy diagramming!