# decodingml/personalized-recommender-course architecture

> Architecture diagram of the GitHub repository decodingml/personalized-recommender-course, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/decodingml/personalized-recommender-course
- Repository: https://github.com/decodingml/personalized-recommender-course

## Overview

##### And Label Main Components
- Data / Raw Data Source: Represent the raw input data (H&M dataset) referenced in "recsys/raw_data_sources" and the external Kaggle dataset.
- Feature Engineering Layer: Show the “recsys/features” module (articles, customers, embeddings, interactions, ranking, transactions) where the key data transformation and feature computations occur.
- Inference / Serving Module: Illustrate “recsys/inference” that contains prediction components (e.g., ranking_predictor, query_transformer) along with the Hopsworks integration endpoints in “recsys/hopsworks_integration” (which include dedicated serving scripts such as two_tower_serving.py, ranking_serving.py, and LLM ranking components). This layer is responsible for the live, online recommendations.
- User Interface (UI): Include the “recsys/ui” module and the standalone “streamlit_app.py” entry point for the web-based interface where users interact with the system.
- Configuration and Deployment: Add the configuration file (“recsys/config.py”) as the central settings hub, plus the deployment scripts and automation elements (Makefile, GitHub Actions in “.github/workflows” and “.github/actions”, and Docker/Devcontainer configurations) that help with environment setup and CI/CD.

##### Interactions and Data Flow
- From the feature layer, show the processed data flowing into the training module where the two-tower and ranking models get trained.

##### Architectural Patterns / Design Principles
- Modular Design: Clearly separate “features,” “training,” “inference,” and “UI” into different modules, showcasing a clean separation of concerns.
- MLOps Best Practices: Mention the presence of CI/CD pipelines (GitHub Actions, Makefile) and configuration management (.env.example, pyproject.toml) that support scalable deployment.

##### Relevant Technologies
- That the system primarily uses Python, with a package-based project layout.
- Use of Streamlit for the UI, Polars (or similar tools) for feature engineering, and possibly Kubernetes/KServe for serving inference pipelines.
- That GitHub Actions is used for automating ML pipelines and deployments.

Happy diagramming!

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Layer"
        DRS["Raw Data Sources"]:::data
    end

    subgraph "Feature Engineering Module"
        FE["Feature Engineering"]:::features
    end

    subgraph "Training Module"
        TR["Training Pipelines"]:::training
        TR_N1["Retrieval Training Notebook"]:::training
        TR_N2["Ranking Training Notebook"]:::training
    end

    subgraph "Inference & Serving Module"
        INFM["Inference Module"]:::inference
        HS["Hopsworks Integration / Feature Store"]:::external
    end

    subgraph "User Interface"
        UI["UI Layer"]:::ui
        SA["Streamlit App"]:::ui
    end

    subgraph "Configuration & Deployment"
        DEP["Config & Deployment"]:::deployment
        MF["Makefile"]:::deployment
        GA["GitHub Actions & CI/CD"]:::deployment
    end

    DRS -->|"processes"| FE
    FE -->|"feature_extraction"| TR
    TR -->|"trained_models"| INFM
    INFM -->|"serves_recommendations"| UI
    UI -->|"renders"| SA
    INFM -->|"integrates"| HS
    DEP -->|"configures"| FE
    DEP -->|"configures"| TR
    DEP -->|"configures"| INFM
    GA -->|"automates"| DEP

    classDef data fill:#ADD8E6,stroke:#000,stroke-width:2px;
    classDef features fill:#90EE90,stroke:#000,stroke-width:2px;
    classDef training fill:#FFFF99,stroke:#000,stroke-width:2px;
    classDef inference fill:#FFB6C1,stroke:#000,stroke-width:2px;
    classDef external fill:#FFD700,stroke:#000,stroke-width:2px;
    classDef ui fill:#DDA0DD,stroke:#000,stroke-width:2px;
    classDef deployment fill:#D3D3D3,stroke:#000,stroke-width:2px;

    click DRS "https://github.com/decodingml/personalized-recommender-course/blob/main/recsys/raw_data_sources/h_and_m.py"
    click FE "https://github.com/decodingml/personalized-recommender-course/tree/main/recsys/features"
    click TR "https://github.com/decodingml/personalized-recommender-course/tree/main/recsys/training"
    click TR_N1 "https://github.com/decodingml/personalized-recommender-course/blob/main/notebooks/2_tp_training_retrieval_model.ipynb"
    click TR_N2 "https://github.com/decodingml/personalized-recommender-course/blob/main/notebooks/3_tp_training_ranking_model.ipynb"
    click INFM "https://github.com/decodingml/personalized-recommender-course/tree/main/recsys/inference"
    click HS "https://github.com/decodingml/personalized-recommender-course/tree/main/recsys/hopsworks_integration"
    click UI "https://github.com/decodingml/personalized-recommender-course/tree/main/recsys/ui"
    click SA "https://github.com/decodingml/personalized-recommender-course/blob/main/streamlit_app.py"
    click DEP "https://github.com/decodingml/personalized-recommender-course/blob/main/recsys/config.py"
    click MF "https://github.com/decodingml/personalized-recommender-course/tree/main/Makefile"
    click GA "https://github.com/decodingml/personalized-recommender-course/blob/main/.github/workflows/ml_pipelines.yaml"
```
