# bhavik-ksvss/cloud-movie-recommender-system architecture

> Architecture diagram of the GitHub repository bhavik-ksvss/cloud-movie-recommender-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhavik-ksvss/cloud-movie-recommender-system
- Repository: https://github.com/bhavik-ksvss/cloud-movie-recommender-system

## Overview

##### Main Components
- Data Ingestion:
– Represent the external dataset source (MovieLens) and the initial data transfer using Cloud Shell commands.
– Show the transition from the external source to a temporary staging area before moving the data.
- Storage:
– Optionally, show the archive directory with raw CSV files as reference to the dataset format.
- Data Processing & Analysis:
– Include the Dataproc cluster as a central component.
– Distinguish the Jupyter Notebook (“Recommendation System.ipynb”) that performs initial data transformation, summary statistics, and visualizations.
- Recommendation Engine:
– Illustrate the PySpark job driven by the “recommender.py” script that runs on the Dataproc cluster, executing the collaborative filtering algorithm.
- Data Warehousing:
– Represent Google BigQuery as the target where the processed data, including predictions, is migrated and stored for querying.
- Additional GCP Components:
– Optionally, add elements for Virtual Private Cloud (VPC) networks and Cloud Shell to capture the complete GCP ecosystem used.

##### Relationships and Data Flow
- From GCS, show the data flowing into the “Dataproc Cluster” where two parallel activities occur:
– A Jupyter Notebook handling exploratory analysis and visualizations.
– A PySpark job (initiated by recommender.py) that performs collaborative filtering and produces recommendations.
- Both processes feed results into “BigQuery”, indicating that the processed dataset and predictions are migrated into tables.

##### Architectural Patterns and Technologies
- Separation of concerns: ingestion, transformation/analysis, recommendation engine, and data warehousing.
- Usage of cloud-native services, where different components (GCS, Dataproc, BigQuery) are orchestrated on GCP.
- Mention that the alternating least squares algorithm (a key point of the recommendation engine) is implemented within the Spark job.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["MovieLens Data Source"]:::ingestion
    B["Cloud Shell / Data Ingestion"]:::ingestion
    C["GCS Bucket (Storage)"]:::storage
    F["Raw Data Archive (CSV)"]:::storage
    D["Dataproc Cluster (Processing)"]:::processing
    D1["Jupyter Notebook (Analysis)"]:::processing
    D2["Recommendation Engine (PySpark job via recommender.py)"]:::processing
    X["BigQuery (Data Warehouse)"]:::warehouse

    A -->|"ingests"| B
    B -->|"transfers"| C
    C -->|"stores_raw_data"| F
    C -->|"feeds_data"| D
    D -->|"runs_exploratory_analysis"| D1
    D -->|"executes_ALS"| D2
    D1 -->|"migrates_results"| X
    D2 -->|"migrates_results"| X

    click F "https://github.com/bhavik-ksvss/cloud-movie-recommender-system/tree/main/archive/"
    click D1 "https://github.com/bhavik-ksvss/cloud-movie-recommender-system/blob/main/Recommendation System.ipynb"
    click D2 "https://github.com/bhavik-ksvss/cloud-movie-recommender-system/blob/main/recommender.py"

    classDef ingestion fill:#8FBC8F,stroke:#006400,stroke-width:2px;
    classDef storage fill:#ADD8E6,stroke:#00008B,stroke-width:2px;
    classDef processing fill:#FFB347,stroke:#FF8C00,stroke-width:2px;
    classDef warehouse fill:#DDA0DD,stroke:#800080,stroke-width:2px;
```
