# annelovepeace/game-recommendation-system architecture

> Architecture diagram of the GitHub repository annelovepeace/game-recommendation-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/annelovepeace/game-recommendation-system
- Repository: https://github.com/annelovepeace/game-recommendation-system

## Overview

##### Main Components
- Data Collection:
- Components: Two web crawling modules (as seen in the notebooks app_detail_crawl.ipynb and user_inventory_crawl.ipynb).
- External Connections: Interaction with the Steam API and Steamspy API to gather user inventories, game details, and game statistics.
- Data Storage:
- Data Files: Raw data stored in the “data” directory (files like app_detail.txt, app_stats.json, steam_user_id.txt, user_inventory.txt).
- Future/Intermediate Storage: The README also mentions writing processed data into a MySQL database, which suggests a dedicated storage component.
- Data Engineering:
- Processes: Feature extraction, transformation of raw txt/JSON files, handling missing values, and selecting relevant features (price, name, score, release date, etc.).
- Data Modeling:
- Models: Includes various recommendation approaches (popularity based, content based with TF-IDF, and collaborative filtering using item-based and user-based (ALS in Pyspark)).
- Code: The recommendation.py file appears to be central in implementing these strategies.

##### Out the Relationships and Interactions
- Data flows from external services (Steam API / Steamspy API) into the Data Collection component. The crawlers interact with these external APIs to fetch data.
- This raw data is stored in the “data” folder.
- Processed data is then written’s into a persistent store (MySQL database), which serves as the input for the Data Modeling component.
- Finally, the Data Modeling component leverages the prepared dataset to apply multiple machine learning models for generating recommendations.

##### Architectural Patterns and Key Design Principles
- Modular Pipeline: Clearly separated stages (Data Collection → Data Engineering → Data Modeling).
- Batch Processing: Web crawlers likely run in a batch mode (as suggested by the different notebooks and scripts).
- ETL Principles: Extract (via crawling), Transform (data cleanup and feature selection), and Load (writing into the database before modeling).
- Scalability and Extensibility: The modular design allows easy replacement or addition of models (such as additional collaborative filtering methods).

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Sources & Collection Layer
    subgraph "Data Sources & Collection"
        A["Steam API/Steamspy API"]:::external
        B["App Details Crawler"]:::crawler
        C["User Inventory Crawler"]:::crawler
    end

    %% Data Storage & Engineering Layer
    subgraph "Data Storage & Engineering"
        D["Raw Data Storage (data/)"]:::storage
        E["Data Processing Pipeline"]:::process
        F[(MySQL Database)]:::database
    end

    %% Data Modeling & Recommendation Layer
    subgraph "Data Modeling & Recommendation"
        G["Recommendation Engine"]:::modeling
    end

    %% Connections
    A -->|"fetches"| B
    A -->|"fetches"| C
    B -->|"saves data"| D
    C -->|"saves data"| D
    D -->|"transforms"| E
    E -->|"loads into"| F
    F -->|"provides data to"| G

    %% Click Events
    click B "https://github.com/annelovepeace/game-recommendation-system/blob/master/app_detail_crawl.ipynb"
    click C "https://github.com/annelovepeace/game-recommendation-system/blob/master/user_inventory_crawl.ipynb"
    click D "https://github.com/annelovepeace/game-recommendation-system/tree/master/data/"
    click G "https://github.com/annelovepeace/game-recommendation-system/blob/master/recommendation.py"

    %% Styles
    classDef external fill:#cce5ff,stroke:#003366,stroke-width:2px;
    classDef crawler fill:#dff0d8,stroke:#3c763d,stroke-width:2px;
    classDef storage fill:#fcf8e3,stroke:#8a6d3b,stroke-width:2px;
    classDef process fill:#d9edf7,stroke:#31708f,stroke-width:2px;
    classDef database fill:#f2dede,stroke:#a94442,stroke-width:2px;
    classDef modeling fill:#e2e2e2,stroke:#666666,stroke-width:2px;
```
