How giyn/doubanmovierecommendationsystem works
Project Type and Purpose
• This is a standalone desktop recommendation tool (Python 3.7) with three sub-systems:
• Spider module for crawling & ETL
• Recommendation Engine module (Item-based CF)
• PyQt5 GUI module for user interaction
• Data is persisted in CSV files and/or a MySQL database.
File Structure
• Top-level directories:
• Spider/: crawlers for movies & users
• Data/: raw CSVs + save_to_MySQL script
• Algorithm/: itemCF recommendation + user-record scripts
• GUI/: PyQt5 entry points and window classes
• Other artifacts: MoviePosters/ (assets), requirements.txt, README.md, LICENSE.
README for Extra Insights
• Confirms three modules (crawl, recommend, GUI)
• Uses requests, json, MySQL, pandas, PyQt5
• Data flow: crawl → store → recommend → GUI.
Main Components & Their Relationships
a. Components:
1) External Web (Douban site)
2) Spider module (Spider/*.py)
3) Data persistence layer (Data/*.csv + MySQL via save_to_MySQL.py)
4) Algorithm module (Algorithm/recommendation.py, record_users.py)
5) GUI module (GUI/*.py)
6) Asset store (MoviePosters/)
b. Interactions / Data Flow:
• Spider fetches JSON/HTML from Douban → parses → writes raw CSV (Data/) and/or pushes into MySQL.
• Algorithm reads user–movie interaction data from CSV or MySQL → computes similarity matrix → produces top-N recommendations per user → may cache or return results in memory.
• GUI at runtime:
• On login → reads users_info.csv or queries MySQL for credential check.
• On “Recommend” → calls recommendation module → displays results.
• On “Search” or “Detail” → queries Data CSV/DB → fetches metadata (director, summary) and poster path (MoviePosters/) → displays.
• On new user registration → appends to CSV/DB and triggers cold-start questionnaire → feeds initial preferences into recommendation input.
c. Architectural Patterns & Principles:
• Layered architecture (Data Access Layer, Business Logic Layer, Presentation Layer)
• Single Responsibility per module
• MVC-style separation inside GUI (windows as Views, recommendation.py as Controller, CSV/DB as Model)
d. Key Technologies:
• Python 3.7.7
• requests, json, BeautifulSoup or lxml (Spider)
• pandas, MySQL connector (Data & Algorithm)
• PyQt5 (GUI)