How duckduck-05/movie_recommendation works
Project Type and Purpose
• This is a full-stack, ML-driven movie recommendation application.
• Purpose: ingest movie rating data, train/serve recommendation models (collaborative and content‐based), and expose results via a Streamlit UI.
File Structure
• data/ml-100k/ : raw user‐movie ratings dataset
• model/ : serialized model parameters (w&b_cb.npz)
• notebooks/ : experimental Jupyter notebooks for data analysis, model training, evaluation
• src/lib/ : core business‐logic modules (collaborative_filtering.py, matrix_factorization.py, get_items_rated.py, evaluate.py)
• src/web.py : Streamlit application entry point
• requirements.txt : Python dependencies
Main Components
A. Data Layer
• Raw data files in data/ml-100k/
• (Optionally) External movie metadata API (mentioned in README but not implemented here)
B. Model Layer
• Notebook pipelines for training (Jupyter)
• Trained model artifact (model/w&b_cb.npz)
C. Business Logic Layer
• src/lib modules that implement algorithms:
– Collaborative filtering
– Matrix factorization
– Content‐based methods
– Evaluation & utility functions
D. Presentation Layer
• Streamlit UI (src/web.py)
• User browser → Streamlit server
Relationships and Data Flows
1. Data Ingestion: src/lib modules load raw data from data/ml-100k/
2. Model Training (offline/notebooks):
– Notebooks read data → train models → serialize parameters to model/w&b_cb.npz
3. Serving (online):
– On UI startup, web.py loads model/w&b_cb.npz via lib modules
– User inputs (e.g., user ID or ratings) in browser → HTTP calls to Streamlit UI
– Streamlit calls lib.get_items_rated, lib.collaborative_filtering, etc., to compute recommendations
– Recommendations returned to web.py → displayed in browser
Architectural Patterns & Principles
• Layered architecture: clear separation of data, model, logic, and UI
• Single‐page application (Streamlit) for rapid prototyping
• Modular design: each algorithm in its own module (supports extensibility and testing)
• Offline training vs. online inference stages
Final Checklist
• Components: Data files, Notebooks (training), Model artifact, lib modules, Streamlit server, Browser UI
• Labels: component names, file paths (optional), operation names
• Level of detail: capture all src/lib modules and their interaction with data/model