How forez0/coursework works
Project type and purpose
• This is a full-stack web application built on Django that implements a personalized film recommendation system.
• It uses Celery + Redis for asynchronous tasks (model training), LightFM as the recommendation algorithm, a SQLite database for user/movie data, and Django templates (and optional DRF endpoints) for the UI.
File structure
Top-level
• main.py, run_worker.py – entry points for web and worker processes
• film_recommender/ – root Django project (settings, ASGI/WSGI, Celery config, manage.py)
• recommender/ – the core Django app with models, views, serializers, Celery tasks, ML code and custom management commands
• ml_models/ – pickled LightFM data and model artifacts
• templates/ – Django HTML templates for UI
• db.sqlite3 – development database
• venv/ – Python virtual environment
• requirements.txt – Python dependencies
Patterns & frameworks
• Django MVC (Models, Views, Templates)
• Django REST Framework serializers and viewsets
• Celery tasks for training and batch recommendation generation
• Custom management commands (generate_test_data)
• LightFM pipeline under recommender/ml/lightfm_model.py
README for additional insights
• Confirms use of Redis (via Docker) as Celery broker, Celery worker invocation, and Django dev server.
• Describes high-level flow: user interactions → stored in DB → Celery-triggered model retraining → persisted model → user recommendation generation.
• Lists relevant endpoints: web UI at /, admin at /admin/, and internal DRF endpoints under recommender/urls.py.
a. Main components
1. User Browser / Frontend (Django Templates + optional AJAX)
2. Django Web Server (Gunicorn/Runserver)
3. SQLite Database (or RDBMS in production)
4. Celery Worker(s)
5. Redis Broker
6. LightFM Model Store (ml_models folder or object storage)
7. External TMDb API (for movie metadata)
b. Relationships & Data Flow
1. User submits rating or requests recommendations → HTTP POST/GET to Django views
2. Django writes user/movie/rating to SQLite through models
3. On schedule or admin action, Django (via Celery beat or management command) enqueues a train_recommendation_model task to Redis
4. Celery Worker pulls task from Redis, loads historical data, calls LightFM training logic, writes updated .pkl files to ml_models
5. When a user requests fresh recommendations, Django view or DRF endpoint loads pickled model, generates ranked list, stores UserRecommendation records, and returns HTML/JSON to user
6. (Optional) Django contacts TMDb API via utils.py to fetch or cache movie details
c. Architectural patterns & design principles
• Separation of concerns: Web handling in Django, asynchronous ML in Celery
• Modular ML pipeline: data prep, training, serialization isolated under recommender/ml
• Reusable management commands for data setup
• Use of DRF serializers for API clients, enabling both web and programmatic access
d. Key technologies and libraries
• Django (ORM, URL routing, templates)
• Django REST Framework (serializers, viewsets)
• Celery (task queue) + eventlet pool
• Redis (message broker)
• LightFM (recommendation algorithm)
• SQLite (dev DB)
• Docker (for Redis in local setup)