How bakaichi/emotionalrec works
Main Components
• Frontend: This is built with React and Tailwind CSS (inside the “frontend” directory). It handles user interactions – logging in with Spotify, video uploads, and displaying the recommended playlist.
• Backend: Built with FastAPI (using “main.py” and the “api” directory with routes in “api/routes.py”). It serves as the central hub to process callbacks, handle Spotify OAuth (with components found under “recommendation”), and provide API endpoints such as /upload_video, /process_latest, /colab_callback, /recommend, /login, and /callback.
• Emotion Detection External Module: In the “emotion_detection” folder, a Google Colab notebook utilizes DeepFace (and associated OpenCV code) to process uploaded videos. This module polls Google Drive and sends the detected emotion to the backend.
• External Services: Several third-party services are integrated:
• Google Drive: For storing user-uploaded videos.
• Spotify API: Accessed via Spotipy in the backend to create playlists and fetch recommendations.
• Ngrok: Used to expose the local FastAPI server to the Colab notebook.
Relationships and Interactions
• User → Frontend: The user logs into the system via the React front end, uploads a video, and views their recommended playlist.
• Frontend → Backend: The React app communicates with the FastAPI backend (HTTP requests with proper CORS configuration) for uploading videos, triggering callbacks, and receiving recommendations.
• Google Colab → Backend: The Colab notebook polls Google Drive for newly uploaded videos, processes them (using DeepFace), and then sends the detected emotion via a POST request (to /colab_callback) to the backend.
• Backend → Recommendation Module: When receiving the callback, the backend calls its internal recommendation logic (found in “recommendation/recommender.py”, “spotify_auth.py”, and “spotify_utils.py”) to process the emotion data and invoke Spotify API interactions.
• Backend → External APIs: The backend makes calls to Spotify’s API (via Spotipy) and may also interact with Google Drive to manage video uploads.
Architectural Patterns and Design Principles
• Separation of Concerns: The system is cleanly divided between the Frontend (UI), Backend (API, business logic), and External Processing (emotion detection via Colab). Within the backend, the “api” and “recommendation” modules keep API route definitions separate from Spotify and recommendation logic.
• Modular Design: Using separate directories for distinct functions (e.g., “emotion_detection”, “api”, “recommendation”) allows each component to evolve independently.
• API-Centric Communication: The use of FastAPI and clearly defined endpoints establishes a network of RESTful interfaces between components.
Technologies and Frameworks
• Frontend: React.js, Tailwind CSS
• Backend: FastAPI, with middlewares for CORS
• Emotion Detection: DeepFace (with RetinaFace), OpenCV, executed via a Google Colab notebook
• Spotify Integration: Spotipy and OAuth2 flows implemented in Python
• External Services: Google Drive API for file storage, Ngrok for public tunnel exposure