How dphieu/emotion_recognition works
This project is an emotion recognition system that processes facial images and live video streams to predict emotions using two variants of the ResNet model (one pretrained and one trained from scratch). The system is built with a Flask web application allowing users to upload images or access a video stream for real-time predictions.
Project Type and Purpose
• The project is a lightweight full-stack web application focused on emotion recognition.
• It integrates computer vision (face detection with OpenCV Haar Cascades), deep learning (ResNet models), and web user interfaces for image upload and video streaming.
• It includes components for both model training (with a separate train.py script and associated dataset under the data directory) and live prediction (via the Flask app).
Analyzing the File Structure
• Top-level files and directories:
• app.py: The Flask application that serves different prediction interfaces.
• config.py: Central configuration which likely manages settings (paths, model parameters, etc.).
• train.py: Script dedicated to model training.
• data/: Contains datasets, especially FER2013 for training and testing.
• model/: Holds implementations of two ResNet architectures (pretrained and from scratch).
• templates/: Contains HTML templates for the different web pages (home, image upload, video stream).
• utils/: Contains utility modules like face_detector.py and preprocessing.py that handle face detection and image preprocessing.
• The file structure indicates clear separation of concerns:
• Web application layer (Flask and templates)
• Machine learning layer (model training and inference)
• Utility layer for data preprocessing and face detection
Key Components
a. User Interface (Frontend):
• Components: Templates (index.html, predict_images.html, video_stream.html).
• Interaction: Web browser where users can upload images or launch live video for emotion recognition.
b. Backend / Application Layer:
• Main component: app.py, which is the Flask application.
• Roles: Routing user requests, interfacing with the model layer for prediction.
• It should show endpoints (image prediction route, video streaming route) and how they feed into the model processing pipeline.
c. Model Layer:
• Consists of two sub-components:
• model/resnet_pretrained.py: Implements the pretrained ResNet model.
• model/resnet_fromscratch.py: Implements the ResNet model built from scratch.
• These components are used for emotion classification.
d. Training Component:
• Represented by train.py, which trains both model types using the FER2013 dataset (located under data/fer2013/).
• Relationship with the data repository and model directory (to save updated model weights).
e. Utility Components:
• utils/face_detector.py: Handles face detection using OpenCV’s Haar Cascade (supported by haarcascade_frontalface_default.xml in the root).
• utils/preprocess.py: (or preprocessing.py as mentioned in the README) handles image preprocessing (resize, grayscale, normalization).
• These utility modules should be depicted as being invoked by both the training and prediction pipelines.
f. Configuration Management:
• config.py acts as a central configuration hub for the Flask application and may also provide necessary parameters for training and model operation.
g. External Data and Model Artifacts:
• Data: The FER2013 CSV files under data/.
• Pretrained Weights: Files like resnet_pretrained.pt and simple_fromscratch.pt.
• Optionally, show the configuration flow from config.py into both the web app and training environment.
Important Architectural Patterns and Design Principles
• Separation of Concerns: Each directory (web, model, utils, data) is responsible for a different aspect of the system.
• Modularity: Clear division between training (train.py) and inference (app.py), as well as between different model types.
• Reusability: Utility modules (face_detector and preprocessing) are used across training and prediction services.
• Scalability: Even though the project is modest, it is designed such that new models or additional preprocessing methods could be integrated with minimal changes.