# dphieu/emotion_recognition architecture

> Architecture diagram of the GitHub repository dphieu/emotion_recognition, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dphieu/emotion_recognition
- Repository: https://github.com/dphieu/emotion_recognition

## Overview

This project is an emotion recognition system that processes facial images and live video streams to predict emotions using two variants of the ResNet model (one pretrained and one trained from scratch). The system is built with a Flask web application allowing users to upload images or access a video stream for real-time predictions.

##### Project Type and Purpose
- The project is a lightweight full-stack web application focused on emotion recognition.
- It integrates computer vision (face detection with OpenCV Haar Cascades), deep learning (ResNet models), and web user interfaces for image upload and video streaming.
- It includes components for both model training (with a separate train.py script and associated dataset under the data directory) and live prediction (via the Flask app).

##### Analyzing the File Structure
- Top-level files and directories:
- app.py: The Flask application that serves different prediction interfaces.
- config.py: Central configuration which likely manages settings (paths, model parameters, etc.).
- train.py: Script dedicated to model training.
- data/: Contains datasets, especially FER2013 for training and testing.
- model/: Holds implementations of two ResNet architectures (pretrained and from scratch).
- templates/: Contains HTML templates for the different web pages (home, image upload, video stream).
- utils/: Contains utility modules like face_detector.py and preprocessing.py that handle face detection and image preprocessing.
- The file structure indicates clear separation of concerns:
- Web application layer (Flask and templates)
- Machine learning layer (model training and inference)
- Utility layer for data preprocessing and face detection

##### Key Components
a. User Interface (Frontend):
- Components: Templates (index.html, predict_images.html, video_stream.html).
- Interaction: Web browser where users can upload images or launch live video for emotion recognition.

b. Backend / Application Layer:
- Main component: app.py, which is the Flask application.
- Roles: Routing user requests, interfacing with the model layer for prediction.
- It should show endpoints (image prediction route, video streaming route) and how they feed into the model processing pipeline.

c. Model Layer:
- Consists of two sub-components:
- model/resnet_pretrained.py: Implements the pretrained ResNet model.
- model/resnet_fromscratch.py: Implements the ResNet model built from scratch.
- These components are used for emotion classification.

d. Training Component:
- Represented by train.py, which trains both model types using the FER2013 dataset (located under data/fer2013/).
- Relationship with the data repository and model directory (to save updated model weights).

e. Utility Components:
- utils/face_detector.py: Handles face detection using OpenCV’s Haar Cascade (supported by haarcascade_frontalface_default.xml in the root).
- utils/preprocess.py: (or preprocessing.py as mentioned in the README) handles image preprocessing (resize, grayscale, normalization).
- These utility modules should be depicted as being invoked by both the training and prediction pipelines.

f. Configuration Management:
- config.py acts as a central configuration hub for the Flask application and may also provide necessary parameters for training and model operation.

g. External Data and Model Artifacts:
- Data: The FER2013 CSV files under data/.
- Pretrained Weights: Files like resnet_pretrained.pt and simple_fromscratch.pt.

- Optionally, show the configuration flow from config.py into both the web app and training environment.

##### Important Architectural Patterns and Design Principles
- Separation of Concerns: Each directory (web, model, utils, data) is responsible for a different aspect of the system.
- Modularity: Clear division between training (train.py) and inference (app.py), as well as between different model types.
- Reusability: Utility modules (face_detector and preprocessing) are used across training and prediction services.
- Scalability: Even though the project is modest, it is designed such that new models or additional preprocessing methods could be integrated with minimal changes.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Frontend Subgraph
    subgraph "User Interface (Frontend)"
        T1["index.html"]:::frontend
        T2["predict_images.html"]:::frontend
        T3["video_stream.html"]:::frontend
    end
    Browser["Web Browser"]:::frontend

    %% Frontend to Backend Connection
    T1 --> Browser
    T2 --> Browser
    T3 --> Browser
    Browser -->|"HTTP_Request"| App["Flask Application (app.py)"]:::backend

    %% Configuration Management
    App -->|"uses_config"| Config["Configuration (config.py)"]:::config

    %% Prediction Pipeline - Utility Components
    App -->|"detects_face"| FDetect["Face Detector (utils/face_detector.py)"]:::utility
    App -->|"preprocesses"| Preproc["Preprocessing (utils/preprocess.py)"]:::utility
    FDetect -->|"uses_cascade"| Cascade["haarcascade_frontalface_default.xml"]:::utility

    %% Model Layer for Predictions
    App -->|"predicts"| Pretrained["Pretrained ResNet (model/resnet_pretrained.py)"]:::model
    App -->|"predicts"| Scratch["ResNet From Scratch (model/resnet_fromscratch.py)"]:::model
    Pretrained ---|"weight"| PretrainedW["Weights (resnet_pretrained.pt)"]:::model
    Scratch ---|"weight"| ScratchW["Weights (simple_fromscratch.pt)"]:::model

    %% Training Pipeline
    Data["Data Repository (data/)"]:::data
    Data -->|"training_data"| Train["Training Module (train.py)"]:::training
    Train -->|"updates_model"| Pretrained
    Train -->|"updates_model"| Scratch
    Train -->|"uses_utilities"| FDetect
    Train -->|"uses_utilities"| Preproc
    Config -->|"config_params"| Train

    %% Styles
    classDef frontend fill:#cce5ff,stroke:#004085,stroke-width:2px;
    classDef backend fill:#d4edda,stroke:#155724,stroke-width:2px;
    classDef config fill:#e2e3e5,stroke:#41464b,stroke-width:2px;
    classDef utility fill:#f8d7da,stroke:#721c24,stroke-width:2px;
    classDef model fill:#fff3cd,stroke:#856404,stroke-width:2px;
    classDef training fill:#f5c6cb,stroke:#721c24,stroke-width:2px;
    classDef data fill:#d1ecf1,stroke:#0c5460,stroke-width:2px;

    %% Click Events
    click T1 "https://github.com/dphieu/emotion_recognition/blob/main/templates/index.html"
    click T2 "https://github.com/dphieu/emotion_recognition/blob/main/templates/predict_images.html"
    click T3 "https://github.com/dphieu/emotion_recognition/blob/main/templates/video_stream.html"
    click App "https://github.com/dphieu/emotion_recognition/blob/main/app.py"
    click Config "https://github.com/dphieu/emotion_recognition/blob/main/config.py"
    click Pretrained "https://github.com/dphieu/emotion_recognition/blob/main/model/resnet_pretrained.py"
    click PretrainedW "https://github.com/dphieu/emotion_recognition/blob/main/resnet_pretrained.pt"
    click Scratch "https://github.com/dphieu/emotion_recognition/blob/main/model/resnet_fromscratch.py"
    click ScratchW "https://github.com/dphieu/emotion_recognition/blob/main/simple_fromscratch.pt"
    click Train "https://github.com/dphieu/emotion_recognition/blob/main/train.py"
    click FDetect "https://github.com/dphieu/emotion_recognition/blob/main/utils/face_detector.py"
    click Preproc "https://github.com/dphieu/emotion_recognition/blob/main/utils/preprocess.py"
    click Data "https://github.com/dphieu/emotion_recognition/tree/main/data/"
    click Cascade "https://github.com/dphieu/emotion_recognition/blob/main/haarcascade_frontalface_default.xml"
```
