How sweeritter/facerecognition works
Project Type and Purpose
• The project is a real‐time, deep learning–based Face Recognition System.
• Its primary function is to capture video streams via FFmpeg, detect faces using a YOLOv8 model, extract features using a FaceNet model (with HNSW for fast retrieval), and record the results in a MySQL database.
• The system integrates a Graphical User Interface (GUI) built with Qt for user interactions (e.g., login, face registration, and querying historical records).
File Structure
• Top-Level Modules:
– CMakeLists.txt: Main build configuration
– Facenet: Contains FaceNet model code and ONNX files for face recognition.
– Ffmpeg: Contains FFmpeg-based modules for video decoding and stream handling.
– Yolov8: Contains code for running face detection using the YOLOv8 model.
– Sql: Contains code and configuration scripts for database operations (using MySQL) and HNSW-based feature search.
– UI: Contains all elements related to the Qt user interface – UI designs, resource files (images, ONNX model by name), and configuration headers.
– installer: Contains scripts and configuration files to generate the installation package.
– Additional files (main.cpp, script files, imageprocessor.h, etc.) are used to glue these modules together.
README for Additional Insights
• The README clearly explains that the system consists of core components: video processing, face detection, face recognition (with a two-stage verification: HNSW for candidate search followed by cosine similarity for precision), database storage, and the Qt-based interactive UI.
• It also details dependency requirements (OpenCV, FFmpeg, ONNX Runtime, Qt, MySQL, Inno Setup) and build/packaging instructions.
• The architecture is multi-tiered with clear separation between video input, processing (deep learning models), data management, and user interaction.
a. Main Components:
– Video Processing Module (FFmpeg): Handles the network video stream capture and decoding.
– Face Detection Module (YOLOv8): Pre-processes frames to determine if faces are present.
– Face Recognition Module (FaceNet): Extracts features and calculates similarity, involving a two-step process (HNSW for fast matching and cosine similarity verification).
– Data Management Module (SQL): Manages interactions with the MySQL database (storage of face features and attendance records).
– User Interface (UI): Built with Qt to interact with the user for registration, login, and monitoring.
– Additional supporting components include the installer module for deployment.
b. Relationships and Interactions:
– The Video Processing Module feeds video frames to the Face Detection Module.
– Once a face is detected, the Face Recognition Module is invoked to extract features from the face region.
– The computed features are then compared against stored data using the HNSW-based fast search, with cosine similarity used for final verification.
– Recognition results are sent to the Data Management Module, where the results (and possibly attendance information) are stored in the MySQL database.
– The UI module displays live video streams, registration forms, and historical query results, and also handles configuration and user settings.
– There should also be a feedback loop allowing the installer module to package and deploy the complete system.
c. Architectural Patterns & Design Principles:
– Modularity: Each core functionality (video processing, detection, recognition, data management, and UI) is encapsulated in its own module/directory.
– Layered Architecture: There is a clear separation of concerns between data input (video), processing (detection/recognition), data storage, and presentation (UI).
– Reusability & Extensibility: Use of configuration files (CMake, INNO Setup, installer scripts) and modular source code encourages future enhancements (e.g., cross-platform support).
– Integration of third-party libraries and frameworks such as OpenCV, FFmpeg, ONNX Runtime, Qt, and MySQL libraries is essential to consider.
d. Technologies, Frameworks, or Libraries:
– FFmpeg (under “Ffmpeg”): streaming and video decoding.
– YOLOv8 (under “Yolov8”): face detection.
– FaceNet (under “Facenet”): feature extraction.
– HNSW Library (under “Sql/hnswlib”): for fast matching in high-dimensional space.
– Qt (under “UI”): for building the graphical user interface.
– MySQL (database interactions described in “Sql”): to store face features.
– CMake is used for configuration and building the project.