How gvern/virtual-makeup-tryon works
Project Type and Purpose
• This is a full-stack, real-time computer vision application that processes webcam video to apply makeup transfer in real time.
• The README outlines that it uses face detection (via MediaPipe), face parsing (via BiSeNet – although not directly visible in the file tree, the README provides context), and makeup transfer algorithms along with a GUI interface (interface.py). Dependencies include OpenCV, PyTorch, and other supporting libraries.
File Structure
• Top-Level Files:
– interface.py: Main GUI application that the user interacts with.
– main.py: Contains the MakeupTryOn class which orchestrates loading images, processing the webcam feed, and orchestrating the makeup application.
– webcam_test.py: Script to test webcam functionality.
– requirements.txt: Lists all necessary dependencies.
• Assets:
– assets/reference_images: Contains reference makeup images.
• src Directory:
– Contains core modules such as face_detection.py (for detecting faces) and makeup_transfer.py (for applying makeup styles). (Note: README also mentions face_parsing but this file might be part of a different branch or integrated into the others.)
• Utils Directory:
– Contains utility functions grouped in utils.py and visualization.py to support image handling and visual overlays.
Key Components and Their Roles
a. User Interface (GUI):
b. Application Core:
– main.py (MakeupTryOn class): Acts as the central controller that ties together modules, manages the webcam feed, and triggers makeup processing.
c. Processing Modules:
– src/face_detection.py: Implements real-time face detection and landmark tracking, likely leveraging MediaPipe.
– (Possibly) a face_parsing component: According to the README, there is a segmentation step using BiSeNet for facial region segmentation.
d. Utility Modules:
– utils/utils.py: Provides helper functions for image manipulation.
– utils/visualization.py: Adds visualization overlays, for example, for segmentation if enabled.
e. Supporting Assets & Tests:
– assets/reference_images: Sources for makeup styles.
– webcam_test.py: Offers a quick test harness for webcam functionality.
Relationships and Data Flow
• The system begins at the User Interface (interface.py) where the user inputs their desired reference image and configuration.
• The GUI delegates control to the main process (the MakeupTryOn class in main.py) which captures and processes the webcam feed.
• Within the processing pipeline, raw frames pass through the face detection module (src/face_detection.py) and then into the makeup transfer algorithm (src/makeup_transfer.py).
• If segmentation is enabled, an additional step handles face parsing (as explained in the README).
• Utility functions (from the utils folder) support these operations by processing images and generating overlays for visualization.
• Finally, the output—augmented video feed—is rendered back to the GUI and optionally snapshot capture saves images.