How mr-khans/face_tracker works
Project’s Main Components
• Entry Point & Orchestration (main.py): This is the central script that parses input arguments (video file, reference image, output directory, plus flags like --frame_skip, --gpu, and --padding) and coordinates the entire face tracking process.
• Video & Image Input Module: Illustrated by the use of OpenCV for reading video frames and the reference image, which are the raw inputs.
• Face Detection and Recognition Engine: Uses dlib and face_recognition for detecting faces in each frame. This component is where the face tracking happens.
• Model & Inference Component: Although the “models” folder only contains a models.txt file, it indicates a component that might be involved in managing/preparing face detection/recognition models. Additionally, ONNXRuntime is integrated here to accelerate inference on GPUs.
• Cropping & Output Module: After detecting and possibly matching the faces, the application crops the face regions from the video frames and saves these outputs into the specified output directory.
• Testing Example: The "test" directory with a sample video (video.mp4) provides an example scenario for functional validation, which is helpful in a high-level design for understanding input formats.
Relationships and Data Flow
• The process begins with the main orchestrator (main.py), which receives the necessary command line parameters.
• It then directs the Input Module to load the video file (and reference image).
• The video frames are sequentially passed into the Face Detection/Recognition Engine. Here, libraries like OpenCV are used for frame extraction while dlib and face_recognition work on face detection.
• When ONNXRuntime is utilized, it is incorporated within the inference stage to speed up model computation using GPU acceleration.
• Once faces are identified, the Cropping Component processes each frame to extract and save the face images to the output directory.
• The overall flow also shows configurable parameters—frame skipping for performance, adjustments for padding around faces, and GPU management—that affect how data flows through these modules.
Architectural Patterns
• Modularity: Despite the code being in a single entry script, conceptually it’s broken down into clear processing stages (input, processing, output) with distinct responsibilities.
• Pipeline Structure: Video frames follow a pipeline beginning with extraction, then face detection/recognition, and finally cropping/output.
• Integration of External Libraries: The design decouples core logic from the specific computer vision libraries, and it integrates hardware acceleration as an independent concern.
Relevant Technologies and Frameworks
• Python 3.10.12 is the runtime environment.
• OpenCV (opencv-python-headless) for video reading and processing.
• dlib and face_recognition for actual face detection/recognition.
• ONNXRuntime-gpu for model inference acceleration.
• numpy and scipy for underlying numerical computations needed in image processing.