How yudhisteer/face-recognition-with-masks works
Project Type and Purpose
• This is a Python‐based data-processing and machine-learning application (“open-source tool”) for automating an office attendance system by combining
• real-time face detection (with mask/no-mask classification)
• face embedding/recognition (Siamese/FaceNet approach)
• attendance logging
• Key indicators: scripts for training mask detector, real_time_face_recognition, and AttendanceProject.py; README describing dataset preparation, model training (Inception-ResNet v1 + triplet loss), and live inference pipeline.
File Structure Analysis
Top-level scripts:
• train_mask_detector.py – trains SSD mask-detector model
• detect_mask_video.py – runs mask detection on video streams
• real_time_face_recognition.py – runs full pipeline: mask detection + face embedding + lookup + attendance mark
• AttendanceProject.py / main.py – likely glue to orchestrate training vs inference vs DB logging
No separate “frontend” or “backend” folders: each script implements a stage in the pipeline. Configuration (e.g. .pb model files, hyperparams) is embedded or read at runtime.
Core Components
a. Data Preparation & Augmentation
– Raw image datasets (CASIA‐Webface, LFW, MS-Celeb)
– Scripts for face cropping, mask overlay, data cleaning (mislabels removal), stratified sampling, augmentation
b. Training Pipelines
– Mask‐detector training (SSD on TensorFlow, .pb export)
– Face-Net training (Inception-ResNet v1, triplet loss / cross‐entropy variants)
c. Model Artifacts & Storage
– .pb files (mask detector, face embedding model)
– embeddings database (serialized NumPy arrays or simple file per identity)
d. Real-Time Inference Service
– Camera input → frame grabber → mask classifier → face crop → embedding extractor → K-NN or thresholded lookup → attendance DB update
e. External Interfaces
– Camera/Webcam (video stream)
– Attendance database or CSV (in/out logs)
– Admin console or CLI (to register new users/images)
Component Interactions and Data Flow
1. Offline Data Prep: Images → face-crop & mask overlay → cleaned + augmented dataset
2. Training Stage A: Dataset → train_mask_detector.py → mask_detector.pb
3. Training Stage B: Dataset → train FaceNet (inception_resnet_v1) → embedding_model.pb
4. Deployment:
• Load mask_detector.pb and embedding_model.pb
• Load embeddings index (one 128-dim vector per employee)
5. Real-Time Loop:
Camera frame → resize/normalize → SSD mask detection → crop face → resize for FaceNet → extract 128-D embedding → distance compare to gallery → match/no-match → mark attendance
Architectural Patterns and Design Principles
• Modular pipeline: separate preprocessing, training, inference
• Reusable models exported as TensorFlow .pb
• Data-centric approach: focus on dataset cleaning and augmentation
• Siamese/FaceNet (triplet-loss) for one-shot learning
• Stratified sampling to prevent class imbalance
Technologies, Frameworks, Libraries
• Language: Python 3
• ML: TensorFlow (frozen .pb models), NumPy
• CV: OpenCV (video I/O, image ops)
• Face detection: SSD (TensorFlow), dlib for landmarks
• Models: Inception-ResNet v1 for embeddings; custom ResNet block prototype
• Storage: file-based (images, embeddings), possible simple SQLite/CSV for attendance
Essential Detail
• Both mask detection and face recognition as sequential steps in inference
• Embedding gallery and distance matching logic
• Data-augmentation and cleaning loops in preprocessing
• Don’t over-fragment: group logically related scripts into “Mask Detector Training” or “Preprocessing Pipeline”
• Avoid missing the attendance update endpoint (CSV/DB)