How megvii-research/caddm works
• Data Preparation & Ingestion:
• Data Directory: Contains raw videos and extracted images.
• lib/extract_frames_ldm_ff++.py: Component to extract frames from videos.
• Preprocessing: This lives in lib/data_preprocess with scripts for augmentations and specific cropping functions.
• Landmark Detector: Represented by the shape_predictor_81_face_landmarks.dat and its association via lib (for facial landmark extraction).
• Configuration and Control:
• configs Directory: Contains configuration files (e.g., caddm_train.cfg and caddm_test.cfg) that define parameters for training and evaluation.
• Command-line scripts (train.py and test.py): Serve as entrypoints for training and evaluation.
• Model and Network Backbones:
• backbones Directory: Contains different backbone implementations (such as ResNet and EfficientNet). Note the nested efficientnet_pytorch subdirectory.
• model.py: Likely the top-level model definition that assembles network components.
• Detection Layers and Loss Functions:
• Utility and Helper Functions:
• lib/util.py and other supporting utilities in the lib directory which handle things like image processing, configuration reading, and other helper processes.
Relationships and Data Flow
• That the Model is built on top of multiple backbone architectures, and detection layers are integrated to compute losses and manage outputs.
• The training and evaluation scripts (train.py and test.py) use the configuration files, thus forming the control layer over the entire process.
Architectural Patterns
• Modularity: The system is divided into modular components (data processing, backbone networks, detection layers) that can be independently maintained and updated.
• Extensibility: With clearly defined directories for different backbones and detection modules, adding new model architectures or detection functions is straightforward.
• Dependency Inversion: The configuration files and command-line interface separate the core model logic from runtime parameters.
• Within the model block, show sub-components for the chosen backbone (either ResNet-based or EfficientNet-based) and attach modules for detection layers and loss function computations.
• “Data Preparation & Extraction”
• “Data Preprocessing (Augmentation, Landmark Detection)”
• “Model (Backbone + Detection Layers)”
• “Training & Evaluation Control (Driven by Config Files)”