How yzy-stack/df40 works
Main components
a. Data Ingestion & Annotation
• Raw video sources: FaceForensics++ (FF++), Celeb-DF (CDF), other unknown methods
• Annotation archives: A-Celeb-DF.zip, A-FF++.zip
b. Preprocessing Layer
• Scripts: preprocess.py (frame extraction, face crop, landmarks, masks)
• Tools: dlib_tools, shape_predictor_81_face_landmarks.dat
• Configuration: preprocessing/config.yaml
c. Data Rearrangement
• Script: rearrange.py
• Output: JSON files under preprocessing/dataset_json
d. Benchmark & Training Framework (DeepfakeBench_DF40)
• Dataset loaders: training/dataset (abstract_dataset.py, pair_dataset.py, utilities)
• Detectors: training/detectors (xception, clip, i3d, etc.)
• Model definitions: training/networks (ResNet, Xception, HRNet, TimeTransformer, etc.)
• Loss & metrics: training/loss, training/metrics
• Optimizers & schedulers: training/optimizor
• Trainer orchestrator: training/trainer (base_trainer.py, trainer.py), train.py
• Configuration: training/config/{backbone, detector, train_config.yaml, test_config.yaml}
• Shell helpers: run_df40.sh, run_df40.sh variants
e. Inference & Evaluation
• Script: training/test.py
• Inputs: chosen detector YAML + checkpoint (.pth) + target dataset
• Outputs: performance metrics, confidence distributions
f. EFS Fine-tuning Pipeline (EFS_finetune_code)
• Submodule: diffusion_based (LoRA scripts, train_ddpm.py, test_ddim.py, bash scripts)
• Pretrained weights: pytorch_lora_weights.safetensors
Relationships & data flow
• Preprocessing outputs (frames, landmarks) → Rearrangement → JSON dataset
• JSON dataset → Dataset loaders → Training
• Training configuration (YAML) → Detector & Model components → Trainer
• Trainer → Checkpoints + Logs
• Checkpoints + JSON dataset → Inference → Metrics
Architectural patterns
• Modular ML pipeline: clear separation between data, preprocessing, training, inference
• Configuration-driven: YAML files govern backbone & detector selection, hyperparameters
• Plug-and-play detectors: base_detector + specific detector subclasses
• Extensible dataset utilities: abstract_dataset with pluggable transforms, augmentation
Be detailed but focused
• Every top-level directory (Annotations_for_Facial_Attributes, DeepfakeBench_DF40, EFS_finetune_code)
• Inside DeepfakeBench_DF40, break out “preprocessing”, “training”, “dataset”, “detectors”, “networks”, “loss”, “metrics”
• For EFS_finetune_code, show the LoRA training & test scripts as a separate mini-pipeline