How dessa-oss/deepfake-detection works
Project Type and Main Purpose
• The project is a research / experimental deep learning system focused on deepfake detection.
• It leverages PyTorch (with a pretrained ResNet18) which is finetuned for detecting image/video manipulations.
• The system utilizes a combination of data ingestion (from FaceForensics++ and augmented YouTube datasets), preprocessing, training, and hyperparameter search (with the hparams_search.py script).
• Docker is used to containerize and run experiments consistently on machines with GPU support (nvidia-docker integration) and the Atlas scheduler.
Key Components
a. Data Ingestion and Preprocessing:
• Data Download: Scripts like faceforensics_download.py and download_T.sh for pulling data.
• Data Restructuring: The restructure_data.sh script prepares datasets into the required structure, as detailed in the README.
• Data Loader: The data_loader.py script (and potentially portions of utils.py) functions to load and preprocess data for training.
b. Training and Experimentation:
• Model Definition: The model is defined in model.py and possibly used in train.py.
• Training and Experiment Scripts:
• main.py and train.py are used to run training/inference experiments.
• hparams_search.py is responsible for running large-scale experiments and hyperparameter search.
c. Docker Containerization:
• Custom Docker Image: The custom_docker_image directory contents (Dockerfile, requirements.txt) show that a dedicated docker image (built with nvidia-docker) is used. Include this container as an “Execution Environment” block.
• job.config.yaml: This file configures the job parameters, including the docker image tag and paths to the datasets.
• That updating job.config.yaml is needed if Docker image names or paths change.
d. External and Supporting Tools/Dependencies:
• Atlas Experiment Manager: Integration for scheduling and managing experiments.
• External Tools: Dependencies like NVIDIA-Docker and ffmpeg.
• Pre-trained Models: The pretrained ResNet18 (mention that it comes from ImageNet) that serves as the starting point for finetuning.
• Reference to external datasets: Links to FaceForensics++ and YouTube/S3 datasets.
Relationships and Interactions
• That once training is scheduled (using hparams_search.py following configuration in job.config.yaml), experiments run inside the Docker container built from the custom_docker_image.
• Experiment manager (Atlas) to the training execution environment, highlighting integration.
Architectural Patterns and Design Principles
• Modular Design: Demonstrate separation between data handling, model training, and experiment management.
• Experiment Configuration: Show the importance of configuration files (job.config.yaml) to link deployment (the docker image) and data paths.