How ondyari/faceforensics works
Project type and purpose
• This is not a web or microservice application but a research‑focused machine learning pipeline and dataset management toolkit.
• Purpose: Provide and generate manipulated facial video/image datasets (DeepFakes, Face2Face, FaceSwap, NeuralTextures, FaceShifter, etc.) and demonstrate a sample classification pipeline to detect forgeries.
File structure
• Top‑level:
• LICENSE, README.md – public information
• classification/ – sample classification demo
• dataset/ – dataset management, download, generation, extraction scripts, and sub‑repos for each manipulation method
• images/ – static assets (README illustrations)
• classification/ subfolders:
• classification/dataset – data loader and transform utilities
• classification/network – model definitions (Xception, custom models)
• detect_from_video.py – end‑to‑end inference script
• requirements.txt – Python dependencies for classification
• dataset/ subfolders:
• DeepFakeDetection, DeepFakes (faceswap-master), FaceShifter, FaceSwapKowalski, NeuralTextures – each with their own README, code, plugins and tools
• general scripts: compress.py, extract_compressed_videos.py, generate_original_images_from_downloaded_videos.py
• conversion_dict.json, splits/*.json – configuration and data splits
• setup_ffmpeg_h264.sh, setup.sh – environment setup scripts
README insights
• Core functions: dataset download, dataset generation (face‑swap pipelines), benchmark evaluation, classification sample.
• Dependencies: Dockerfiles exist for CPU/GPU faceswap builds; Python ML frameworks implied (likely TensorFlow or PyTorch) in classification.
Main components
a. Data Ingestion & Generation
• Original raw videos (YouTube source)
• Download script (dataset/README.md)
• Face manipulation modules (DeepFakes, Face2Face, FaceSwap, NeuralTextures, FaceShifter)
• Dataset scripts: video compression, extraction, splitting
b. Data Storage
• On‑disk directory structure under dataset/
• Versioned subfolders per manipulation method
• JSON split files for train/val/test
c. Data Processing & Transformation
• classification/dataset/transform.py – image/video frame transforms (resize, normalization)
• Dataset loader (classification/dataset/__init__.py)
d. Model Training & Inference
• classification/network/models.py – generic model wrappers
• classification/network/xception.py – Xception architecture for classification
• detect_from_video.py – inference pipeline combining loader, model, output
e. External Tools & Environments
• Docker (faceswap-master Dockerfiles)
• FFmpeg (setup_ffmpeg_h264.sh)
• GPU/CPU execution contexts
Relationships and interactions
• Data ingestion → dataset generation modules → processed/manipulated video files
• Processed files + splits → classification dataset loader → preprocessed tensors
• Preprocessed tensors → model training (Xception, other nets) → trained weights
• Trained weights + inference script → detection results on new videos
• Environment scripts and Docker images feed into dataset generation and execution contexts
Architectural patterns and design principles
• Pipeline pattern: clearly separated stages (ingest → generate → preprocess → train → infer)
• Modular structure: each face‑swap method encapsulated in its own subfolder, plugin architecture in faceswap-master/plugins
• Reusability: shared transform utilities, generic model wrappers
Relevant technologies and libraries
• Python 3.x
• Deep learning framework (TensorFlow or PyTorch inferred from network scripts)
• Docker for containerized builds of face‑swap tools
• FFmpeg for video decoding/encoding
• JSON for configuration (splits, conversion dict)