How taeyeonmik/vit_deepfake works
Project Type and Purpose
• This is a machine-learning pipeline for image classification (deepfake detection) combining two neural architectures: EfficientNet and Vision Transformer (ViT).
• It supports data loading, preprocessing, model training, testing, and inference, with Jupyter notebooks for exploratory analysis and result visualization.
File Structure
Top-level components:
• colab-notebooks – exploratory/training/inference notebooks
• efficient-vit – core ViT training & orchestration code
• efficientnet – standalone EfficientNet model code
• inference – (inference scripts for production/CLI usage)
• train-results – output graphs & metrics
• presentation.pdf – high-level project overview
• requirements.txt – Python dependencies
Files
• efficient-vit/configs/configuration.yaml – hyperparameters & paths
• efficient-vit/train.py – training loop combining EfficientNet + ViT
• efficient-vit/test.py – model evaluation/inference script
• efficient-vit/efficient_vit.py – model definition integrating both architectures
• efficient-vit/dataset.py – data-loading & preprocessing pipeline
• efficientnet/model.py, efficientnet/utils.py – EfficientNet building blocks
Main Components
a. Data Layer
• DFFD Dataset (raw images)
• Data ingestion & preprocessing (efficient-vit/dataset.py)
b. Model Layer
• EfficientNet submodule (efficientnet/model.py)
• Vision Transformer submodule (efficient-vit/efficient_vit.py)
• Combined architecture (feature fusion or classifier head)
c. Training Orchestrator
• train.py reading configuration.yaml
• Optimizer, scheduler, back-prop, checkpointing
d. Testing/Inference
• test.py command-line script
• Colab notebooks for exploratory inference (colab-notebooks/*.ipynb)
• Inference folder for deployment scripts
e. Results & Visualization
• train-results folder (plots, metrics)
• presentation.pdf
Relationships and Data Flows
• Dataset loader → Model input (batch of images)
• Model input → EfficientNet module → feature vectors
• Model input → ViT module → feature vectors
• Feature vectors → Fusion/Classifier → output logits
• Output → Loss & metric computation → back to Training orchestrator
• After training: checkpoint files → Google Drive (external storage)
• test.py / notebooks → Trained model → inference predictions → train-results
Architectural Patterns and Key Principles
• Modular separation: data ingestion, model definition, training orchestration, evaluation
• Configuration-driven design via YAML (separates code from hyperparameters)
• Reusable components: EfficientNet implemented once in efficientnet/, reused by ViT integration
• Notebook layer for research vs. CLI scripts for reproducible pipelines