How radekd91/emoca works
Project type and purpose
• This is a research‐oriented, open‐source deep learning library and application suite for 3D monocular face reconstruction and emotion capture (“EMOCA”).
• It splits into two major subpackages:
• gdl: the core “Generic Deep Learning” library (datasets, models, layers, transforms, utils, optimizers)
• gdl_apps: concrete applications built on gdl, notably EMOCA and EmotionRecognition, each with demos, training scripts, evaluation scripts, and extensive YAML configs
File structure
Top level
• setup.py, install scripts, conda environment specs, requirements.txt – define dependencies and build/install flow
• external/ – third‐party submodules (DECA, SwinTransformer, EmoNet, etc.) that gdl models import or wrap
gdl/ (core library)
• datasets/ – PyTorch Lightning DataModules and Dataset classes to load and preprocess inputs (images, landmarks, video)
• transforms/ – image and keypoint augmentations and normalization
• layers/ – custom neural network layers and loss functions
• models/ – PyTorch modules for 3D face model (DECA, FLAME, Coma), CNN backbones (ResNet, Swin, VGG), MLPs, Renderer, and temporal extensions
• optimizers/ – custom optimizers (e.g., LARS)
• utils/ – file I/O, mesh operations, landmark detection wrappers, logging utilities, PyTorch Lightning callback integrations
gdl_apps/ (applications)
• EMOCA/ – training, inference, demo scripts, evaluation, and a rich hierarchy of YAML configuration files for data, learning (batching, optimizer, callbacks), model settings, in/out paths
• EmotionRecognition/ – analogous structure for emotion classification tasks on AffectNet, AFEW, etc.
• scripts/ – utility scripts to download datasets, sync logs, etc.
Key insights from the README
• The repo uses PyTorch and PyTorch Lightning for model definition and training.
• Configuration is entirely YAML‐driven, loaded at runtime to instantiate DataModules, Models, and Lightning Trainers.
• External submodules are statically included for 3D face reconstruction backbones (DECA, SwinTransformer, EmoNet, Spectre etc.).
• Two tiers: gdl as reusable library vs. gdl_apps as end‐to‐end experiments/demos.