# radekd91/emoca architecture

> Architecture diagram of the GitHub repository radekd91/emoca, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/radekd91/emoca
- Repository: https://github.com/radekd91/emoca

## Overview

##### Project type and purpose
- This is a research‐oriented, open‐source deep learning library and application suite for 3D monocular face reconstruction and emotion capture (“EMOCA”).
- It splits into two major subpackages:
- gdl: the core “Generic Deep Learning” library (datasets, models, layers, transforms, utils, optimizers)
- gdl_apps: concrete applications built on gdl, notably EMOCA and EmotionRecognition, each with demos, training scripts, evaluation scripts, and extensive YAML configs

##### File structure
Top level
- setup.py, install scripts, conda environment specs, requirements.txt – define dependencies and build/install flow
- external/ – third‐party submodules (DECA, SwinTransformer, EmoNet, etc.) that gdl models import or wrap
gdl/ (core library)
- datasets/ – PyTorch Lightning DataModules and Dataset classes to load and preprocess inputs (images, landmarks, video)
- transforms/ – image and keypoint augmentations and normalization
- layers/ – custom neural network layers and loss functions
- models/ – PyTorch modules for 3D face model (DECA, FLAME, Coma), CNN backbones (ResNet, Swin, VGG), MLPs, Renderer, and temporal extensions
- optimizers/ – custom optimizers (e.g., LARS)
- utils/ – file I/O, mesh operations, landmark detection wrappers, logging utilities, PyTorch Lightning callback integrations
gdl_apps/ (applications)
- EMOCA/ – training, inference, demo scripts, evaluation, and a rich hierarchy of YAML configuration files for data, learning (batching, optimizer, callbacks), model settings, in/out paths
- EmotionRecognition/ – analogous structure for emotion classification tasks on AffectNet, AFEW, etc.
- scripts/ – utility scripts to download datasets, sync logs, etc.

##### Key insights from the README
- The repo uses PyTorch and PyTorch Lightning for model definition and training.
- Configuration is entirely YAML‐driven, loaded at runtime to instantiate DataModules, Models, and Lightning Trainers.
- External submodules are statically included for 3D face reconstruction backbones (DECA, SwinTransformer, EmoNet, Spectre etc.).
- Two tiers: gdl as reusable library vs. gdl_apps as end‐to‐end experiments/demos.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration
    Config["YAML Config"]:::infra

    %% Infrastructure Layer
    subgraph "Infrastructure"
        CondaEnv["Conda Environments"]:::infra
        InstallScripts["Install & Setup Scripts"]:::infra
        Requirements["Requirements Files"]:::infra
    end

    %% External Dependencies
    subgraph "External Dependencies"
        External["Third‐Party Plugins & Frameworks"]:::ext
    end

    %% Core Library Layer
    subgraph "Core Library (gdl)"
        DataModules["DataModules & Datasets"]:::core
        Transforms["Transforms & Augmentations"]:::core
        Layers["Custom Layers"]:::core
        Losses["Loss Functions"]:::core
        Optimizers["Optimizers"]:::core
        Models["Neural Network Models"]:::core
        ExternalPlugins["External Model Plugins"]:::core
        Utils["Utilities & Logging"]:::core
        Trainer["Lightning Trainer"]:::core
        DataLoader["DataLoader"]:::core
        MeshOutput["3D Mesh / Emotion Labels"]:::core
    end

    %% Application Layer
    subgraph "Applications (gdl_apps)"
        subgraph "EMOCA App" 
            EMOCA["EMOCA App"]:::app
            EMOCA_data["EMOCA Data Scripts"]:::app
            EMOCA_demos["EMOCA Demos"]:::app
            EMOCA_conf["EMOCA Configs"]:::app
            EMOCA_training["EMOCA Training Scripts"]:::app
            EMOCA_evaluation["EMOCA Evaluation Scripts"]:::app
            EMOCA_utils["EMOCA Utilities"]:::app
        end
        subgraph "EmotionRecognition App"
            ER["EmotionRecognition App"]:::app
            ER_demos["ER Demos"]:::app
            ER_conf["ER Configs"]:::app
            ER_training["ER Training Scripts"]:::app
            ER_evaluation["ER Evaluation Scripts"]:::app
            ER_utils["ER Utilities"]:::app
        end
        Scripts["Shared Scripts"]:::app
    end

    %% Connections: Config to core modules
    Config -->|"data config"| DataModules
    Config -->|"model config"| Models
    Config -->|"learning config"| Trainer

    %% Data Flow
    DataModules --> DataLoader
    DataLoader --> Transforms
    Transforms --> Models
    Models --> ExternalPlugins
    Models --> MeshOutput

    %% Training Flow
    Models --> Losses
    Losses --> Optimizers
    Optimizers --> Trainer
    Trainer --> MeshOutput

    %% Utilities & Rendering
    Utils --> Transforms
    Utils --> MeshOutput

    %% Application orchestration
    EMOCA --> Config
    EMOCA --> Trainer
    EMOCA_data --> DataModules
    EMOCA_training --> Trainer
    EMOCA_evaluation --> MeshOutput
    EMOCA_demos --> MeshOutput
    EMOCA_conf --> Config
    EMOCA_utils --> Utils

    ER --> Config
    ER --> Trainer
    ER_demos --> MeshOutput
    ER_training --> Trainer
    ER_evaluation --> MeshOutput
    ER_conf --> Config
    ER_utils --> Utils

    Scripts --> DataModules
    Scripts --> Utils

    %% Dependency arrows
    External --> Models
    External --> ExternalPlugins

    CondaEnv --> Config
    InstallScripts --> External
    Requirements --> External

    %% Click Events
    click DataModules "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl/datasets/"
    click Transforms "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl/transforms/"
    click Layers "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl/layers/"
    click Losses "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl/layers/losses/"
    click Optimizers "https://github.com/radekd91/emoca/blob/release/EMOCA_v2/gdl/optimizers/LARS.py"
    click Models "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl/models/"
    click ExternalPlugins "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl/models/external/"
    click Utils "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl/utils/"
    click EMOCA "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EMOCA/"
    click EMOCA_data "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EMOCA/data/"
    click EMOCA_demos "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EMOCA/demos/"
    click EMOCA_conf "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EMOCA/emoca_conf/"
    click EMOCA_training "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EMOCA/training/"
    click EMOCA_evaluation "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EMOCA/evaluation/"
    click EMOCA_utils "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EMOCA/utils/"
    click ER "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EmotionRecognition/"
    click ER_demos "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EmotionRecognition/demos/"
    click ER_conf "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EmotionRecognition/emodeca_conf/"
    click ER_training "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EmotionRecognition/training/"
    click ER_evaluation "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EmotionRecognition/evaluation/"
    click ER_utils "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/EmotionRecognition/utils/"
    click Scripts "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/gdl_apps/scripts/"
    click External "https://github.com/radekd91/emoca/tree/release/EMOCA_v2/external/"

    %% Styles
    classDef core fill:#AEDFF7,stroke:#0288D1,stroke-width:1px;
    classDef app fill:#D0F2D0,stroke:#2E7D32,stroke-width:1px;
    classDef ext fill:#FFE0B2,stroke:#FB8C00,stroke-width:1px;
    classDef infra fill:#E0E0E0,stroke:#757575,stroke-width:1px;
```
