# ubisoft/ubisoft-laforge-zeroeggs architecture

> Architecture diagram of the GitHub repository ubisoft/ubisoft-laforge-zeroeggs, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ubisoft/ubisoft-laforge-zeroeggs
- Repository: https://github.com/ubisoft/ubisoft-laforge-zeroeggs

## Overview

##### Project Type & Purpose
- This is a research / experimental full-stack project dealing with gesture generation from speech. In essence, it is an end‐to‐end system that processes raw audio and motion capture (BVH) data, trains a model to learn examples of speech-to-gesture mapping, and finally runs inference to generate gesture animations.
- The project also features a data pipeline, training/inference routines, and even a rendering toolchain (bvh2fbx) for conversion to FBX files intended for integration with motion editing/rendering software.

##### File Structure
- The major directories/components are:
- The “ZEGGS” package: This contains the main Python modules for the project. Within it:
- The “anim” subfolder deals with BVH file formats and transformations (files like bvh.py, xform.py, etc.).
- The “audio” subfolder handles audio processing including signal manipulation, logs, and spectrogram generation.
- Modules for training (train.py, main.py) and data preparation (data_pipeline.py) are provided as well as utilities and optimizers.
- “configs” holds configuration files (JSON) for different versions (v1, v2) and for the data pipeline.
- “data” includes the large raw dataset (with multipart zip files), processed data (processed_v1 and processed_v2), outputs from training (saved model files), and test/evaluation files.
- “bvh2fbx” contains scripts and assets for converting BVH animation files to FBX using external tools like MotionBuilder.
- “media” and “README.md” provide visual references and documentation.
- This structure reveals a modular architecture divided by functional responsibilities (data processing, training/inference, file conversion).

##### Core Components and Their Relationships
- Data Preparation Pipeline:
- Raw data (zipped dataset) → Extraction → data_pipeline.py reading configuration (via configs/*.json) → Processed data (stored in data/processed_v1 and processed_v2)
- Model Training:
- The training routines (handled primarily by main.py, train.py) pick up processed data and configuration settings; they train the gesture generation model and output saved models into data/outputs.
- Inference / Gesture Generation:
- generate.py, together with command-line interfaces (CLI), accepts input audio files and style files, uses pre-trained models, and produces gesture animation outputs in BVH format.
- Rendering:
- The bvh2fbx module takes the generated BVH files and translates them into FBX files (with provided character models); note that both a Python script and a batch script are provided to bridge with MotionBuilder.
- Support Modules:
- The “anim” and “audio” subpackages serve as libraries that define transformation routines, mathematical helpers, and utility functions used across training, inference, and evaluation.
- External Resources/Dependencies:
- Environment setup details (Python 3.8, CUDA, PyTorch) indicate deep learning components; integration with external tools for audio (such as sox) and rendering (MotionBuilder) also plays a role.

##### Architectural Patterns & Design Principles
- Modularity: Clear separation of concerns is visible via separate directories for animation, audio, and data processing. This modular design promotes maintainability and extensibility.
- Configurability: The use of external JSON configuration files for both training and data pipeline steps suggests a flexible, parameter-driven design.
- Pipeline-based Processing: Data flows sequentially from raw extraction → processing → training → inference, and finally to rendering conversion.
- Reusability: Core helper and utility functions (helpers.py, utils.py) support multiple stages of the workflow.
- CLI and Batch Scripting: The project supports both interactive (CLI-based) and batch-processing modes, which is common for research prototypes.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data Preparation Subgraph
    subgraph "Data Preparation"
        R["Raw Data (data folder)"]:::data
        subgraph "Configurations"
            ConfigV1["Config v1 (configs/data_pipeline_conf_v1.json)"]:::data
            ConfigV2["Config v2 (configs/data_pipeline_conf_v2.json)"]:::data
        end
        DP["Data Pipeline (ZEGGS/data_pipeline.py)"]:::data
        R --> DP
        ConfigV1 --> DP
        ConfigV2 --> DP
    end

    %% Model Training Subgraph
    subgraph "Model Training"
        TM["Model Training Module (ZEGGS/main.py, ZEGGS/train.py)"]:::logic
        ArtifactV1["Model Artifacts v1 (data/outputs/v1/saved_models)"]:::logic
        ArtifactV2["Model Artifacts v2 (data/outputs/v2/saved_models)"]:::logic
        TM --> ArtifactV1
        TM --> ArtifactV2
    end

    %% Inference and Rendering
    IM["Inference Module (ZEGGS/generate.py)"]:::logic
    RC["Rendering Converter (ZEGGS/bvh2fbx)"]:::render

    %% Support Libraries and External Dependencies
    AL["Animation Processing Library (ZEGGS/anim)"]:::support
    AU["Audio Processing Library (ZEGGS/audio)"]:::support
    ED["External Dependencies (CUDA,PyTorch,sox,MotionBuilder)"]:::external

    %% Main Flow Connections
    DP --> TM
    TM --> IM
    IM --> RC

    %% Support Relationships (dashed lines)
    AL -.-> TM
    AL -.-> IM
    AU -.-> TM
    AU -.-> IM
    ED -.-> TM
    ED -.-> IM
    ED -.-> RC

    %% Click Events
    click DP "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/blob/main/ZEGGS/data_pipeline.py"
    click ConfigV1 "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/blob/main/configs/data_pipeline_conf_v1.json"
    click ConfigV2 "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/blob/main/configs/data_pipeline_conf_v2.json"
    click TM "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/blob/main/ZEGGS/main.py"
    click TM "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/blob/main/ZEGGS/train.py"
    click ArtifactV1 "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/tree/main/data/outputs/v1/saved_models"
    click ArtifactV2 "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/tree/main/data/outputs/v2/saved_models"
    click IM "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/blob/main/ZEGGS/generate.py"
    click RC "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/tree/main/ZEGGS/bvh2fbx"
    click AL "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/tree/main/ZEGGS/anim"
    click AU "https://github.com/ubisoft/ubisoft-laforge-zeroeggs/tree/main/ZEGGS/audio"

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000,stroke-width:1px;
    classDef logic fill:#90EE90,stroke:#000,stroke-width:1px;
    classDef render fill:#FFA07A,stroke:#000,stroke-width:1px;
    classDef support fill:#DDA0DD,stroke:#000,stroke-width:1px;
    classDef external fill:#F0E68C,stroke:#000,stroke-width:1px;
```
