# powerhouseman/comfyui-advancedliveportrait architecture

> Architecture diagram of the GitHub repository powerhouseman/comfyui-advancedliveportrait, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/powerhouseman/comfyui-advancedliveportrait
- Repository: https://github.com/powerhouseman/comfyui-advancedliveportrait

## Overview

##### Project type and purpose
- This is a ComfyUI plugin (“AdvancedLivePortrait”) for real-time facial expression editing, motion transfer, and animation generation using deep-learning models.
- It extends ComfyUI with new node types (defined in nodes.py) that orchestrate a sequence of neural-network modules to process source images/videos and driving videos.

##### File structure
- Top level
- nodes.py – defines the custom ComfyUI nodes that wrap the core pipeline
- pyproject.toml, requirements.txt – Python packaging and dependencies (PyTorch, ComfyUI, etc.)
- install.bat – install script for Windows
- LivePortrait/
- config/ – inference parameters (inference_config.py) and models.yaml (paths, hyperparameters)
- modules/ – deep-learning building blocks (appearance_feature_extractor, motion_extractor, dense_motion, spade_generator, warping_network, stitching_retargeting_network)
- utils/ – helper routines for I/O, video handling, camera processing, face analysis, timing, resource management
- sample/
- workflows/ – example JSON graphs for ComfyUI (various use cases)
- original_sample_asset/ – driving video frames and source images
- sample_exp_data.exp – serialized facial‐expression data
- .github/workflows/publish.yml – CI for publishing the plugin to ComfyUI-Manager

##### Key insights from README
- Purpose: “Faster with real-time preview,” supports editing expressions in photos and videos, animation generation from multiple expressions, extract & reuse expression data.
- Installation via ComfyUI-Manager; sample workflows placed under custom_nodes.
- Workflows use JSON commands to link nodes, specifying motion indices, frame linking, save/load expression nodes.
- Relies on models converted by Kijai and the original LivePortrait models.

a. Main components
1. ComfyUI Core – node graph engine, GUI, scheduler
2. AdvancedLivePortrait Plugin
- Node Definitions (nodes.py)
- Module Library (LivePortrait/modules)
- Config Loader (LivePortrait/config)
- Utils Layer (LivePortrait/utils)
3. Pre-trained Models – files referenced by models.yaml
4. Sample Data & Workflows – demonstration JSON graphs, videos, expression data
5. External Services/Libraries – PyTorch runtime, GPU, face-tracking, ComfyUI-Manager registry
b. Relationships and interactions
1. User places nodes in ComfyUI canvas → invokes node callbacks in nodes.py
2. Each node loads config parameters and calls into specific module classes
3. Modules chain data:
Source Image/Video → Appearance Feature Extractor → Motion Extractor
→ Dense Motion → SPADE Generator → Warping Network
→ Stitching & Retargeting Network → Final Output Frames
4. Utils handle file I/O, video decoding/encoding, timing, and resource logging
5. Sample workflows glue these nodes together in predefined sequences
6. Models are loaded from locations in models.yaml; inference_config.py supplies runtime flags
c. Architectural patterns & principles
- Modular decomposition: each neural-network stage lives in its own module file
- Pipeline pattern: data flows sequentially through feature extraction, motion analysis, synthesis
- Plugin-based extension: clean separation between ComfyUI core and AdvancedLivePortrait code
- Configuration-driven: models.yaml and inference_config.py control hyperparameters and file paths
d. Relevant technologies/frameworks
- Python 3.x, PyTorch (model inference)
- ComfyUI (node graph framework)
- OpenCV/FFmpeg (video I/O via utils/video.py)
- Face‐tracking library (in utils/face_analysis_diy.py)

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "ComfyUI Core"
        CU["ComfyUI Core"]:::blue
    end

    subgraph "Plugin Node Layer"
        NP["nodes.py"]:::blue
        WP["live_portrait_wrapper.py"]:::blue
    end

    CU --> NP
    NP --> WP

    subgraph "Configuration & Models"
        CI["inference_config.py"]:::orange
        MY["models.yaml"]:::orange
    end

    WP --> CI
    WP --> MY

    subgraph "Module Library"
        AFE["Appearance Feature Extractor"]:::green
        ME["Motion Extractor"]:::green
        DM["Dense Motion"]:::green
        SG["SPADE Generator"]:::green
        WN["Warping Network"]:::green
        SRN["Stitching & Retargeting Network"]:::green
        CN["ConvNextV2"]:::green
        MU["modules/util.py"]:::green
    end

    CI --> AFE
    AFE --> ME
    ME --> DM
    DM --> SG
    SG --> WN
    WN --> SRN
    WP --> CN
    WP --> MU

    subgraph "Utils Layer"
        CAM["camera.py"]:::orange
        FA["face_analysis_diy.py"]:::orange
        HE["helper.py"]:::orange
        IO["io.py"]:::orange
        TI["timer.py"]:::orange
        VID["video.py"]:::orange
        RP["rprint.py"]:::orange
        RS["resources/"]:::orange
    end

    WP --> CAM
    WP --> FA
    WP --> HE
    WP --> IO
    WP --> TI
    WP --> VID
    WP --> RP
    WP --> RS

    subgraph "Sample Data & Workflows"
        DV["driving_video.mp4"]:::gray
        SA["original_sample_asset/"]:::gray
        EX["sample_exp_data.exp"]:::gray
        W1["Advanced_Animate_without_vid.json"]:::gray
        W2["Advanced_Inserting_expressions_into_vid.json"]:::gray
        W3["Bacis_Extracting_expression_from_photo.json"]:::gray
        W4["Bacis_Liveportrait_with_expression_editing.json"]:::gray
        W5["Bacis_Simple_expression_editing.json"]:::gray
    end

    NP --> W1
    NP --> W2
    NP --> W3
    NP --> W4
    NP --> W5
    SA --> IO
    DV --> IO
    EX --> IO

    subgraph "Packaging & CI"
        PY["pyproject.toml"]:::gray
        RT["requirements.txt"]:::gray
        IB["install.bat"]:::gray
        CIY[".github/workflows/publish.yml"]:::gray
    end

    classDef blue fill:#AECFFF,stroke:#0366d6,color:#000
    classDef green fill:#C6F6D5,stroke:#2F855A,color:#000
    classDef orange fill:#FBD38D,stroke:#DD6B20,color:#000
    classDef gray fill:#E2E8F0,stroke:#A0AEC0,color:#000

    click NP "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/nodes.py"
    click WP "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/live_portrait_wrapper.py"
    click CI "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/config/inference_config.py"
    click MY "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/config/models.yaml"
    click AFE "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/appearance_feature_extractor.py"
    click ME "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/motion_extractor.py"
    click DM "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/dense_motion.py"
    click SG "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/spade_generator.py"
    click WN "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/warping_network.py"
    click SRN "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/stitching_retargeting_network.py"
    click CN "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/convnextv2.py"
    click MU "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/modules/util.py"
    click CAM "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/utils/camera.py"
    click FA "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/utils/face_analysis_diy.py"
    click HE "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/utils/helper.py"
    click IO "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/utils/io.py"
    click TI "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/utils/timer.py"
    click VID "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/utils/video.py"
    click RP "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/LivePortrait/utils/rprint.py"
    click RS "https://github.com/powerhouseman/comfyui-advancedliveportrait/tree/main/LivePortrait/utils/resources/"
    click DV "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/sample/driving_video.mp4"
    click SA "https://github.com/powerhouseman/comfyui-advancedliveportrait/tree/main/sample/original_sample_asset/"
    click EX "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/sample/sample_exp_data.exp"
    click W1 "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/sample/workflows/Advanced_Animate_without_vid.json"
    click W2 "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/sample/workflows/Advanced_Inserting_expressions_into_vid.json"
    click W3 "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/sample/workflows/Bacis_Extracting_expression_from_photo.json"
    click W4 "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/sample/workflows/Bacis_Liveportrait_with_expression_editing.json"
    click W5 "https://github.com/powerhouseman/comfyui-advancedliveportrait/blob/main/sample/workflows/Bacis_Simple_expression_editing.json"
```
