# shenhanqian/vhap architecture

> Architecture diagram of the GitHub repository shenhanqian/vhap, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2024-12-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shenhanqian/vhap
- Repository: https://github.com/shenhanqian/vhap

## Overview

##### Main Components

a) Core Processing Pipeline:
- Input Processing Layer
- Video Input Handler (monocular/multiview)
- Image Processing Components
- FLAME Model Integration Layer
- Optimization Engine
- Output Generation Layer

b) Supporting Systems:
- Asset Management
- Configuration System
- Utility Services
- Visualization Tools

**[Input Sources] → [Preprocessing] → [Core Pipeline] → [Output Generation] → [Export Formats]**

##### Detailed Component Breakdown

a) Input Processing Layer:
- Monocular Video Input
- NeRSemble Dataset Input
- Video Dataset Handler
- Image Folder Dataset Handler

b) FLAME Integration:
- FLAME Model Component
- Landmark Detection Systems
- Mesh Processing
- UV Mapping

c) Core Processing:
- Photometric optimization pipeline:
- Differentiable Mesh Rasterization
- Appearance Prior Extraction
- Regional Alignment Processing
- Optimization Controller

d) Utility Services (as side components):
- Camera Utilities
- Landmark Detection
- Rendering Services (nvdiffrast)
- Visualization Tools

##### Key Relationships to Highlight

- Data flow from input processing through the pipeline
- Feedback loops in the optimization process
- Utility services to relevant main components

##### Technical Details

- Python/CUDA integration points
- Configuration parameters
- Integration points with external libraries (nvdiffrast, FLAME)
- Data transformation steps

##### Implementation Notes

- How /vhap/model/ components interact
- How /vhap/util/ services support the main pipeline
- Demonstrate how /vhap/data/ handlers feed into the system
- Configuration management from /vhap/config/

##### Export Capabilities

A section showing export options:
- NeRF Dataset Export
- GaussianAvatars Integration
- Visualization Outputs

## Diagram (Mermaid)

```mermaid
graph TB
    subgraph Input
        VideoInput["Video Input Handler"]:::input
        NersembleInput["NeRSemble Dataset Handler"]:::input
        ImageInput["Image Folder Handler"]:::input
        PreProcess["Preprocessing"]:::input
    end

    subgraph FLAME["FLAME Integration"]
        FlameCore["FLAME Core"]:::flame
        LBS["Linear Blend Skinning"]:::flame
        FlameAssets["FLAME Assets"]:::flame
        FlameTools["FLAME Tools"]:::flame
    end

    subgraph OptEngine["Optimization Engine"]
        Tracker["Tracker"]:::opt
        TrackPipeline["Main Tracking Pipeline"]:::opt
        NersembleTrack["NeRSemble Tracking"]:::opt
    end

    subgraph Utils["Utility Services"]
        Camera["Camera Utilities"]:::util
        Landmark["Landmark Detection"]:::util
        Rendering["Rendering Services"]:::util
        MeshOps["Mesh Operations"]:::util
        Viz["Visualization"]:::util
    end

    subgraph Config["Configuration System"]
        BaseConfig["Base Configuration"]:::config
        NersembleConfig["NeRSemble Configuration"]:::config
    end

    subgraph Output["Output Generation"]
        NerfExport["NeRF Dataset Export"]:::output
        DatasetCombine["Dataset Combination"]:::output
    end

    subgraph DataMgmt["Data Management"]
        NerfData["NeRF Dataset Handling"]:::data
        VideoData["Video Dataset Handling"]:::data
        ImageData["Image Dataset Handling"]:::data
    end

    VideoInput --> PreProcess
    NersembleInput --> PreProcess
    ImageInput --> PreProcess
    PreProcess --> TrackPipeline

    FlameCore --> Tracker
    LBS --> FlameCore
    FlameAssets --> FlameCore
    FlameTools --> FlameCore

    Tracker --> TrackPipeline
    Tracker --> NersembleTrack

    Camera --> TrackPipeline
    Landmark --> TrackPipeline
    Rendering --> TrackPipeline
    MeshOps --> TrackPipeline
    Viz --> TrackPipeline

    BaseConfig --> TrackPipeline
    NersembleConfig --> NersembleTrack

    TrackPipeline --> NerfExport
    NersembleTrack --> NerfExport
    NerfExport --> DatasetCombine

    NerfData --> DataMgmt
    VideoData --> DataMgmt
    ImageData --> DataMgmt

    %% Click events
    click VideoInput "https://github.com/ShenhanQian/VHAP/blob/main/vhap/data/video_dataset.py"
    click NersembleInput "https://github.com/ShenhanQian/VHAP/blob/main/vhap/data/nersemble_dataset.py"
    click ImageInput "https://github.com/ShenhanQian/VHAP/blob/main/vhap/data/image_folder_dataset.py"
    click PreProcess "https://github.com/ShenhanQian/VHAP/blob/main/vhap/preprocess_video.py"
    click FlameCore "https://github.com/ShenhanQian/VHAP/blob/main/vhap/model/flame.py"
    click LBS "https://github.com/ShenhanQian/VHAP/blob/main/vhap/model/lbs.py"
    click FlameAssets "https://github.com/ShenhanQian/VHAP/tree/main/asset/flame/"
    click FlameTools "https://github.com/ShenhanQian/VHAP/blob/main/vhap/flame_editor.py"
    click Tracker "https://github.com/ShenhanQian/VHAP/blob/main/vhap/model/tracker.py"
    click TrackPipeline "https://github.com/ShenhanQian/VHAP/blob/main/vhap/track.py"
    click NersembleTrack "https://github.com/ShenhanQian/VHAP/blob/main/vhap/track_nersemble.py"
    click Camera "https://github.com/ShenhanQian/VHAP/blob/main/vhap/util/camera.py"
    click Landmark "https://github.com/ShenhanQian/VHAP/blob/main/vhap/util/landmark_detector_fa.py"
    click Rendering "https://github.com/ShenhanQian/VHAP/blob/main/vhap/util/render_nvdiffrast.py"
    click MeshOps "https://github.com/ShenhanQian/VHAP/blob/main/vhap/util/mesh.py"
    click Viz "https://github.com/ShenhanQian/VHAP/blob/main/vhap/util/visualization.py"
    click BaseConfig "https://github.com/ShenhanQian/VHAP/blob/main/vhap/config/base.py"
    click NersembleConfig "https://github.com/ShenhanQian/VHAP/blob/main/vhap/config/nersemble.py"
    click NerfExport "https://github.com/ShenhanQian/VHAP/blob/main/vhap/export_as_nerf_dataset.py"
    click DatasetCombine "https://github.com/ShenhanQian/VHAP/blob/main/vhap/combine_nerf_datasets.py"
    click NerfData "https://github.com/ShenhanQian/VHAP/blob/main/vhap/data/nerf_dataset.py"
    click VideoData "https://github.com/ShenhanQian/VHAP/blob/main/vhap/data/video_dataset.py"
    click ImageData "https://github.com/ShenhanQian/VHAP/blob/main/vhap/data/image_folder_dataset.py"

    %% Styling
    classDef input fill:#90EE90
    classDef flame fill:#FFB6C1
    classDef opt fill:#FF6B6B
    classDef util fill:#87CEEB
    classDef config fill:#DDA0DD
    classDef output fill:#98FB98
    classDef data fill:#F0E68C
```
