# zju3dv/easymocap architecture

> Architecture diagram of the GitHub repository zju3dv/easymocap, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/zju3dv/easymocap
- Repository: https://github.com/zju3dv/easymocap

## Overview

##### Project Overview and Purpose
- EasyMocap is an open‐source toolbox developed for markerless human motion capture and novel view synthesis from RGB videos.
- It supports multiple scenarios—from controlled multi‐view setups (with calibrated cameras) to internet videos (including mirror setups) and even multi‐person scenes. It integrates 2D keypoint estimation, 3D body model fitting (SMPL, MANO, SMPL-X, etc.), and downstream visualization/demo rendering.
- Funding and citations in the README indicate integration of research components (e.g., HRNet, YOLOv4, SPIN, and Neural Body).

##### Architectural Components (High-Level Modules)
- Data Input and Preprocessing:
– Components for reading videos, calibration and camera parameter estimation (see apps/calibration and scripts/dataset).
– Preprocessing modules (apps/preprocess) for image extraction, keypoints extraction, and tracking.
- 2D/3D Estimation and Annotation:
– Estimators for 2D human pose (using HRNet, OpenPose, or MediaPipe as wrapped under easymocap/estimator).
- Model Fitting and Body Modeling:
– Core model fitting pipelines using SMPL-based modules (easymocap/bodymodel, easymocap/pyfitting, and myeasymocap) that fit parametric models (SMPL, SMPL-X, MANO, etc.) onto input keypoints.
– Multistage pipelines (easymocap/multistage) and optimization modules (operations, pyfitting) that integrate iterative solving and loss functions.
- Postprocessing, Rendering, and Novel View Synthesis:
– Postprocessing scripts (apps/postprocess) and configuration (config/render) for converting fitted results into final renderable output.
– Novel view synthesis pipelines also integrated with the neuralbody components (apps/neuralbody and config/neuralbody).
- Integration and Pipeline Coordination:
– Pipeline and configuration management: a large set of configuration files (config/*) and pipeline modules (easymocap/pipeline) organize the execution flow for different scenarios (mirror, multi-person, etc.).
– Different “apps” folders (annotation, calibration, demo, mocap, etc.) provide user-facing scripts or demo entry points, each wrapping different parts of the processing pipeline.

##### External Dependencies and Third-Party Libraries
- The project bundles 3rd-party libraries such as Eigen (for linear algebra) and Pybind11 (bridging C++ and Python code)–these underpin several lower-level modules (like library/pymatch and some performance-critical parts of the fitting code).
- Other dependencies include key deep learning frameworks and detectors that have been integrated or wrapped (e.g., HRNet, YOLOv4, MediaPipe).

## Diagram (Mermaid)

```mermaid
graph TD
    %% Input Layer
    A["Preprocessing (Data Input)"]:::inputLayer

    %% Estimation & Annotation Layer
    B1["2D Pose Estimation"]:::estimationLayer
    B2["Annotation Tools (App)"]:::estimationLayer
    B3["Annotation Tools (Annotator)"]:::estimationLayer

    %% Model Fitting & Body Modeling Layer
    C1["Body Model Fitting"]:::modelFitting
    C2["Fitting Demo"]:::modelFitting

    %% Multistage Pipelines & Optimization Layer
    D1["Multistage Pipelines"]:::multistage
    D2["Optimization Modules"]:::multistage

    %% Postprocessing & Rendering Layer
    E1["Postprocessing"]:::postprocessing
    E2["Rendering Configuration"]:::postprocessing
    E3["Neural Synthesis"]:::postprocessing

    %% Visualization Layer
    F1["Visualization - Vis"]:::visualization
    F2["Visualization - Vis3D"]:::visualization
    F3["Visualization - Visualizer"]:::visualization

    %% Integration & Coordination
    G1["Pipeline Management"]:::integration
    G2["Central Configuration"]:::integration

    %% External Dependencies & Libraries
    H1["Eigen"]:::external
    H2["Pybind11"]:::external
    H3["Performance Matching"]:::external

    %% Main Data Flow
    A --> B1
    B1 --> B2
    B2 --> B3
    B3 --> C1
    C1 --> C2
    C2 --> D1
    D1 --> D2
    D2 --> E1
    E1 --> E2
    E2 --> E3
    E3 --> F1
    F1 --> F2
    F2 --> F3

    %% Integration Connections (dashed lines)
    G2 --- A
    G1 --- D2

    %% External Dependencies Connections (dashed lines)
    H1 --- C1
    H2 --- C1
    H3 --- D2

    %% Click Events
    click A "https://github.com/zju3dv/easymocap/tree/master/apps/preprocess"
    click B1 "https://github.com/zju3dv/easymocap/tree/master/easymocap/estimator"
    click B2 "https://github.com/zju3dv/easymocap/tree/master/apps/annotation"
    click B3 "https://github.com/zju3dv/easymocap/tree/master/easymocap/annotator"
    click C1 "https://github.com/zju3dv/easymocap/tree/master/easymocap/bodymodel"
    click C2 "https://github.com/zju3dv/easymocap/tree/master/apps/fit"
    click D1 "https://github.com/zju3dv/easymocap/tree/master/easymocap/multistage"
    click D2 "https://github.com/zju3dv/easymocap/tree/master/myeasymocap/operations"
    click E1 "https://github.com/zju3dv/easymocap/tree/master/apps/postprocess"
    click E2 "https://github.com/zju3dv/easymocap/tree/master/config/render"
    click E3 "https://github.com/zju3dv/easymocap/tree/master/apps/neuralbody"
    click F1 "https://github.com/zju3dv/easymocap/tree/master/apps/vis"
    click F2 "https://github.com/zju3dv/easymocap/tree/master/apps/vis3d"
    click F3 "https://github.com/zju3dv/easymocap/tree/master/easymocap/visualize"
    click G1 "https://github.com/zju3dv/easymocap/tree/master/easymocap/pipeline"
    click G2 "https://github.com/zju3dv/easymocap/tree/master/config"
    click H1 "https://github.com/zju3dv/easymocap/tree/master/3rdparty/eigen"
    click H2 "https://github.com/zju3dv/easymocap/tree/master/3rdparty/pybind11"
    click H3 "https://github.com/zju3dv/easymocap/tree/master/library/pymatch"

    %% Styles
    classDef inputLayer fill:#AED6F1,stroke:#1F618D,stroke-width:2px;
    classDef estimationLayer fill:#ABEBC6,stroke:#229954,stroke-width:2px;
    classDef modelFitting fill:#F9E79F,stroke:#B7950B,stroke-width:2px;
    classDef multistage fill:#E8DAEF,stroke:#8E44AD,stroke-width:2px;
    classDef postprocessing fill:#FADBD8,stroke:#C0392B,stroke-width:2px;
    classDef visualization fill:#D6EAF8,stroke:#2471A3,stroke-width:2px;
    classDef integration fill:#FCF3CF,stroke:#B9770E,stroke-width:2px;
    classDef external fill:#D5F5E3,stroke:#27AE60,stroke-width:2px;
```
