# kijai/comfyui-wananimatepreprocess architecture

> Architecture diagram of the GitHub repository kijai/comfyui-wananimatepreprocess, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kijai/comfyui-wananimatepreprocess
- Repository: https://github.com/kijai/comfyui-wananimatepreprocess

## Overview

1. **What this repo is**
   - This is a **ComfyUI custom nodes package** for the preprocessing stage of **Wan Video 2.2 Animate**.
   - Its job is not to generate video itself, but to **prepare pose- and face-related conditioning data**: whole-body keypoints, face crops, and keypoint lists suitable for downstream **SAM2 segmentation** or related animation pipelines.

2. **Top-level architecture**
   - The code is organized around a **node layer** that ComfyUI loads, plus several **utility/model wrapper layers** underneath.
   - The main visible boundaries are:
     - **ComfyUI node registration and orchestration**
     - **ONNX model loading/inference**
     - **Pose/keypoint conversion and visualization**
     - **Retargeting / preprocessing helpers**
     - **Example workflows and packaging**

3. **ComfyUI integration boundary**
   - `__init__.py` and `nodes.py` are the entry points for ComfyUI.
   - `nodes.py` is likely where the custom node classes are defined and exposed to the UI.
   - This is the boundary where ComfyUI inputs/outputs are translated into the repo’s internal pose-processing functions.

4. **Inference subsystem**
   - `models/onnx_models.py` and `onetoall/infer_function.py` form the model execution layer.
   - The README makes it clear the repo uses **ONNX models**, including:
     - **YOLOv10m ONNX** for detection
     - **ViTPose ONNX** for whole-body pose estimation
   - This layer is responsible for loading model artifacts from disk and running inference on images/frames.

5. **Model artifact boundary**
   - The repository does **not** vendor the model weights; they are external artifacts placed under `ComfyUI/models/detection`.
   - This matters architecturally because the code expects a **local model cache / filesystem boundary** rather than an API service.
   - The huge ViTPose model is split into two files (`.onnx` plus `.bin`) to work around ONNX size limits, so the model loader likely needs to handle that split format.

6. **Pose processing subsystem**
   - `pose_utils/pose2d_utils.py` and `pose_utils/human_visualization.py` are the pose-specific helpers.
   - Based on their names and the README, these utilities likely:
     - normalize or transform 2D keypoints
     - map model outputs into a consistent human-pose schema
     - render skeletons / overlays for debugging or preview
   - This is the layer that turns raw detector output into something usable by downstream nodes.

7. **Retargeting / preprocessing logic**
   - `retarget_pose.py` and the top-level `utils.py` suggest higher-level preprocessing steps built on top of raw keypoints.
   - In this repo’s context, “retargeting” likely means transforming pose data from detected people into a form suitable for animation transfer or conditioning.
   - This sits between inference and the final exported conditioning data.

8. **One-to-all support code**
   - The `onetoall/` package (`infer_function.py`, `utils.py`) appears to be a reusable helper module for inference-time operations.
   - Architecturally, it likely contains glue code for:
     - batching or dispatching model calls
     - image/array preprocessing
     - postprocessing model outputs into standardized formats
   - It acts as a shared service layer under the nodes, rather than as a user-facing component.

9. **Example workflow boundary**
   - `example_workflows/WanAnimate_native_example_01.json` shows how the nodes are expected to be wired in ComfyUI.
   - This is important for diagramming because it reveals the **data flow expected by the package**: input media → detection/pose extraction → crops/keypoints → downstream animation pipeline.
   - The example workflow is effectively a reference integration, not core runtime logic.

10. **External dependencies and runtime**
   - The repo is Python-based and packaged with `pyproject.toml` and `requirements.txt`.
   - Its runtime dependencies are driven by:
     - **ComfyUI** as the host environment
     - **ONNX / ONNX Runtime**-style model execution
     - likely image processing and numerical libraries used for pose decoding and visualization
   - The architecture assumes local execution with GPU/accelerated inference where available, but the repo itself is not a service backend.

11. **Operational output of the pipeline**
   - The practical outputs of the repo are:
     - whole-body pose keypoints
     - face crops
     - visualized pose overlays
     - keypoint lists that can feed segmentation or animation modules
   - These outputs are passed onward to other ComfyUI nodes or to downstream Wan Animate preprocessing steps.

12. **Packaging and release flow**
   - `.github/workflows/publish.yml` indicates the repo is packaged and published through GitHub Actions.
   - That means the architectural deployment boundary is a **ComfyUI extension install/update workflow**, not a standalone application deployment.

13. **How to draw the diagram**
   - Draw the center as a **ComfyUI custom node extension**.
   - Put **external model files on disk** as a separate dependency box.
   - Show the main flow as:
     - input image/frame from ComfyUI
     - detection and ViTPose ONNX inference
     - pose postprocessing / visualization / retargeting
     - output keypoints and crops to downstream nodes
   - Keep `pose_utils`, `onetoall`, and `models` as internal support subsystems behind the node layer.

Main entry points: [`nodes.py`](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/nodes.py) (Nodes), [`onetoall/infer_function.py`](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/onetoall/infer_function.py) (Infer Func), [`pose_utils/pose2d_utils.py`](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/pose_utils/pose2d_utils.py) (Pose2D), [`retarget_pose.py`](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/retarget_pose.py) (Retarget)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_comfyui["ComfyUI Extension"]
  node_init["Init<br/>entrypoint<br/>[__init__.py]"]
  node_nodes["Nodes<br/>[nodes.py]"]
end

subgraph group_inference["Inference Stack"]
  node_onnx_models["ONNX Loader<br/>model loader<br/>[onnx_models.py]"]
  node_infer_fn["Infer Func<br/>runtime inference<br/>[infer_function.py]"]
  node_oto_utils["O2A Utils<br/>inference helpers<br/>[utils.py]"]
end

subgraph group_pose["Pose Processing"]
  node_pose2d["Pose2D<br/>keypoint ops<br/>[pose2d_utils.py]"]
  node_viz["Visualize<br/>overlay render"]
  node_retarget["Retarget<br/>preprocess transform<br/>[retarget_pose.py]"]
  node_utils["Utils<br/>shared helpers<br/>[utils.py]"]
end

subgraph group_delivery["Packaging & Examples"]
  node_workflow["Example Flow<br/>comfyui workflow"]
  node_pyproject["Package Meta<br/>build config<br/>[pyproject.toml]"]
end

subgraph group_assets["Model Cache"]
  node_models_disk[("Model Files<br/>local artifacts")]
end

node_init -->|"loads"| node_nodes
node_nodes -->|"calls"| node_infer_fn
node_nodes -->|"uses"| node_pose2d
node_nodes -->|"uses"| node_retarget
node_infer_fn -->|"executes"| node_onnx_models
node_infer_fn -->|"uses"| node_oto_utils
node_onnx_models -->|"reads"| node_models_disk
node_pose2d -->|"renders"| node_viz
node_pose2d -->|"feeds"| node_retarget
node_utils -->|"supports"| node_pose2d
node_retarget -->|"depends"| node_utils
node_workflow -->|"demonstrates"| node_nodes
node_pyproject -->|"packages"| node_nodes
node_pyproject -->|"depends on"| node_infer_fn

click node_init "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/__init__.py"
click node_nodes "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/nodes.py"
click node_onnx_models "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/models/onnx_models.py"
click node_infer_fn "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/onetoall/infer_function.py"
click node_oto_utils "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/onetoall/utils.py"
click node_pose2d "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/pose_utils/pose2d_utils.py"
click node_viz "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/pose_utils/human_visualization.py"
click node_retarget "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/retarget_pose.py"
click node_utils "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/utils.py"
click node_workflow "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/example_workflows/WanAnimate_native_example_01.json"
click node_pyproject "https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/pyproject.toml"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_init,node_nodes toneBlue
class node_onnx_models,node_infer_fn,node_oto_utils toneAmber
class node_pose2d,node_viz,node_retarget,node_utils toneMint
class node_workflow,node_pyproject toneRose
class node_models_disk toneIndigo
```

## Components

### ComfyUI Extension

Custom node entrypoints and orchestration layer.

- [Init](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/__init__.py): entrypoint · Registers the package with ComfyUI. · `__init__.py`
- [Nodes](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/nodes.py): comfyui nodes · Defines the custom nodes exposed in the UI. · `nodes.py`

### Inference Stack

ONNX model loading and runtime execution.

- [ONNX Loader](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/models/onnx_models.py): model loader · Loads detector and pose ONNX artifacts from disk. · `models/onnx_models.py`
- [Infer Func](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/onetoall/infer_function.py): runtime inference · Runs image preprocessing, model dispatch, and postprocessing. · `onetoall/infer_function.py`
- [O2A Utils](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/onetoall/utils.py): inference helpers · Shared utility layer for inference-time array and image handling. · `onetoall/utils.py`

### Pose Processing

Keypoint normalization, visualization, and retargeting.

- [Pose2D](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/pose_utils/pose2d_utils.py): keypoint ops · Transforms and standardizes 2D pose keypoints. · `pose_utils/pose2d_utils.py`
- [Visualize](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/pose_utils/human_visualization.py): overlay render · Draws skeletons and pose previews for debugging. · `pose_utils/human_visualization.py`
- [Retarget](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/retarget_pose.py): preprocess transform · Converts detected pose data into animation-ready conditioning. · `retarget_pose.py`
- [Utils](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/utils.py): shared helpers · General-purpose preprocessing helpers used by pose flow. · `utils.py`

### Packaging & Examples

Distribution metadata and reference workflow.

- [Example Flow](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/example_workflows/WanAnimate_native_example_01.json): comfyui workflow · Reference graph showing the intended node wiring. · `example_workflows/WanAnimate_native_example_01.json`
- [Package Meta](https://github.com/kijai/comfyui-wananimatepreprocess/blob/main/pyproject.toml): build config · Packaging and dependency metadata for the extension. · `pyproject.toml`

### Model Cache

External model artifacts expected on local disk.

- **Model Files**: local artifacts · External YOLOv10m and ViTPose ONNX assets placed under the ComfyUI model cache.

## Connections

- Init → Nodes: loads · ComfyUI imports the node definitions through the package entrypoint.
- Nodes → Infer Func: calls · Node execution delegates preprocessing and inference work.
- Nodes → Pose2D: uses · Nodes convert raw results into the package pose schema.
- Nodes → Retarget: uses · Nodes expose retargeting/preprocessing operations.
- Infer Func → ONNX Loader: executes · Inference function loads and runs the ONNX models.
- Infer Func → O2A Utils: uses · Shared runtime helpers support preprocessing and batching.
- ONNX Loader → Model Files: reads · Model loader resolves artifacts from the local ComfyUI model directory.
- Pose2D → Visualize: renders · Keypoints are optionally visualized as overlays.
- Pose2D → Retarget: feeds · Normalized pose data drives retargeting transforms.
- Utils → Pose2D: supports · General helpers assist pose conversion and cleanup.
- Retarget → Utils: depends · Retargeting reuses shared preprocessing helpers.
- Example Flow → Nodes: demonstrates · Example workflow shows how the nodes are composed in ComfyUI.
- Package Meta → Nodes: packages · Build metadata defines the extension as a distributable package.
- Package Meta → Infer Func: depends on · Packaging captures runtime dependencies for inference.