How kijai/comfyui-hunyuan3dwrapper works
1. This repository is a ComfyUI custom-node wrapper around Tencent’s Hunyuan3D-2 pipeline. The top-level responsibility is to take 2D image inputs and produce 3D assets, with the graph exposed through nodes.py, frontend glue in web/js/jsnodes.js, and example node graphs in example_workflows/.
2. The runtime is split into two major inference subsystems: hy3dgen/shapegen for geometry generation and hy3dgen/texgen for texture generation. These are orchestrated by package-level pipeline modules rather than a single monolith, which makes the shape and texture stages separable in an architecture diagram.
3. Shape generation is centered in hy3dgen/shapegen/pipelines.py and the models/ subtree. That layer includes preprocessing, conditioning, diffusion/denoising, schedulers, and postprocessing. The model stack combines transformer/DiT-style components, autoencoders, and surface extraction utilities to turn conditioned latent predictions into a mesh or volume representation.
4. Geometry extraction and representation conversion are handled by supporting modules such as surface_extractors.py, volume_decoders.py, and the standalone ODC/occupancy_dual_contouring.py. This indicates the shape subsystem is not only generating latent geometry but also explicitly converting occupancy/volume outputs into watertight mesh surfaces.
5. Texture generation is a separate pipeline under hy3dgen/texgen. It contains mesh and UV utilities, multiview alignment, texture warping, mesh simplification, and dehighlighting helpers, plus a hunyuanpaint submodule for paint/inpainting-style texture synthesis. The texture stage consumes the mesh produced by shape generation and writes UV-consistent appearance back onto it.
6. The texture subsystem depends on two native acceleration layers: custom_rasterizer and differentiable_renderer. The rasterizer package includes C++/CUDA kernels and OBJ/GLB I/O, while the differentiable renderer includes mesh processing and rendering code, plus a compiled mesh_processor extension used for vertex inpainting. These are performance-critical boundaries in the architecture.
7. Input normalization and asset preparation are handled before either model stage. The repo includes hy3dgen/rembg.py for background removal, preprocessors/postprocessors in both shapegen and texgen, and various utility modules for image alignment, UV handling, and mesh cleanup. This suggests the runtime expects relatively clean foreground-object images as the main input.
8. There is also an optional BPT-based shape branch under hy3dgen/shapegen/bpt. It contains a separate model implementation, Michelangelo/BPT-specific modules, and its own requirements/weight download path. Architecturally, this is an alternate geometry pathway rather than the main inference path.
9. The hy3dshape/ package is the training/fine-tuning side of the codebase, distinct from hy3dgen/ inference. It contains data loaders, diffusion/flow-matching code, denoisers, autoencoders, training utilities, EMA, logging callbacks, and visualization helpers. If you are drawing boundaries, this is the model development subsystem rather than the ComfyUI runtime subsystem.
10. Configuration and model loading are externalized. YAML configs live in configs/ and hy3dshape/configs/, while the README points to Hugging Face-hosted checkpoints and diffusers-style models that are auto-downloaded at runtime. That means the repo code is mostly orchestration and model glue, with weights supplied separately.
11. The end-to-end data flow is: user input image in ComfyUI -> preprocessing/background removal -> shape generation -> surface extraction/mesh formation -> optional UV/texture generation -> postprocessing and export/preview. The outputs are 3D assets suitable for ComfyUI’s 3D preview and export paths, with OBJ/GLB support visible in the rasterizer and renderer utilities.
12. Operationally, this repo has a mixed Python + native-extension architecture. Python coordinates the graph, model loading, and data flow; C++/CUDA components accelerate rasterization and mesh operations; and ComfyUI provides the host UI/runtime. That separation is the main architectural boundary to capture in a diagram.
Main entry points: shapegen/pipelines.py Shape Pipe · texgen/pipelines.py Texture Pipe · nodes.py Nodes · surface_extractors.py Surface Decode
Components 20
ComfyUI Host
Wrapper layer exposed to ComfyUI
- Nodes · Custom node entrypoints and runtime wiring
nodes.py - Frontend JS · Browser-side node integration for ComfyUI
web/js/jsnodes.js
Shape Generation
Geometry inference subsystem
- Shape Pipe · Coordinates shape generation from conditioned inputs
hy3dgen/shapegen/pipelines.py - Shape Preprocess · Image normalization and conditioning preparation
hy3dgen/shapegen/preprocessors.py - Shape Models · Core geometry models including conditioner, DiT, and autoencoder pieces
hy3dgen/shapegen/models/hunyuan3ddit.py - Surface Decode · Volume decoding and surface extraction into usable geometry
hy3dgen/shapegen/models/autoencoders/surface_extractors.py - Dual Contouring · Occupancy-to-mesh extraction utility
ODC/occupancy_dual_contouring.py - Shape Postprocess · Geometry cleanup and final output handling
hy3dgen/shapegen/postprocessors.py - Background Removal · Foreground extraction before geometry and texture stages
hy3dgen/rembg.py
Texture Generation
Appearance inference subsystem
- Texture Pipe · Coordinates texture synthesis and UV-consistent painting
hy3dgen/texgen/pipelines.py - Texture Utils · Alignment, UV warping, multiview, simplification, and dehighlighting
hy3dgen/texgen/utils/uv_warp_utils.py - HunyuanPaint · Paint and inpainting-style texture generation branch
hy3dgen/texgen/hunyuanpaint/pipeline.py
Native Acceleration
C++/CUDA rendering and mesh ops
- Rasterizer · Native rasterization and mesh IO acceleration
hy3dgen/texgen/custom_rasterizer/custom_rasterizer/render.py - Renderer · Differentiable rendering and mesh processing layer
hy3dgen/texgen/differentiable_renderer/mesh_render.py - Mesh Processor · Native mesh processing used for vertex inpainting and cleanup
hy3dgen/texgen/differentiable_renderer/mesh_processor.cpp
Training Stack
Fine-tuning and model development subsystem
- Shape Training · Fine-tuning and diffusion/flow-matching training stack
hy3dshape/minimal_demo.py - Train Models · Training-side denoisers, diffusion transport, and autoencoders
hy3dshape/hy3dshape/models/denoisers/hunyuandit.py - Train Utils · EMA, logging, visualizers, and training callbacks
hy3dshape/hy3dshape/utils/utils.py
Examples and Configs
Workflows and config assets
- Example Graphs · Sample ComfyUI workflows for end-to-end usage
example_workflows/hy3d_example_01.json - Configs · Model and training configuration files
configs/dit_config.yaml
Connections 23
- Nodes → Shape Pipe: invokes · Custom nodes call into the geometry pipeline
- Nodes → Texture Pipe: invokes · Custom nodes expose texture generation stages
- Frontend JS → Nodes: binds · Frontend glue maps UI controls to custom nodes
- Example Graphs → Nodes: demonstrates · Example workflows show how the wrapper is used
- Shape Pipe → Background Removal: prepares input · Shape generation starts with foreground cleanup
- Shape Pipe → Shape Preprocess: uses · Pipeline applies input preprocessing and conditioning
- Shape Preprocess → Shape Models: feeds · Preprocessed inputs are passed to the geometry model stack
- Shape Pipe → Shape Models: orchestrates · Main geometry pipeline coordinates the core model stack
- Shape Models → Surface Decode: decodes · Latent geometry is converted into surfaces/volumes
- Surface Decode → Dual Contouring: can use · Occupancy outputs may be converted with dual contouring
- Surface Decode → Shape Postprocess: emits · Extracted geometry flows into postprocessing
- Shape Pipe → Shape Postprocess: finishes · Pipeline returns cleaned mesh/geometry outputs
- Texture Pipe → Texture Utils: prepares mesh · Texture pipeline uses UV, alignment, and multiview helpers
- Texture Pipe → HunyuanPaint: orchestrates · Texture generation delegates synthesis to the painting branch
- Texture Pipe → Rasterizer: accelerates · Rasterization is handled by native CUDA/C++ code
- Texture Pipe → Renderer: renders · Differentiable rendering supports texture projection and refinement
- Renderer → Mesh Processor: extends · Compiled mesh processing backs renderer-side mesh edits
- Texture Utils → Renderer: supports · Mesh and UV utilities prepare data for rendering and painting
- Shape Postprocess → Texture Pipe: hands off · Generated geometry becomes the input mesh for texturing
- Shape Training → Train Models: trains · Training pipelines drive the model implementations
- Train Models → Train Utils: uses · Model development relies on training helpers and visualizers
- Shape Training → Configs: loads · Training and model settings are externalized in YAML configs
- Shape Pipe → Configs: loads · Inference pipelines also depend on config files