# kijai/comfyui-dsine architecture

> Architecture diagram of the GitHub repository kijai/comfyui-dsine, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kijai/comfyui-dsine
- Repository: https://github.com/kijai/comfyui-dsine

## Overview

1. **Repository purpose**
   - This is a **ComfyUI integration for DSINE**, a surface-normal estimation model.
   - The repo combines two layers: the **core DSINE inference/model code** and a **ComfyUI node wrapper** that exposes it inside a workflow UI.

2. **Top-level architecture**
   - The most important boundary is between:
     - **UI/runtime integration**: `nodes.py` and `examples/DSINE_example_workflow_01.json`
     - **Model/inference implementation**: `models/`, `utils/`, and `test.py`
   - Documentation and visual assets live separately under `docs/`, and are not part of the runtime path.

3. **ComfyUI entry point**
   - `nodes.py` is the integration surface for ComfyUI.
   - This file is where the model is likely registered as a node, inputs/outputs are defined, and inference is triggered from the graph.
   - In an architecture diagram, treat this as the **adapter layer** between ComfyUI and the DSINE model code.

4. **Model subsystem**
   - `models/dsine.py` contains the main DSINE network definition and forward logic.
   - `models/submodules.py` holds reusable building blocks used by the model, such as intermediate layers, encoders/decoders, or helper modules.
   - This subsystem is the **core compute layer**: it takes an image plus camera geometry and produces surface normals.

5. **Geometry and rotation utilities**
   - `utils/rotation.py` likely contains the math for converting between rotations, vectors, and normal representations.
   - This is important because DSINE’s method explicitly models **relative rotation between neighboring surface normals**.
   - In the architecture, this sits between the neural network outputs and the geometric interpretation of normals.

6. **General utility layer**
   - `utils/utils.py` appears to house shared helpers such as image preprocessing, file/path handling, intrinsics loading, checkpoint loading, or tensor conversion utilities.
   - The README indicates the model can use either provided camera intrinsics or approximate intrinsics from a default field of view, so this utility layer likely handles that camera-parameter path.

7. **Standalone inference path**
   - `test.py` is the repo’s command-line evaluation/inference entry point.
   - It reads sample images from `samples/img/`, loads any matching `.txt` intrinsics files, runs the DSINE model, and writes results to `samples/output/`.
   - This script is the clearest end-to-end dataflow example outside ComfyUI.

8. **Input/output data flow**
   - The expected inference flow is:
     - image file in `samples/img/`
     - optional camera intrinsics from sibling `.txt` file
     - preprocessing and tensor conversion
     - DSINE model inference
     - predicted surface-normal output saved to an output directory
   - The architecture is therefore **image + camera geometry in, normal map out**.

9. **Camera model dependency**
   - Camera intrinsics are a first-class part of the pipeline, not optional metadata.
   - The README says performance improves with known intrinsics, but the code can fall back to approximate values using a **60° FOV assumption**.
   - That makes camera geometry a distinct subsystem in the architecture, adjacent to preprocessing and model execution.

10. **Examples and workflow packaging**
   - `examples/DSINE_example_workflow_01.json` is a ComfyUI workflow example showing how the node is wired in practice.
   - This file is not core logic, but it is useful for understanding the node’s expected inputs/outputs and how it fits into a broader graph.

11. **Documentation and presentation assets**
   - `docs/` contains the original project’s static documentation site assets: HTML, CSS, JS, images, and comparison visuals.
   - `paper.pdf` is included for reference, but it is not part of the runtime system.
   - These assets support understanding and validation, but do not participate in inference.

12. **External dependencies and runtime boundaries**
   - Runtime depends on **PyTorch** and likely additional model-specific packages listed in `requirements.txt`.
   - The README also mentions downloading pretrained weights into `./checkpoints/`, which is an external artifact dependency separate from source control.
   - So the deployable system is split across **source code + local checkpoint files + input images/intrinsics**.

13. **What to draw in an architecture diagram**
   - A good diagram would show:
     - ComfyUI graph / node runtime
     - `nodes.py` as the adapter
     - preprocessing + intrinsics handling
     - DSINE model core in `models/`
     - geometry helpers in `utils/rotation.py`
     - output surface-normal maps
     - optional standalone `test.py` inference path
     - external checkpoint weights as a separate dependency

Main entry points: [`utils/utils.py`](https://github.com/kijai/comfyui-dsine/blob/main/utils/utils.py) (Shared utils), [`nodes.py`](https://github.com/kijai/comfyui-dsine/blob/main/nodes.py) (ComfyUI node), [`models/dsine.py`](https://github.com/kijai/comfyui-dsine/blob/main/models/dsine.py) (DSINE model), [`test.py`](https://github.com/kijai/comfyui-dsine/blob/main/test.py) (Inference script)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime integration"]
  node_comfy_node["ComfyUI node<br/>adapter<br/>[nodes.py]"]
  node_cli_test["Inference script<br/>cli entrypoint<br/>[test.py]"]
  node_package_init["Package init<br/>package marker<br/>[__init__.py]"]
end

subgraph group_model["DSINE core"]
  node_dsine_model["DSINE model<br/>network<br/>[dsine.py]"]
  node_model_blocks["Model blocks<br/>modules<br/>[submodules.py]"]
end

subgraph group_geometry["Geometry utils"]
  node_rotation_math(("Rotation math<br/>geometry utils<br/>[rotation.py]"))
  node_shared_utils(("Shared utils<br/>preprocess/helpers<br/>[utils.py]"))
end

subgraph group_data["Inputs and outputs"]
  node_sample_images[("Sample images<br/>image inputs")]
  node_intrinsics_txt["Intrinsics files<br/>camera metadata<br/>[office_01.txt]"]
  node_output_dir[("Output dir<br/>normal outputs")]
  node_checkpoints[("Checkpoints<br/>external weights")]
end

subgraph group_docs["Docs and examples"]
  node_workflow_example["Workflow example<br/>comfyui workflow"]
  node_docs_site["Docs site<br/>static docs<br/>[index.html]"]
  node_readme["Project README<br/>overview<br/>[README.md]"]
end

node_comfy_node -->|"runs inference"| node_dsine_model
node_comfy_node -->|"preprocesses"| node_shared_utils
node_comfy_node -->|"interprets normals"| node_rotation_math
node_cli_test -->|"loads inputs"| node_shared_utils
node_cli_test -->|"executes"| node_dsine_model
node_dsine_model -->|"uses"| node_model_blocks
node_dsine_model -->|"outputs normals"| node_rotation_math
node_shared_utils -->|"loads weights"| node_checkpoints
node_shared_utils -->|"reads camera data"| node_intrinsics_txt
node_shared_utils -->|"ingests"| node_sample_images
node_cli_test -->|"writes results"| node_output_dir
node_workflow_example -->|"demonstrates"| node_comfy_node
node_docs_site -->|"documents"| node_readme
node_package_init -->|"exposes"| node_comfy_node

click node_comfy_node "https://github.com/kijai/comfyui-dsine/blob/main/nodes.py"
click node_workflow_example "https://github.com/kijai/comfyui-dsine/blob/main/examples/DSINE_example_workflow_01.json"
click node_cli_test "https://github.com/kijai/comfyui-dsine/blob/main/test.py"
click node_dsine_model "https://github.com/kijai/comfyui-dsine/blob/main/models/dsine.py"
click node_model_blocks "https://github.com/kijai/comfyui-dsine/blob/main/models/submodules.py"
click node_rotation_math "https://github.com/kijai/comfyui-dsine/blob/main/utils/rotation.py"
click node_shared_utils "https://github.com/kijai/comfyui-dsine/blob/main/utils/utils.py"
click node_sample_images "https://github.com/kijai/comfyui-dsine/tree/main/samples/img"
click node_intrinsics_txt "https://github.com/kijai/comfyui-dsine/blob/main/samples/img/office_01.txt"
click node_output_dir "https://github.com/kijai/comfyui-dsine/tree/main/samples/output"
click node_docs_site "https://github.com/kijai/comfyui-dsine/blob/main/docs/index.html"
click node_readme "https://github.com/kijai/comfyui-dsine/blob/main/README.md"
click node_package_init "https://github.com/kijai/comfyui-dsine/blob/main/__init__.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_comfy_node,node_cli_test,node_package_init toneBlue
class node_dsine_model,node_model_blocks toneAmber
class node_rotation_math,node_shared_utils toneMint
class node_sample_images,node_intrinsics_txt,node_output_dir,node_checkpoints toneRose
class node_workflow_example,node_docs_site,node_readme toneIndigo
```

## Components

### Runtime integration

ComfyUI adapter and standalone inference entrypoints.

- [ComfyUI node](https://github.com/kijai/comfyui-dsine/blob/main/nodes.py): adapter · Registers DSINE inside ComfyUI and bridges workflow inputs to model inference. · `nodes.py`
- [Inference script](https://github.com/kijai/comfyui-dsine/blob/main/test.py): cli entrypoint · Standalone image evaluation path that loads inputs, runs DSINE, and writes outputs. · `test.py`
- [Package init](https://github.com/kijai/comfyui-dsine/blob/main/__init__.py): package marker · Repository package entry point for Python importability. · `__init__.py`

### DSINE core

Neural network and reusable building blocks for surface-normal estimation.

- [DSINE model](https://github.com/kijai/comfyui-dsine/blob/main/models/dsine.py): network · Main surface-normal estimation network and forward pass logic. · `models/dsine.py`
- [Model blocks](https://github.com/kijai/comfyui-dsine/blob/main/models/submodules.py): modules · Reusable internal layers and components used by the DSINE network. · `models/submodules.py`

### Geometry utils

Rotation and normal-representation helpers.

- [Rotation math](https://github.com/kijai/comfyui-dsine/blob/main/utils/rotation.py): geometry utils · Conversions between rotations, vectors, and normal representations. · `utils/rotation.py`
- [Shared utils](https://github.com/kijai/comfyui-dsine/blob/main/utils/utils.py): preprocess/helpers · Common helpers for preprocessing, camera parameters, checkpoint loading, and tensor/file handling. · `utils/utils.py`

### Inputs and outputs

Sample images, intrinsics sidecars, and generated results.

- [Sample images](https://github.com/kijai/comfyui-dsine/tree/main/samples/img): image inputs · Example input images used by the standalone inference path. · `samples/img`
- [Intrinsics files](https://github.com/kijai/comfyui-dsine/blob/main/samples/img/office_01.txt): camera metadata · Optional per-image .txt camera intrinsics accompanying sample images. · `samples/img/office_01.txt`
- [Output dir](https://github.com/kijai/comfyui-dsine/tree/main/samples/output): normal outputs · Destination for predicted surface-normal results from the CLI path. · `samples/output`
- **Checkpoints**: external weights · Downloaded pretrained DSINE weights used at runtime but stored outside source control.

### Docs and examples

Workflow example and static documentation assets.

- [Workflow example](https://github.com/kijai/comfyui-dsine/blob/main/examples/DSINE_example_workflow_01.json): comfyui workflow · Example ComfyUI graph showing how the DSINE node is wired. · `examples/DSINE_example_workflow_01.json`
- [Docs site](https://github.com/kijai/comfyui-dsine/blob/main/docs/index.html): static docs · Static project documentation, visuals, and comparison assets. · `docs/index.html`
- [Project README](https://github.com/kijai/comfyui-dsine/blob/main/README.md): overview · Primary description of setup, usage, inputs, and model behavior. · `README.md`

## Connections

- ComfyUI node → DSINE model: runs inference · ComfyUI node delegates prediction to the DSINE network.
- ComfyUI node → Shared utils: preprocesses · Adapter uses shared helpers for input conversion and runtime setup.
- ComfyUI node → Rotation math: interprets normals · Geometry helpers support normal/rotation handling at the adapter boundary.
- Inference script → Shared utils: loads inputs · CLI script uses shared helpers for image, intrinsics, and checkpoint handling.
- Inference script → DSINE model: executes · Standalone inference script instantiates and runs the DSINE model.
- DSINE model → Model blocks: uses · Core network composes reusable model submodules.
- DSINE model → Rotation math: outputs normals · Model predictions are converted through rotation/normal math.
- Shared utils → Checkpoints: loads weights · Utility layer resolves pretrained model artifacts from checkpoint storage.
- Shared utils → Intrinsics files: reads camera data · Utility layer can parse per-image intrinsics sidecars.
- Shared utils → Sample images: ingests · Preprocessing starts from sample or user-provided input images.
- Inference script → Output dir: writes results · Predicted normal maps are saved to the output directory.
- Workflow example → ComfyUI node: demonstrates · Example workflow shows the ComfyUI node in context.
- Docs site → Project README: documents · Static docs and README together explain usage and results.
- Package init → ComfyUI node: exposes · Package entry point makes the ComfyUI integration importable.