# muntelco/comfyui architecture

> Architecture diagram of the GitHub repository muntelco/comfyui, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muntelco/comfyui
- Repository: https://github.com/muntelco/comfyui

## Overview

#### Purpose and entry points
ComfyUI is a local, Python-based node-graph execution engine for image, video, audio, and 3D generation. It serves an installed frontend plus HTTP/WebSocket APIs and can run workflows programmatically. Primary startup is `main.py`, which composes runtime configuration, model/node discovery, execution, and the server; request handling centers on `server.py`. Programmatic examples are in `script_examples/basic_api_example.py` and `script_examples/websockets_api_example.py`.

#### Server and frontend boundary
The backend exposes workflow submission, queueing, progress, output, and asset-facing endpoints to a separately developed, packaged frontend. `server.py` is the primary transport/control boundary, with protocol structures in `protocol.py`; the frontend lifecycle/version management is isolated in `app/frontend_management.py`. TLS is configured at launch rather than requiring a reverse-proxy component in this repository. `openapi.yaml` defines the public API contract.

#### Workflow graph and execution
Submitted prompts/workflows are represented, validated, and scheduled as directed node graphs. `comfy_execution/graph.py` and `comfy_execution/validation.py` establish graph semantics and input validity; `execution.py` drives node evaluation. Job lifecycle, cancellation, progress, and partial re-execution/caching are separated into `comfy_execution/jobs.py`, `comfy_execution/progress.py`, and `comfy_execution/caching.py`. Only valid output-reachable subgraphs run; caching avoids recomputation for unchanged graph portions.

#### Node ecosystem and extension boundary
`nodes.py` is the central node registry/runtime integration point. Core operations, model loaders, samplers, and output nodes are available there; broad built-in feature nodes live in `comfy_extras/`, such as `comfy_extras/nodes_video.py` and `comfy_extras/nodes_controlnet.py`. Third-party local extensions are loaded from `custom_nodes/`, while `app/custom_node_manager.py` governs optional manager behavior. The versioned SDK/type surface for node authors is `comfy_api/latest/_io.py` and `comfy_api/latest/_ui.py`.

#### Local inference runtime
PyTorch-based inference support is concentrated in `comfy/`: model format detection/loading (`comfy/model_detection.py`, `comfy/sd.py`), sampling (`comfy/samplers.py`, `comfy/sample.py`), and device/VRAM offloading (`comfy/model_management.py`, `comfy/memory_management.py`). Architecture-specific implementations are grouped beneath `comfy/ldm/`, e.g. `comfy/ldm/flux/model.py` and `comfy/ldm/wan/model.py`. This layer supports CUDA, ROCm, Intel XPU, Apple Metal, CPU, and additional PyTorch hardware backends.

#### Models, files, and assets
Model storage is filesystem-backed and categorized beneath `models/`; `folder_paths.py` resolves discoverable model locations, including optional external roots through `extra_model_paths.yaml.example`. Generated artifacts use `output/`; inputs use `input/`. A separate asset catalog provides ingestion, metadata/tagging, references, and cursor-based APIs: `app/assets/api/routes.py`, `app/assets/services/ingest.py`, and `app/assets/database/models.py`. Its schema evolves through Alembic migrations in `alembic_db/versions/`.

#### External services and distribution
Optional API nodes call paid remote model providers through provider adapters in `comfy_api_nodes/apis/` and node wrappers in `comfy_api_nodes/`; examples include `comfy_api_nodes/apis/openai.py` and `comfy_api_nodes/apis/gemini.py`. Core local inference remains offline unless these nodes or frontend acquisition options are enabled. Dependencies are Python/PyTorch (`requirements.txt`, `pyproject.toml`); Windows portable launchers target NVIDIA, AMD, Intel, or CPU under `.ci/windows_nvidia_base_files/` and related directories.

Main entry points: [`server.py`](https://github.com/muntelco/comfyui/blob/master/server.py) (HTTP/WebSocket server), [`nodes.py`](https://github.com/muntelco/comfyui/blob/master/nodes.py) (Node registry), [`execution.py`](https://github.com/muntelco/comfyui/blob/master/execution.py) (Node graph executor), [`main.py`](https://github.com/muntelco/comfyui/blob/master/main.py) (Application startup)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_transport["Transport & Frontend"]
  node_server["HTTP/WebSocket server<br/>transport controller<br/>[server.py]"]
  node_protocol["Protocol structures<br/>API protocol<br/>[protocol.py]"]
  node_openapi["Public API contract<br/>OpenAPI spec<br/>[openapi.yaml]"]
  node_frontend["Packaged frontend lifecycle<br/>frontend management"]
end

subgraph group_execution["Workflow Execution"]
  node_graph["Workflow graph semantics<br/>graph model<br/>[graph.py]"]
  node_validation["Workflow validation<br/>graph validation<br/>[validation.py]"]
  node_executor["Node graph executor<br/>execution engine<br/>[execution.py]"]
  node_jobs["Job lifecycle<br/>queue and cancellation<br/>[jobs.py]"]
  node_progress_cache["Progress &amp; execution cache<br/>execution state<br/>[caching.py]"]
end

subgraph group_nodes["Node Ecosystem"]
  node_registry["Node registry<br/>node runtime<br/>[nodes.py]"]
  node_builtin_nodes["Built-in feature nodes<br/>node collection"]
  node_custom_nodes["Local extensions<br/>third-party nodes"]
  node_node_sdk["Node author SDK<br/>versioned API surface<br/>[_io.py]"]
  node_api_nodes["Remote provider nodes<br/>optional API adapters"]
end

subgraph group_inference["Local Inference"]
  node_model_runtime["Model loading &amp; detection<br/>PyTorch model runtime<br/>[sd.py]"]
  node_sampling["Sampling runtime<br/>local inference<br/>[samplers.py]"]
  node_memory["Device &amp; VRAM management<br/>hardware runtime"]
end

subgraph group_assets["Models &amp; Assets"]
  node_model_storage[("Model storage &amp; discovery<br/>filesystem models<br/>[folder_paths.py]")]
  node_asset_catalog[("Asset catalog<br/>asset service<br/>[ingest.py]")]
  node_asset_routes["Asset API routes<br/>asset transport<br/>[routes.py]"]
end

node_main{{"Application startup<br/>Python entry point<br/>[main.py]"}}

node_main -->|"starts"| node_server
node_main -->|"discovers nodes"| node_registry
node_main -->|"initializes"| node_executor
node_server -->|"uses"| node_protocol
node_server -->|"implements"| node_openapi
node_server -->|"serves/manages"| node_frontend
node_server -->|"submits workflows"| node_graph
node_server -->|"queues jobs"| node_jobs
node_server -->|"mounts"| node_asset_routes
node_graph -->|"validates"| node_validation
node_validation -->|"permits valid graph"| node_executor
node_jobs -->|"schedules"| node_executor
node_executor -->|"updates"| node_progress_cache
node_executor -->|"evaluates nodes"| node_registry
node_registry -->|"loads"| node_builtin_nodes
node_registry -.->|"loads"| node_custom_nodes
node_node_sdk -.->|"authoring surface"| node_custom_nodes
node_registry -.->|"registers optional nodes"| node_api_nodes
node_registry -->|"loads models"| node_model_runtime
node_registry -->|"invokes"| node_sampling
node_model_runtime -->|"resolves files"| node_model_storage
node_sampling -->|"uses devices"| node_memory
node_asset_routes -->|"ingests and queries"| node_asset_catalog

click node_main "https://github.com/muntelco/comfyui/blob/master/main.py"
click node_server "https://github.com/muntelco/comfyui/blob/master/server.py"
click node_protocol "https://github.com/muntelco/comfyui/blob/master/protocol.py"
click node_openapi "https://github.com/muntelco/comfyui/blob/master/openapi.yaml"
click node_frontend "https://github.com/muntelco/comfyui/blob/master/app/frontend_management.py"
click node_graph "https://github.com/muntelco/comfyui/blob/master/comfy_execution/graph.py"
click node_validation "https://github.com/muntelco/comfyui/blob/master/comfy_execution/validation.py"
click node_executor "https://github.com/muntelco/comfyui/blob/master/execution.py"
click node_jobs "https://github.com/muntelco/comfyui/blob/master/comfy_execution/jobs.py"
click node_progress_cache "https://github.com/muntelco/comfyui/blob/master/comfy_execution/caching.py"
click node_registry "https://github.com/muntelco/comfyui/blob/master/nodes.py"
click node_node_sdk "https://github.com/muntelco/comfyui/blob/master/comfy_api/latest/_io.py"
click node_model_runtime "https://github.com/muntelco/comfyui/blob/master/comfy/sd.py"
click node_sampling "https://github.com/muntelco/comfyui/blob/master/comfy/samplers.py"
click node_memory "https://github.com/muntelco/comfyui/blob/master/comfy/model_management.py"
click node_model_storage "https://github.com/muntelco/comfyui/blob/master/folder_paths.py"
click node_asset_catalog "https://github.com/muntelco/comfyui/blob/master/app/assets/services/ingest.py"
click node_asset_routes "https://github.com/muntelco/comfyui/blob/master/app/assets/api/routes.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_server,node_protocol,node_openapi,node_frontend toneBlue
class node_graph,node_validation,node_executor,node_jobs,node_progress_cache toneAmber
class node_registry,node_builtin_nodes,node_custom_nodes,node_node_sdk,node_api_nodes toneMint
class node_model_runtime,node_sampling,node_memory toneRose
class node_model_storage,node_asset_catalog,node_asset_routes toneIndigo
class node_main toneNeutral
```

## Components

### Transport & Frontend

- [HTTP/WebSocket server](https://github.com/muntelco/comfyui/blob/master/server.py): transport controller · Accepts workflows and exposes queue, progress, output, and asset APIs. · `server.py`
- [Protocol structures](https://github.com/muntelco/comfyui/blob/master/protocol.py): API protocol · `protocol.py`
- [Public API contract](https://github.com/muntelco/comfyui/blob/master/openapi.yaml): OpenAPI spec · `openapi.yaml`
- [Packaged frontend lifecycle](https://github.com/muntelco/comfyui/blob/master/app/frontend_management.py): frontend management · `app/frontend_management.py`

### Workflow Execution

- [Workflow graph semantics](https://github.com/muntelco/comfyui/blob/master/comfy_execution/graph.py): graph model · `comfy_execution/graph.py`
- [Workflow validation](https://github.com/muntelco/comfyui/blob/master/comfy_execution/validation.py): graph validation · `comfy_execution/validation.py`
- [Node graph executor](https://github.com/muntelco/comfyui/blob/master/execution.py): execution engine · Evaluates valid output-reachable subgraphs. · `execution.py`
- [Job lifecycle](https://github.com/muntelco/comfyui/blob/master/comfy_execution/jobs.py): queue and cancellation · `comfy_execution/jobs.py`
- [Progress & execution cache](https://github.com/muntelco/comfyui/blob/master/comfy_execution/caching.py): execution state · Reports progress and skips unchanged graph portions. · `comfy_execution/caching.py`

### Node Ecosystem

- [Node registry](https://github.com/muntelco/comfyui/blob/master/nodes.py): node runtime · Integrates core operations, loaders, samplers, and outputs. · `nodes.py`
- **Built-in feature nodes**: node collection · Video, ControlNet, and other optional built-in operations.
- **Local extensions**: third-party nodes
- [Node author SDK](https://github.com/muntelco/comfyui/blob/master/comfy_api/latest/_io.py): versioned API surface · `comfy_api/latest/_io.py`
- **Remote provider nodes**: optional API adapters · Node wrappers access paid remote model providers when enabled.

### Local Inference

- [Model loading & detection](https://github.com/muntelco/comfyui/blob/master/comfy/sd.py): PyTorch model runtime · `comfy/sd.py`
- [Sampling runtime](https://github.com/muntelco/comfyui/blob/master/comfy/samplers.py): local inference · `comfy/samplers.py`
- [Device & VRAM management](https://github.com/muntelco/comfyui/blob/master/comfy/model_management.py): hardware runtime · Supports offloading across PyTorch hardware backends. · `comfy/model_management.py`

### Models & Assets

- [Model storage & discovery](https://github.com/muntelco/comfyui/blob/master/folder_paths.py): filesystem models · Resolves categorized local and optional external model roots. · `folder_paths.py`
- [Asset catalog](https://github.com/muntelco/comfyui/blob/master/app/assets/services/ingest.py): asset service · Ingests assets and serves metadata, tags, references, and cursor APIs. · `app/assets/services/ingest.py`
- [Asset API routes](https://github.com/muntelco/comfyui/blob/master/app/assets/api/routes.py): asset transport · `app/assets/api/routes.py`

### Other components

- [Application startup](https://github.com/muntelco/comfyui/blob/master/main.py): Python entry point · Composes configuration, discovery, execution, and server. · `main.py`

## Connections

- Application startup → HTTP/WebSocket server: starts
- Application startup → Node registry: discovers nodes
- Application startup → Node graph executor: initializes
- HTTP/WebSocket server → Protocol structures: uses
- HTTP/WebSocket server → Public API contract: implements
- HTTP/WebSocket server → Packaged frontend lifecycle: serves/manages
- HTTP/WebSocket server → Workflow graph semantics: submits workflows
- HTTP/WebSocket server → Job lifecycle: queues jobs
- HTTP/WebSocket server → Asset API routes: mounts
- Workflow graph semantics → Workflow validation: validates
- Workflow validation → Node graph executor: permits valid graph
- Job lifecycle → Node graph executor: schedules
- Node graph executor → Progress & execution cache: updates
- Node graph executor → Node registry: evaluates nodes
- Node registry → Built-in feature nodes: loads
- Node registry → Local extensions: loads
- Node author SDK → Local extensions: authoring surface
- Node registry → Remote provider nodes: registers optional nodes
- Node registry → Model loading & detection: loads models
- Node registry → Sampling runtime: invokes
- Model loading & detection → Model storage & discovery: resolves files
- Sampling runtime → Device & VRAM management: uses devices
- Asset API routes → Asset catalog: ingests and queries