muntelco/comfyui

Purpose and entry points

ComfyUI is a local, Python-based node-graph execution engine for image, video, audio, and 3D generation. It serves an installed frontend plus HTTP/WebSocket APIs and can run workflows programmatically. Primary startup is main.py, which composes runtime configuration, model/node discovery, execution, and the server; request handling centers on server.py. Programmatic examples are in script_examples/basic_api_example.py and script_examples/websockets_api_example.py.

Server and frontend boundary

The backend exposes workflow submission, queueing, progress, output, and asset-facing endpoints to a separately developed, packaged frontend. server.py is the primary transport/control boundary, with protocol structures in protocol.py; the frontend lifecycle/version management is isolated in app/frontend_management.py. TLS is configured at launch rather than requiring a reverse-proxy component in this repository. openapi.yaml defines the public API contract.

Workflow graph and execution

Submitted prompts/workflows are represented, validated, and scheduled as directed node graphs. comfy_execution/graph.py and comfy_execution/validation.py establish graph semantics and input validity; execution.py drives node evaluation. Job lifecycle, cancellation, progress, and partial re-execution/caching are separated into comfy_execution/jobs.py, comfy_execution/progress.py, and comfy_execution/caching.py. Only valid output-reachable subgraphs run; caching avoids recomputation for unchanged graph portions.

Node ecosystem and extension boundary

nodes.py is the central node registry/runtime integration point. Core operations, model loaders, samplers, and output nodes are available there; broad built-in feature nodes live in comfy_extras/, such as comfy_extras/nodes_video.py and comfy_extras/nodes_controlnet.py. Third-party local extensions are loaded from custom_nodes/, while app/custom_node_manager.py governs optional manager behavior. The versioned SDK/type surface for node authors is comfy_api/latest/_io.py and comfy_api/latest/_ui.py.

Local inference runtime

PyTorch-based inference support is concentrated in comfy/: model format detection/loading (comfy/model_detection.py, comfy/sd.py), sampling (comfy/samplers.py, comfy/sample.py), and device/VRAM offloading (comfy/model_management.py, comfy/memory_management.py). Architecture-specific implementations are grouped beneath comfy/ldm/, e.g. comfy/ldm/flux/model.py and comfy/ldm/wan/model.py. This layer supports CUDA, ROCm, Intel XPU, Apple Metal, CPU, and additional PyTorch hardware backends.

Models, files, and assets

Model storage is filesystem-backed and categorized beneath models/; folder_paths.py resolves discoverable model locations, including optional external roots through extra_model_paths.yaml.example. Generated artifacts use output/; inputs use input/. A separate asset catalog provides ingestion, metadata/tagging, references, and cursor-based APIs: app/assets/api/routes.py, app/assets/services/ingest.py, and app/assets/database/models.py. Its schema evolves through Alembic migrations in alembic_db/versions/.

External services and distribution

Optional API nodes call paid remote model providers through provider adapters in comfy_api_nodes/apis/ and node wrappers in comfy_api_nodes/; examples include comfy_api_nodes/apis/openai.py and comfy_api_nodes/apis/gemini.py. Core local inference remains offline unless these nodes or frontend acquisition options are enabled. Dependencies are Python/PyTorch (requirements.txt, pyproject.toml); Windows portable launchers target NVIDIA, AMD, Intel, or CPU under .ci/windows_nvidia_base_files/ and related directories.