How city96/comfyui-gguf works
Project Type and Purpose
• This is an open-source “plugin” repository (ComfyUI custom nodes) plus a small toolkit of scripts.
• Purpose: provide ComfyUI users with custom nodes that load and quantize models in the GGUF format, and supply CLI tools to convert/fix GGUF files.
File Structure
• Top-level Python package (root files: __init__.py, loader.py, nodes.py, ops.py, dequant.py).
• pyproject.toml / requirements.txt for dependencies (notably gguf).
• tools/ folder containing standalone scripts (convert.py, fix_5d_tensors.py, etc.) and its own README.
• No web server or UI code—this integrates into the existing ComfyUI host environment.
Extract Insights from the README
• Core feature: new “Unet Loader (GGUF)” and “CLIPLoader (gguf)” nodes under ComfyUI’s “bootleg” category.
• External dependency: gguf Python package.
• Workflow: place .gguf files in ComfyUI’s models folder; use nodes in a user-defined graph.
Main Components
a. ComfyUI Host Application
b. Custom Nodes Package (this repository)
• Node Definitions (nodes.py)
• Loader Logic (loader.py)
• Tensor Operations & Dequant (ops.py, dequant.py)
c. External Library
• gguf Python package (handles on-disk reading, quantization metadata)
d. Model Files
• .gguf-formatted UNET and T5/CLIP files on disk
e. Tools Subsystem
• CLI utilities (tools/convert.py, etc.) for creating/fixing GGUF files
f. (Optional) Downstream ComfyUI Graph Execution Engine
Relationships and Data Flows
• User opens ComfyUI → it discovers custom nodes from this package.
• In a graph, a “GGUF Unet Loader” node calls into loader.py, which:
• Uses gguf library to read binary data
• Passes raw tensors through dequant.py and ops.py to create PyTorch tensors
• Emits a loaded model object downstream in the ComfyUI graph.
• The “CLIPLoader” node follows a similar path for text-encoder files.
• Separately, a user runs tools/convert.py → reads a standard checkpoint → writes a .gguf file.
Architectural Patterns
• Plugin/Extension pattern: this repo lives in ComfyUI/custom_nodes and uses ComfyUI’s node discovery.
• Separation of concerns:
• Node definitions (exposed API) vs. loader internals vs. tensor ops vs. conversion tools.
• Batch processing CLI tools isolated from the runtime graph.
Select Technologies/Frameworks
• Python 3.x, PyTorch (via ComfyUI), gguf (for on-disk format support)
• CLI scripts use standard argparse/file I/O
• Node discovery and registration
• Data flow during graph execution
• Offline conversion flow