# minuszoneai/comfyui-prompt-mz architecture

> Architecture diagram of the GitHub repository minuszoneai/comfyui-prompt-mz, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/minuszoneai/comfyui-prompt-mz
- Repository: https://github.com/minuszoneai/comfyui-prompt-mz

## Overview

1. **What this repo is**
   This is a **ComfyUI custom node package** for prompt-generation workflows. It does not define a standalone app; instead it plugs into ComfyUI and adds nodes for **prompt beautification, translation, and image interrogation/captioning**. The primary output of the package is text that can be fed back into ComfyUI text-conditioning nodes.

2. **Top-level integration boundary**
   The package boundary is the ComfyUI extension layer: `__init__.py` is the registration entrypoint, while `web/prompt_mz.js` adds frontend behavior for the custom nodes. The Python side defines node classes; the JS side handles UI wiring in the ComfyUI browser interface.

3. **Core node layer**
   `mz_llama_core_nodes.py` looks like the shared node implementation layer: common node definitions, parameter schemas, and orchestration for “CLIPTextEncode” style outputs. This is where the repo centralizes prompt assembly and the shared logic used by multiple backends.

4. **LLM backend adapters**
   The repo supports multiple model runtimes through separate adapters:
   - `mz_llama_cpp.py` for **llama.cpp / llama-cpp-python**
   - `mz_openaiapi.py` for **OpenAI-compatible API calls**
   - `mz_transformers.py` for **local Hugging Face / transformers models**
   - `mz_gen_translate.py` for translation-oriented generation
   These modules are backend boundaries: they normalize different inference providers into the same ComfyUI node style.

5. **Prompt and instruction pipeline**
   `mz_prompts.py` and `mz_prompt_utils.py` provide the reusable prompt templates, instruction text, formatting helpers, and post-processing logic. The README mentions custom instructions, chat format selection, and A1111-style weight scaling, so these utilities are the main place where raw model output is transformed into workflow-ready prompt text.

6. **Image interrogation path**
   Image-to-text functionality is a separate branch of the architecture. Nodes such as **LLava**, **Florence-2**, and the “ImageInterrogator” variants take an image input, run caption/interrogation inference, and return prompt text. The outputs can optionally be formatted for Stable Diffusion / A1111-style prompt usage.

7. **Florence-2 and model hooks**
   `hook/modeling_florence2.py` indicates a targeted runtime hook/override for Florence-2 integration, likely to adapt the upstream model behavior to ComfyUI’s execution model. This is an important boundary because it suggests the repo sometimes patches third-party model code rather than wrapping it only at the node level.

8. **Model selection and distribution**
   `configs/model_zoo.json` is the model catalog used by the downloader/select nodes. The repo supports both **manual model path selection** and **predefined/downloadable model presets**, which means model management is part of the architecture rather than being left entirely to the user.

9. **Auxiliary utilities**
   `half_json.py` is a resilience utility: it likely repairs or parses partially malformed JSON emitted by LLMs, which matters because prompt-generation models often return non-strict structured output. `ADV_CLIP_emb_encode.py` suggests compatibility with ComfyUI_ADV_CLIP_emb-style weighted prompt encoding behavior.

10. **Versioning and compatibility**
   The `v1/` folder contains older or deprecated node implementations (`mz_llama3.py`, `mz_llava.py`, `mz_phi3.py`, etc.), showing an explicit compatibility layer for legacy workflows. The newer “universal” and manual-select nodes appear to replace those older per-model nodes while preserving existing graphs.

11. **Execution flow**
   Typical data flow is: **user prompt or image input → node-specific preprocessing/instruction assembly → backend inference (llama.cpp, OpenAI, transformers, Ollama-like local model selection) → output post-processing → text returned to ComfyUI**. For image interrogation, the path starts with an image tensor and ends with generated caption/prompt text; for prompt beautification, it starts with text and returns revised text.

12. **Operational dependencies**
   The architecture materially depends on **ComfyUI**, **llama.cpp / llama-cpp-python**, and sometimes **OpenAI-compatible APIs** or local model runtimes. The repo is designed to be installed under `custom_nodes`, with model files managed externally rather than bundled into the package.

Main entry points: [`mz_llama_core_nodes.py`](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_llama_core_nodes.py) (Core nodes), [`mz_prompt_utils.py`](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_prompt_utils.py) (Prompt utils), [`v1/mz_deprecated.py`](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_deprecated.py) (Legacy nodes), [`__init__.py`](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/__init__.py) (Package init)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_g1["ComfyUI extension"]
  node_n1["Package init<br/>ComfyUI entrypoint<br/>[__init__.py]"]
  node_n2["Web UI<br/>ComfyUI frontend<br/>[prompt_mz.js]"]
end

subgraph group_g2["Prompt core"]
  node_n3["Core nodes<br/>shared node layer"]
  node_n4["Prompt utils<br/>format helpers<br/>[mz_prompt_utils.py]"]
  node_n5["Prompt templates<br/>instruction library<br/>[mz_prompts.py]"]
end

subgraph group_g3["Model backends"]
  node_n6["Llama.cpp<br/>local backend<br/>[mz_llama_cpp.py]"]
  node_n7["OpenAI API<br/>remote backend<br/>[mz_openaiapi.py]"]
  node_n8["Transformers<br/>HF backend<br/>[mz_transformers.py]"]
  node_n9["Translate gen<br/>translation backend"]
end

subgraph group_g4["Vision path"]
  node_n10(("Image nodes<br/>interrogation nodes"))
  node_n11["LLaVA path<br/>vision LLM<br/>[mz_llava.py]"]
  node_n12["Florence-2 path<br/>vision captioning"]
  node_n13["Florence hook<br/>model patch"]
end

subgraph group_g5["Compatibility"]
  node_n15["Legacy nodes<br/>compat layer<br/>[mz_deprecated.py]"]
  node_n16["Llama3 legacy<br/>legacy node<br/>[mz_llama3.py]"]
  node_n17["Phi3 legacy<br/>legacy node<br/>[mz_phi3.py]"]
  node_n18["JSON repair<br/>resilience util<br/>[half_json.py]"]
  node_n19["CLIP encode<br/>prompt encoding compat"]
end

subgraph group_g6["Model catalog"]
  node_n14["Model zoo<br/>catalog<br/>[model_zoo.json]"]
end

node_n1 -->|"registers"| node_n3
node_n2 -->|"drives"| node_n3
node_n3 -->|"uses"| node_n4
node_n3 -->|"builds from"| node_n5
node_n3 -->|"dispatches to"| node_n6
node_n3 -->|"dispatches to"| node_n7
node_n3 -->|"dispatches to"| node_n8
node_n3 -->|"dispatches to"| node_n9
node_n3 -->|"exposes"| node_n10
node_n10 -->|"includes"| node_n11
node_n10 -->|"includes"| node_n12
node_n12 -->|"adapts via"| node_n13
node_n10 -->|"formats with"| node_n4
node_n4 -->|"recovers"| node_n18
node_n4 -->|"aligns with"| node_n19
node_n3 -->|"selects from"| node_n14
node_n1 -.->|"keeps legacy"| node_n15
node_n15 -->|"contains"| node_n16
node_n15 -->|"contains"| node_n17

click node_n1 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/__init__.py"
click node_n2 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/web/prompt_mz.js"
click node_n3 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_llama_core_nodes.py"
click node_n4 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_prompt_utils.py"
click node_n5 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_prompts.py"
click node_n6 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_llama_cpp.py"
click node_n7 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_openaiapi.py"
click node_n8 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_transformers.py"
click node_n9 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_gen_translate.py"
click node_n11 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_llava.py"
click node_n13 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/hook/modeling_florence2.py"
click node_n14 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/configs/model_zoo.json"
click node_n15 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_deprecated.py"
click node_n16 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_llama3.py"
click node_n17 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_phi3.py"
click node_n18 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/half_json.py"
click node_n19 "https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/ADV_CLIP_emb_encode.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_n1,node_n2 toneBlue
class node_n3,node_n4,node_n5 toneAmber
class node_n6,node_n7,node_n8,node_n9 toneMint
class node_n10,node_n11,node_n12,node_n13 toneRose
class node_n15,node_n16,node_n17,node_n18,node_n19 toneIndigo
class node_n14 toneTeal
```

## Components

### ComfyUI extension

Package entrypoints and frontend wiring for the custom node extension.

- [Package init](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/__init__.py): ComfyUI entrypoint · Registers the custom nodes with ComfyUI. · `__init__.py`
- [Web UI](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/web/prompt_mz.js): ComfyUI frontend · Browser-side wiring for custom node controls and behavior. · `web/prompt_mz.js`

### Prompt core

Shared prompt assembly, formatting, and node orchestration logic.

- [Core nodes](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_llama_core_nodes.py): shared node layer · Common node definitions, schemas, and orchestration shared across backends. · `mz_llama_core_nodes.py`
- [Prompt utils](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_prompt_utils.py): format helpers · Reusable formatting, post-processing, and prompt transformation helpers. · `mz_prompt_utils.py`
- [Prompt templates](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_prompts.py): instruction library · Reusable prompt text, system instructions, and task-specific templates. · `mz_prompts.py`

### Model backends

Adapters that normalize different text-generation runtimes and APIs.

- [Llama.cpp](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_llama_cpp.py): local backend · Adapter for llama.cpp / llama-cpp-python style inference. · `mz_llama_cpp.py`
- [OpenAI API](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_openaiapi.py): remote backend · Adapter for OpenAI-compatible text generation APIs. · `mz_openaiapi.py`
- [Transformers](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_transformers.py): HF backend · Adapter for local Hugging Face / transformers model execution. · `mz_transformers.py`
- [Translate gen](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/mz_gen_translate.py): translation backend · Generation path specialized for translation-oriented prompt output. · `mz_gen_translate.py`

### Vision path

Image interrogation and captioning nodes and hooks.

- **Image nodes**: interrogation nodes · Image-to-text nodes for captioning and prompt generation.
- [LLaVA path](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_llava.py): vision LLM · Vision-language interrogation flow for captioning and prompt extraction. · `v1/mz_llava.py`
- **Florence-2 path**: vision captioning · Florence-2 based interrogation flow with model-specific handling.
- [Florence hook](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/hook/modeling_florence2.py): model patch · Targeted override or adaptation layer for Florence-2 model behavior. · `hook/modeling_florence2.py`

### Compatibility

Legacy and auxiliary modules that preserve older workflows or recover model output.

- [Legacy nodes](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_deprecated.py): compat layer · Older node implementations retained for existing workflows. · `v1/mz_deprecated.py`
- [Llama3 legacy](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_llama3.py): legacy node · Older model-specific node implementation kept for compatibility. · `v1/mz_llama3.py`
- [Phi3 legacy](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/v1/mz_phi3.py): legacy node · Older model-specific node implementation kept for compatibility. · `v1/mz_phi3.py`
- [JSON repair](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/half_json.py): resilience util · Helper for recovering or parsing malformed JSON emitted by models. · `half_json.py`
- [CLIP encode](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/ADV_CLIP_emb_encode.py): prompt encoding compat · Compatibility support for weighted prompt encoding behavior. · `ADV_CLIP_emb_encode.py`

### Model catalog

Preset model selection and download metadata.

- [Model zoo](https://github.com/minuszoneai/comfyui-prompt-mz/blob/main/configs/model_zoo.json): catalog · Preset model catalog used for selection and download workflows. · `configs/model_zoo.json`

## Connections

- Package init → Core nodes: registers · Package initialization exposes the core node set to ComfyUI.
- Web UI → Core nodes: drives · Frontend controls feed the shared node implementations.
- Core nodes → Prompt utils: uses · Core nodes rely on shared formatting and post-processing helpers.
- Core nodes → Prompt templates: builds from · Prompt assembly is driven by reusable instruction templates.
- Core nodes → Llama.cpp: dispatches to · Shared node layer can execute through llama.cpp.
- Core nodes → OpenAI API: dispatches to · Shared node layer can execute through OpenAI-compatible APIs.
- Core nodes → Transformers: dispatches to · Shared node layer can execute through local transformers models.
- Core nodes → Translate gen: dispatches to · Shared node layer can route into translation-oriented generation.
- Core nodes → Image nodes: exposes · Core package includes image interrogation node families.
- Image nodes → LLaVA path: includes · One branch uses LLaVA-style vision-language inference.
- Image nodes → Florence-2 path: includes · Another branch uses Florence-2 captioning/interrogation.
- Florence-2 path → Florence hook: adapts via · Florence-2 behavior is adjusted through a model hook.
- Image nodes → Prompt utils: formats with · Caption outputs are normalized into workflow-ready text.
- Prompt utils → JSON repair: recovers · Prompt utilities can depend on JSON repair for model output.
- Prompt utils → CLIP encode: aligns with · Formatting helpers support weighted prompt encoding conventions.
- Core nodes → Model zoo: selects from · Model-aware nodes can use the preset catalog for selection/download.
- Package init → Legacy nodes: keeps legacy · Entry point preserves compatibility with older node sets.
- Legacy nodes → Llama3 legacy: contains · Legacy compatibility includes older Llama3-specific nodes.
- Legacy nodes → Phi3 legacy: contains · Legacy compatibility includes older Phi3-specific nodes.
