# yawiii/comfyui-prompt-assistant architecture

> Architecture diagram of the GitHub repository yawiii/comfyui-prompt-assistant, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yawiii/comfyui-prompt-assistant
- Repository: https://github.com/yawiii/comfyui-prompt-assistant

## Overview

ComfyUI Prompt Assistant is a custom-node extension for prompt editing, translation, image/video captioning, and related workflow utilities. The main user journey is to interact with an assistant or node in ComfyUI, submit text or media for processing, and receive generated or translated content in the workflow. The README documents additional UI features including tags, history, Markdown and node-document translation. Connections below rely on sampled excerpts, SOURCE INDEX references, and README statements; unsampled subsystem internals are represented conservatively.

Main entry points: [`services/vlm.py`](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/vlm.py) (Vision model service), [`js/modules/PromptAssistant.js`](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/PromptAssistant.js) (Prompt assistant), [`server.py`](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/server.py) (Extension API), [`services/llm.py`](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/llm.py) (Language model service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["ComfyUI Interface"]
  node_assistant["Prompt assistant<br/>[PromptAssistant.js]"]
  node_assistant_ui["Assistant container"]
  node_tags["Tag library<br/>[tag.js]"]
  node_history["Prompt history<br/>[history.js]"]
  node_caption_ui["Image caption UI<br/>[imageCaption.js]"]
  node_doc_translate["Document translation"]
  node_markdown_translate["Markdown translation"]
  node_api_client["API client<br/>[api.js]"]
end

subgraph group_nodes["Workflow Nodes"]
  node_expand_node["Prompt optimization<br/>[expand_node.py]"]
  node_translate_node["Translation node<br/>[translate_node.py]"]
  node_image_node["Image caption node"]
  node_video_node["Video caption node"]
  node_kontext_node["Kontext presets"]
  node_node_base["Node foundations<br/>[base_node.py]"]
end

subgraph group_backend["Request Backend"]
  node_http_api["Extension API<br/>[server.py]"]
  node_llm["Language model service<br/>[llm.py]"]
  node_vision["Vision model service<br/>[vlm.py]"]
  node_baidu["Baidu translation<br/>[baidu.py]"]
  node_thinking["Thinking controls"]
end

subgraph group_state["Configuration"]
  node_config[("Settings and presets<br/>[config_manager.py]")]
end

node_user(("ComfyUI user"))
node_provider(("Model provider"))

node_user -->|"edits prompts"| node_assistant
node_assistant -->|"inserts tags"| node_tags
node_assistant -->|"uses history"| node_history
node_assistant -->|"sends requests"| node_api_client
node_assistant -->|"uses container"| node_assistant_ui
node_caption_ui -->|"submits images"| node_api_client
node_doc_translate -.->|"requests translation"| node_api_client
node_markdown_translate -.->|"requests translation"| node_api_client
node_user -->|"runs node"| node_expand_node
node_user -->|"runs node"| node_translate_node
node_user -->|"runs node"| node_image_node
node_user -->|"runs node"| node_video_node
node_user -->|"uses preset"| node_kontext_node
node_expand_node -->|"uses foundation"| node_node_base
node_translate_node -.->|"uses foundation"| node_node_base
node_image_node -->|"uses foundation"| node_node_base
node_video_node -->|"uses foundation"| node_node_base
node_api_client -->|"calls backend"| node_http_api
node_http_api -->|"dispatches requests"| node_llm
node_http_api -->|"dispatches requests"| node_vision
node_http_api -->|"dispatches translation"| node_baidu
node_expand_node -->|"requests generation"| node_llm
node_image_node -->|"requests analysis"| node_vision
node_video_node -->|"requests analysis"| node_vision
node_llm -->|"loads settings"| node_config
node_vision -->|"loads settings"| node_config
node_http_api -->|"updates configuration"| node_config
node_llm -->|"selects controls"| node_thinking
node_vision -->|"selects controls"| node_thinking
node_llm -.->|"sends model request"| node_provider
node_vision -.->|"sends model request"| node_provider

click node_assistant "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/PromptAssistant.js"
click node_assistant_ui "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/AssistantContainer.js"
click node_tags "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/tag.js"
click node_history "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/history.js"
click node_caption_ui "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/imageCaption.js"
click node_doc_translate "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/nodeHelpTranslator.js"
click node_markdown_translate "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/utils/markdownNoteTranslate.js"
click node_api_client "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/services/api.js"
click node_expand_node "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/expand_node.py"
click node_translate_node "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/translate_node.py"
click node_image_node "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/image_caption_node.py"
click node_video_node "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/video_caption_node.py"
click node_kontext_node "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/kontext_preset_node.py"
click node_node_base "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/base/base_node.py"
click node_http_api "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/server.py"
click node_llm "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/llm.py"
click node_vision "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/vlm.py"
click node_baidu "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/baidu.py"
click node_config "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/config_manager.py"
click node_thinking "https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/thinking_control.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_assistant,node_assistant_ui,node_tags,node_history,node_caption_ui,node_doc_translate,node_markdown_translate,node_api_client,node_user toneBlue
class node_expand_node,node_translate_node,node_image_node,node_video_node,node_kontext_node,node_node_base toneAmber
class node_http_api,node_llm,node_vision,node_baidu,node_thinking toneMint
class node_config toneRose
class node_provider toneIndigo
```

## Components

### ComfyUI Interface

- [Prompt assistant](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/PromptAssistant.js): `js/modules/PromptAssistant.js`
- [Assistant container](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/AssistantContainer.js): `js/modules/AssistantContainer.js`
- [Tag library](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/tag.js): `js/modules/tag.js`
- [Prompt history](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/history.js): `js/modules/history.js`
- [Image caption UI](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/imageCaption.js): `js/modules/imageCaption.js`
- [Document translation](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/modules/nodeHelpTranslator.js): `js/modules/nodeHelpTranslator.js`
- [Markdown translation](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/utils/markdownNoteTranslate.js): `js/utils/markdownNoteTranslate.js`
- [API client](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/js/services/api.js): `js/services/api.js`

### Workflow Nodes

- [Prompt optimization](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/expand_node.py): `node/expand_node.py`
- [Translation node](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/translate_node.py): `node/translate_node.py`
- [Image caption node](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/image_caption_node.py): `node/image_caption_node.py`
- [Video caption node](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/video_caption_node.py): `node/video_caption_node.py`
- [Kontext presets](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/kontext_preset_node.py): `node/kontext_preset_node.py`
- [Node foundations](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/node/base/base_node.py): `node/base/base_node.py`

### Request Backend

- [Extension API](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/server.py): `server.py`
- [Language model service](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/llm.py): `services/llm.py`
- [Vision model service](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/vlm.py): `services/vlm.py`
- [Baidu translation](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/baidu.py): `services/baidu.py`
- [Thinking controls](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/services/thinking_control.py): `services/thinking_control.py`

### Configuration

- [Settings and presets](https://github.com/yawiii/comfyui-prompt-assistant/blob/main/config_manager.py): `config_manager.py`

### Other components

- **ComfyUI user**
- **Model provider**

## Connections

- ComfyUI user → Prompt assistant: edits prompts (evidence: `README.md`)
- Prompt assistant → Tag library: inserts tags (evidence: `js/modules/PromptAssistant.js`)
- Prompt assistant → Prompt history: uses history (evidence: `js/modules/PromptAssistant.js`)
- Prompt assistant → API client: sends requests (evidence: `js/modules/PromptAssistant.js`)
- Prompt assistant → Assistant container: uses container (evidence: `js/modules/PromptAssistant.js`)
- Image caption UI → API client: submits images (evidence: `js/modules/imageCaption.js`)
- Document translation → API client: requests translation
- Markdown translation → API client: requests translation
- ComfyUI user → Prompt optimization: runs node (evidence: `README.md`)
- ComfyUI user → Translation node: runs node (evidence: `README.md`)
- ComfyUI user → Image caption node: runs node (evidence: `README.md`)
- ComfyUI user → Video caption node: runs node (evidence: `README.md`)
- ComfyUI user → Kontext presets: uses preset (evidence: `README.md`)
- Prompt optimization → Node foundations: uses foundation (evidence: `node/expand_node.py`)
- Translation node → Node foundations: uses foundation
- Image caption node → Node foundations: uses foundation (evidence: `node/image_caption_node.py`)
- Video caption node → Node foundations: uses foundation (evidence: `node/video_caption_node.py`)
- API client → Extension API: calls backend (evidence: `js/services/api.js`)
- Extension API → Language model service: dispatches requests (evidence: `server.py`)
- Extension API → Vision model service: dispatches requests (evidence: `server.py`)
- Extension API → Baidu translation: dispatches translation (evidence: `server.py`)
- Prompt optimization → Language model service: requests generation (evidence: `node/expand_node.py`)
- Image caption node → Vision model service: requests analysis (evidence: `node/image_caption_node.py`)
- Video caption node → Vision model service: requests analysis (evidence: `node/video_caption_node.py`)
- Language model service → Settings and presets: loads settings (evidence: `services/llm.py`)
- Vision model service → Settings and presets: loads settings (evidence: `services/vlm.py`)
- Extension API → Settings and presets: updates configuration (evidence: `server.py`)
- Language model service → Thinking controls: selects controls (evidence: `services/llm.py`)
- Vision model service → Thinking controls: selects controls (evidence: `services/vlm.py`)
- Language model service → Model provider: sends model request
- Vision model service → Model provider: sends model request
