# chenfei-wu/taskmatrix architecture

> Architecture diagram of the GitHub repository chenfei-wu/taskmatrix, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/chenfei-wu/taskmatrix
- Repository: https://github.com/chenfei-wu/taskmatrix

## Overview

TaskMatrix is a visual-chat application: a user describes an image task, ChatGPT coordinates visual foundation-model tools, and the application returns text or image results. Its documented capabilities include image generation, editing, question answering, image-to-control-map workflows, object detection, segmentation, and reusable multi-model templates. The repository also contains a separate LowCodeLLM service for planning and executing SOP-based tasks. The visual application is largely implemented in one file; unsampled wiring is omitted except where the README documents it.

Main entry points: [`visual_chatgpt.py`](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py) (Visual ChatGPT), [`LowCodeLLM/src/lowCodeLLM.py`](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/lowCodeLLM.py) (Workflow Coordinator), [`LowCodeLLM/src/planningLLM.py`](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/planningLLM.py) (Workflow Planner), [`LowCodeLLM/src/openAIWrapper.py`](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/openAIWrapper.py) (OpenAI Wrapper)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_visual_app["Visual Chat"]
  node_visual_chat["Visual ChatGPT<br/>[visual_chatgpt.py]"]
  node_image_input["Image Input<br/>[visual_chatgpt.py]"]
  node_image_output["Image Results<br/>[visual_chatgpt.py]"]
end

subgraph group_visual_models["Visual Models"]
  node_image_generation["Image Generation<br/>[visual_chatgpt.py]"]
  node_image_transforms["Image Transforms<br/>[visual_chatgpt.py]"]
  node_image_qa["Visual Q&amp;A<br/>[visual_chatgpt.py]"]
  node_text_detection["Text Detection<br/>[visual_chatgpt.py]"]
  node_segmentation["Segmentation<br/>[visual_chatgpt.py]"]
  node_inpainting["Inpainting<br/>[visual_chatgpt.py]"]
end

subgraph group_visual_templates["Model Templates"]
  node_templates["Workflow Templates<br/>[visual_chatgpt.py]"]
end

subgraph group_lowcode_service["LowCodeLLM Service"]
  node_lowcode_ui["LowCode UI<br/>[index.html]"]
  node_api["Flask API<br/>[app.py]"]
  node_lowcode["Workflow Coordinator<br/>[lowCodeLLM.py]"]
  node_planning["Workflow Planner<br/>[planningLLM.py]"]
  node_execution["SOP Executor<br/>[executingLLM.py]"]
  node_openai_wrapper["OpenAI Wrapper<br/>[openAIWrapper.py]"]
end

node_user(("User"))
node_lowcode_user(("Workflow Client"))
node_openai_service["OpenAI API"]

node_user -->|"describes task"| node_visual_chat
node_visual_chat -->|"sends images"| node_image_input
node_visual_chat -->|"requests generation"| node_image_generation
node_visual_chat -->|"requests transforms"| node_image_transforms
node_visual_chat -->|"requests answers"| node_image_qa
node_visual_chat -->|"locates objects"| node_text_detection
node_visual_chat -->|"requests masks"| node_segmentation
node_visual_chat -->|"uses templates"| node_templates
node_text_detection -->|"provides boxes"| node_segmentation
node_segmentation -->|"provides masks"| node_inpainting
node_inpainting -->|"returns edited image"| node_image_output
node_image_generation -.->|"returns image"| node_image_output
node_image_transforms -.->|"returns transformed image"| node_image_output
node_image_qa -.->|"returns answers"| node_visual_chat
node_image_output -->|"presents results"| node_user
node_lowcode_user -.->|"submits task"| node_lowcode_ui
node_lowcode_ui -.->|"calls endpoints"| node_api
node_api -->|"routes requests"| node_lowcode
node_lowcode -->|"plans workflow"| node_planning
node_lowcode -->|"extends workflow"| node_planning
node_lowcode -->|"executes workflow"| node_execution
node_planning -->|"requests completion"| node_openai_wrapper
node_execution -->|"requests completion"| node_openai_wrapper
node_openai_wrapper -->|"calls chat API"| node_openai_service

click node_visual_chat "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_image_input "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_image_output "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_image_generation "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_image_transforms "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_image_qa "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_text_detection "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_segmentation "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_inpainting "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_templates "https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py"
click node_lowcode_ui "https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/index.html"
click node_api "https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/app.py"
click node_lowcode "https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/lowCodeLLM.py"
click node_planning "https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/planningLLM.py"
click node_execution "https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/executingLLM.py"
click node_openai_wrapper "https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/openAIWrapper.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_visual_chat,node_image_input,node_image_output,node_user,node_lowcode_user toneBlue
class node_image_generation,node_image_transforms,node_image_qa,node_text_detection,node_segmentation,node_inpainting toneAmber
class node_templates,node_openai_service toneMint
class node_lowcode_ui,node_api,node_lowcode,node_planning,node_execution,node_openai_wrapper toneRose
```

## Components

### Visual Chat

- [Visual ChatGPT](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`
- [Image Input](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`
- [Image Results](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`

### Visual Models

- [Image Generation](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`
- [Image Transforms](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`
- [Visual Q&A](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`
- [Text Detection](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`
- [Segmentation](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`
- [Inpainting](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`

### Model Templates

- [Workflow Templates](https://github.com/chenfei-wu/taskmatrix/blob/main/visual_chatgpt.py): `visual_chatgpt.py`

### LowCodeLLM Service

- [LowCode UI](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/index.html): `LowCodeLLM/src/index.html`
- [Flask API](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/app.py): `LowCodeLLM/src/app.py`
- [Workflow Coordinator](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/lowCodeLLM.py): `LowCodeLLM/src/lowCodeLLM.py`
- [Workflow Planner](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/planningLLM.py): `LowCodeLLM/src/planningLLM.py`
- [SOP Executor](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/executingLLM.py): `LowCodeLLM/src/executingLLM.py`
- [OpenAI Wrapper](https://github.com/chenfei-wu/taskmatrix/blob/main/LowCodeLLM/src/openAIWrapper.py): `LowCodeLLM/src/openAIWrapper.py`

### Other components

- **User**
- **Workflow Client**
- **OpenAI API**

## Connections

- User → Visual ChatGPT: describes task (evidence: `README.md`)
- Visual ChatGPT → Image Input: sends images (evidence: `README.md`)
- Visual ChatGPT → Image Generation: requests generation (evidence: `README.md`)
- Visual ChatGPT → Image Transforms: requests transforms (evidence: `README.md`)
- Visual ChatGPT → Visual Q&A: requests answers (evidence: `README.md`)
- Visual ChatGPT → Text Detection: locates objects (evidence: `README.md`)
- Visual ChatGPT → Segmentation: requests masks (evidence: `README.md`)
- Visual ChatGPT → Workflow Templates: uses templates (evidence: `README.md`)
- Text Detection → Segmentation: provides boxes (evidence: `README.md`)
- Segmentation → Inpainting: provides masks (evidence: `README.md`)
- Inpainting → Image Results: returns edited image (evidence: `README.md`)
- Image Generation → Image Results: returns image (evidence: `visual_chatgpt.py`)
- Image Transforms → Image Results: returns transformed image (evidence: `visual_chatgpt.py`)
- Visual Q&A → Visual ChatGPT: returns answers (evidence: `visual_chatgpt.py`)
- Image Results → User: presents results (evidence: `README.md`)
- Workflow Client → LowCode UI: submits task
- LowCode UI → Flask API: calls endpoints
- Flask API → Workflow Coordinator: routes requests (evidence: `LowCodeLLM/src/app.py`)
- Workflow Coordinator → Workflow Planner: plans workflow (evidence: `LowCodeLLM/src/lowCodeLLM.py`)
- Workflow Coordinator → Workflow Planner: extends workflow (evidence: `LowCodeLLM/src/lowCodeLLM.py`)
- Workflow Coordinator → SOP Executor: executes workflow (evidence: `LowCodeLLM/src/lowCodeLLM.py`)
- Workflow Planner → OpenAI Wrapper: requests completion (evidence: `LowCodeLLM/src/planningLLM.py`)
- SOP Executor → OpenAI Wrapper: requests completion (evidence: `LowCodeLLM/src/executingLLM.py`)
- OpenAI Wrapper → OpenAI API: calls chat API (evidence: `LowCodeLLM/src/openAIWrapper.py`)