How chenfei-wu/taskmatrix works
TaskMatrix is a visual-chat application: a user describes an image task, ChatGPT coordinates visual foundation-model tools, and the application returns text or image results. Its documented capabilities include image generation, editing, question answering, image-to-control-map workflows, object detection, segmentation, and reusable multi-model templates. The repository also contains a separate LowCodeLLM service for planning and executing SOP-based tasks. The visual application is largely implemented in one file; unsampled wiring is omitted except where the README documents it.
Main entry points: visual_chatgpt.py Visual ChatGPT · lowCodeLLM.py Workflow Coordinator · planningLLM.py Workflow Planner · openAIWrapper.py OpenAI Wrapper
Components 19
Visual Chat
- Visual ChatGPT
visual_chatgpt.py - Image Input
visual_chatgpt.py - Image Results
visual_chatgpt.py
Visual Models
- Image Generation
visual_chatgpt.py - Image Transforms
visual_chatgpt.py - Visual Q&A
visual_chatgpt.py - Text Detection
visual_chatgpt.py - Segmentation
visual_chatgpt.py - Inpainting
visual_chatgpt.py
Model Templates
- Workflow Templates
visual_chatgpt.py
LowCodeLLM Service
- LowCode UI
LowCodeLLM/src/index.html - Flask API
LowCodeLLM/src/app.py - Workflow Coordinator
LowCodeLLM/src/lowCodeLLM.py - Workflow Planner
LowCodeLLM/src/planningLLM.py - SOP Executor
LowCodeLLM/src/executingLLM.py - OpenAI Wrapper
LowCodeLLM/src/openAIWrapper.py
Other components
- User
- Workflow Client
- OpenAI API
Connections 24
- User → Visual ChatGPT: describes task
README.md - Visual ChatGPT → Image Input: sends images
README.md - Visual ChatGPT → Image Generation: requests generation
README.md - Visual ChatGPT → Image Transforms: requests transforms
README.md - Visual ChatGPT → Visual Q&A: requests answers
README.md - Visual ChatGPT → Text Detection: locates objects
README.md - Visual ChatGPT → Segmentation: requests masks
README.md - Visual ChatGPT → Workflow Templates: uses templates
README.md - Text Detection → Segmentation: provides boxes
README.md - Segmentation → Inpainting: provides masks
README.md - Inpainting → Image Results: returns edited image
README.md - Image Generation → Image Results: returns image
visual_chatgpt.py - Image Transforms → Image Results: returns transformed image
visual_chatgpt.py - Visual Q&A → Visual ChatGPT: returns answers
visual_chatgpt.py - Image Results → User: presents results
README.md - Workflow Client → LowCode UI: submits task
- LowCode UI → Flask API: calls endpoints
- Flask API → Workflow Coordinator: routes requests
LowCodeLLM/src/app.py - Workflow Coordinator → Workflow Planner: plans workflow
LowCodeLLM/src/lowCodeLLM.py - Workflow Coordinator → Workflow Planner: extends workflow
LowCodeLLM/src/lowCodeLLM.py - Workflow Coordinator → SOP Executor: executes workflow
LowCodeLLM/src/lowCodeLLM.py - Workflow Planner → OpenAI Wrapper: requests completion
LowCodeLLM/src/planningLLM.py - SOP Executor → OpenAI Wrapper: requests completion
LowCodeLLM/src/executingLLM.py - OpenAI Wrapper → OpenAI API: calls chat API
LowCodeLLM/src/openAIWrapper.py