# milkiller/visual-troubleshooting-agent architecture

> Architecture diagram of the GitHub repository milkiller/visual-troubleshooting-agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/milkiller/visual-troubleshooting-agent
- Repository: https://github.com/milkiller/visual-troubleshooting-agent

## Overview

Visual Troubleshooting Agent accepts text errors or screenshots, turns them into a troubleshooting task, and iteratively asks a language model to choose actions. Its main loop dispatches shell, file-reading, and file-editing tools, feeds results back for replanning, and runs verification after a successful edit when a verification command is supplied. Screenshot analysis uses a separate vision service. README capabilities include environment troubleshooting and source localization; source excerpts show these are mediated through shell commands. The architecture below is based on the README and supplied source excerpts; unsampled behavior is not inferred.

Main entry points: [`main.py`](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/main.py) (Agent loop), [`tools/vision/qwen_vision.py`](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/vision/qwen_vision.py) (Screenshot analysis), [`tools/shell_tool.py`](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/shell_tool.py) (Shell tool), [`tools/verify_tool.py`](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/verify_tool.py) (Verification tool)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_agent["Agent workflow"]
  node_main["Agent loop<br/>[main.py]"]
  node_skill["Troubleshooting skill<br/>[SKILL.md]"]
  node_stepfun["StepFun model"]
end

subgraph group_understanding["Problem understanding"]
  node_vision["Screenshot analysis<br/>[qwen_vision.py]"]
  node_qwen["Qwen Vision API"]
end

subgraph group_tools["Troubleshooting tools"]
  node_shell["Shell tool<br/>[shell_tool.py]"]
  node_reader["File reader<br/>[file_read_tool.py]"]
  node_editor["File editor<br/>[file_edit_tool.py]"]
  node_verifier["Verification tool<br/>[verify_tool.py]"]
end

subgraph group_state["Project and execution"]
  node_project["Project files"]
  node_runtime["Command environment"]
end

node_user(("User"))

node_user -->|"provides problem"| node_main
node_main -->|"analyzes screenshot"| node_vision
node_vision -->|"sends image"| node_qwen
node_qwen -->|"returns analysis"| node_vision
node_vision -->|"returns diagnosis"| node_main
node_main -->|"sends task and history"| node_stepfun
node_stepfun -->|"returns action"| node_main
node_main -->|"loads instructions"| node_skill
node_main -->|"dispatches command"| node_shell
node_shell -->|"executes command"| node_runtime
node_runtime -->|"returns output"| node_shell
node_shell -->|"returns result"| node_main
node_main -->|"dispatches read"| node_reader
node_reader -->|"reads source"| node_project
node_reader -->|"returns contents"| node_main
node_main -->|"dispatches edit"| node_editor
node_editor -->|"writes source"| node_project
node_editor -->|"returns edit result"| node_main
node_main -->|"dispatches verification"| node_verifier
node_verifier -->|"executes check"| node_runtime
node_runtime -->|"returns output"| node_verifier
node_verifier -->|"returns result"| node_main
node_main -->|"reports completion"| node_user

click node_main "https://github.com/milkiller/visual-troubleshooting-agent/blob/main/main.py"
click node_skill "https://github.com/milkiller/visual-troubleshooting-agent/blob/main/skills/troubleshooting/SKILL.md"
click node_vision "https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/vision/qwen_vision.py"
click node_shell "https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/shell_tool.py"
click node_reader "https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/file_read_tool.py"
click node_editor "https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/file_edit_tool.py"
click node_verifier "https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/verify_tool.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_skill,node_stepfun,node_user toneBlue
class node_vision,node_qwen toneAmber
class node_shell,node_reader,node_editor,node_verifier toneMint
class node_project,node_runtime toneRose
```

## Components

### Agent workflow

- [Agent loop](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/main.py): `main.py`
- [Troubleshooting skill](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/skills/troubleshooting/SKILL.md): `skills/troubleshooting/SKILL.md`
- **StepFun model**

### Problem understanding

- [Screenshot analysis](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/vision/qwen_vision.py): `tools/vision/qwen_vision.py`
- **Qwen Vision API**

### Troubleshooting tools

- [Shell tool](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/shell_tool.py): `tools/shell_tool.py`
- [File reader](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/file_read_tool.py): `tools/file_read_tool.py`
- [File editor](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/file_edit_tool.py): `tools/file_edit_tool.py`
- [Verification tool](https://github.com/milkiller/visual-troubleshooting-agent/blob/main/tools/verify_tool.py): `tools/verify_tool.py`

### Project and execution

- **Project files**
- **Command environment**

### Other components

- **User**

## Connections

- User → Agent loop: provides problem
- Agent loop → Screenshot analysis: analyzes screenshot
- Screenshot analysis → Qwen Vision API: sends image
- Qwen Vision API → Screenshot analysis: returns analysis
- Screenshot analysis → Agent loop: returns diagnosis
- Agent loop → StepFun model: sends task and history
- StepFun model → Agent loop: returns action
- Agent loop → Troubleshooting skill: loads instructions
- Agent loop → Shell tool: dispatches command
- Shell tool → Command environment: executes command
- Command environment → Shell tool: returns output
- Shell tool → Agent loop: returns result
- Agent loop → File reader: dispatches read
- File reader → Project files: reads source
- File reader → Agent loop: returns contents
- Agent loop → File editor: dispatches edit
- File editor → Project files: writes source
- File editor → Agent loop: returns edit result
- Agent loop → Verification tool: dispatches verification
- Verification tool → Command environment: executes check
- Command environment → Verification tool: returns output
- Verification tool → Agent loop: returns result
- Agent loop → User: reports completion