How milkiller/visual-troubleshooting-agent works
Visual Troubleshooting Agent accepts text errors or screenshots, turns them into a troubleshooting task, and iteratively asks a language model to choose actions. Its main loop dispatches shell, file-reading, and file-editing tools, feeds results back for replanning, and runs verification after a successful edit when a verification command is supplied. Screenshot analysis uses a separate vision service. README capabilities include environment troubleshooting and source localization; source excerpts show these are mediated through shell commands. The architecture below is based on the README and supplied source excerpts; unsampled behavior is not inferred.
Main entry points: main.py Agent loop · qwen_vision.py Screenshot analysis · shell_tool.py Shell tool · verify_tool.py Verification tool
Components 12
Agent workflow
- Agent loop
main.py - Troubleshooting skill
skills/troubleshooting/SKILL.md - StepFun model
Problem understanding
- Screenshot analysis
tools/vision/qwen_vision.py - Qwen Vision API
Troubleshooting tools
- Shell tool
tools/shell_tool.py - File reader
tools/file_read_tool.py - File editor
tools/file_edit_tool.py - Verification tool
tools/verify_tool.py
Project and execution
- Project files
- Command environment
Other components
- User
Connections 23
- User → Agent loop: provides problem
- Agent loop → Screenshot analysis: analyzes screenshot
- Screenshot analysis → Qwen Vision API: sends image
- Qwen Vision API → Screenshot analysis: returns analysis
- Screenshot analysis → Agent loop: returns diagnosis
- Agent loop → StepFun model: sends task and history
- StepFun model → Agent loop: returns action
- Agent loop → Troubleshooting skill: loads instructions
- Agent loop → Shell tool: dispatches command
- Shell tool → Command environment: executes command
- Command environment → Shell tool: returns output
- Shell tool → Agent loop: returns result
- Agent loop → File reader: dispatches read
- File reader → Project files: reads source
- File reader → Agent loop: returns contents
- Agent loop → File editor: dispatches edit
- File editor → Project files: writes source
- File editor → Agent loop: returns edit result
- Agent loop → Verification tool: dispatches verification
- Verification tool → Command environment: executes check
- Command environment → Verification tool: returns output
- Verification tool → Agent loop: returns result
- Agent loop → User: reports completion