# dleungdl/vision-for-llm architecture

> Architecture diagram of the GitHub repository dleungdl/vision-for-llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dleungdl/vision-for-llm
- Repository: https://github.com/dleungdl/vision-for-llm

## Overview

Vision lets an AI agent analyze images through an OpenAI-compatible vision API, either via a skill script or MCP tools. The main workflow accepts an image or PDF, selects the appropriate analysis or extraction path, and returns text to the agent. Image handling, model fallback, and model discovery are implemented in the shared script; the MCP server adds tool dispatch and PDF-specific workflows. The README documents additional skill modes and PDF capabilities; unsampled skill instructions and PDF library internals are represented at directory or external-service level.

Main entry points: [`skills/vision/scripts/vision.js`](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js) (Vision CLI), [`mcp/server.js`](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js) (PDF input), [`skills/vision/SKILL.md`](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/SKILL.md) (Skill instructions)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_skill["Agent skill"]
  node_skillguide["Skill instructions<br/>[SKILL.md]"]
  node_cli["Vision CLI<br/>[vision.js]"]
end

subgraph group_mcp["MCP interface"]
  node_mcpserver["MCP server<br/>[server.js]"]
  node_imagetool["Image tool<br/>[server.js]"]
end

subgraph group_vision["Vision processing"]
  node_modes["Analysis modes<br/>[vision.js]"]
  node_imageprep["Image preparation<br/>[vision.js]"]
  node_fallback["Model fallback<br/>[vision.js]"]
  node_modeltools["Model discovery<br/>[vision.js]"]
  node_transport["API requests<br/>[vision.js]"]
end

subgraph group_pdf["PDF processing"]
  node_pdfinput["PDF input<br/>[server.js]"]
  node_pdfclassify["PDF classification<br/>[server.js]"]
  node_pdftext["PDF text extraction<br/>[server.js]"]
  node_pdfmarkdown["PDF to Markdown<br/>[server.js]"]
  node_pdfrender["Page rendering<br/>[server.js]"]
  node_pdfdescribe["Page descriptions<br/>[server.js]"]
end

node_agent(("AI agent"))
node_visionapi{{"Vision API"}}
node_mcpclient(("MCP client"))
node_pdfinspector{{"PDF inspector"}}
node_pdftoppm{{"pdftoppm renderer"}}

node_agent -->|"uses skill"| node_skillguide
node_skillguide -.->|"invokes script"| node_cli
node_agent -->|"runs analysis"| node_cli
node_cli -->|"selects mode"| node_modes
node_cli -->|"resolves images"| node_imageprep
node_cli -->|"requests analysis"| node_fallback
node_cli -->|"lists or checks"| node_modeltools
node_fallback -->|"calls model"| node_transport
node_modeltools -->|"fetches models"| node_transport
node_transport -->|"sends API requests"| node_visionapi
node_mcpclient -->|"calls tools"| node_mcpserver
node_mcpserver -->|"registers tool"| node_imagetool
node_imagetool -->|"uses shared logic"| node_fallback
node_mcpserver -->|"loads PDF"| node_pdfinput
node_pdfinput -->|"classifies bytes"| node_pdfclassify
node_pdfinput -->|"extracts text"| node_pdftext
node_pdfinput -->|"converts PDF"| node_pdfmarkdown
node_pdfinput -->|"renders pages"| node_pdfrender
node_pdfclassify -->|"classifies PDF"| node_pdfinspector
node_pdftext -->|"extracts layer"| node_pdfinspector
node_pdfmarkdown -->|"processes PDF"| node_pdfinspector
node_pdfrender -->|"renders page"| node_pdftoppm
node_mcpserver -->|"registers tool"| node_pdfdescribe
node_pdfdescribe -->|"loads PDF"| node_pdfinput
node_pdfdescribe -->|"selects pages"| node_pdfclassify
node_pdfdescribe -->|"renders targets"| node_pdfrender
node_pdfdescribe -->|"describes pages"| node_imagetool

click node_skillguide "https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/SKILL.md"
click node_cli "https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js"
click node_modes "https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js"
click node_imageprep "https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js"
click node_fallback "https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js"
click node_modeltools "https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js"
click node_transport "https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js"
click node_mcpserver "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"
click node_imagetool "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"
click node_pdfinput "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"
click node_pdfclassify "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"
click node_pdftext "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"
click node_pdfmarkdown "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"
click node_pdfrender "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"
click node_pdfdescribe "https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_skillguide,node_cli,node_mcpclient toneBlue
class node_mcpserver,node_imagetool toneAmber
class node_modes,node_imageprep,node_fallback,node_modeltools,node_transport,node_visionapi toneMint
class node_pdfinput,node_pdfclassify,node_pdftext,node_pdfmarkdown,node_pdfrender,node_pdfdescribe toneRose
class node_agent,node_pdfinspector,node_pdftoppm toneIndigo
```

## Components

### Agent skill

- [Skill instructions](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/SKILL.md): `skills/vision/SKILL.md`
- [Vision CLI](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js): `skills/vision/scripts/vision.js`

### MCP interface

- [MCP server](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`
- [Image tool](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`

### Vision processing

- [Analysis modes](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js): `skills/vision/scripts/vision.js`
- [Image preparation](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js): `skills/vision/scripts/vision.js`
- [Model fallback](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js): `skills/vision/scripts/vision.js`
- [Model discovery](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js): `skills/vision/scripts/vision.js`
- [API requests](https://github.com/dleungdl/vision-for-llm/blob/main/skills/vision/scripts/vision.js): `skills/vision/scripts/vision.js`

### PDF processing

- [PDF input](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`
- [PDF classification](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`
- [PDF text extraction](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`
- [PDF to Markdown](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`
- [Page rendering](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`
- [Page descriptions](https://github.com/dleungdl/vision-for-llm/blob/main/mcp/server.js): `mcp/server.js`

### Other components

- **AI agent**
- **Vision API**
- **MCP client**
- **PDF inspector**
- **pdftoppm renderer**

## Connections

- AI agent → Skill instructions: uses skill (evidence: `README.md`)
- Skill instructions → Vision CLI: invokes script
- AI agent → Vision CLI: runs analysis (evidence: `README.md`)
- Vision CLI → Analysis modes: selects mode (evidence: `skills/vision/scripts/vision.js`)
- Vision CLI → Image preparation: resolves images (evidence: `skills/vision/scripts/vision.js`)
- Vision CLI → Model fallback: requests analysis (evidence: `skills/vision/scripts/vision.js`)
- Vision CLI → Model discovery: lists or checks (evidence: `skills/vision/scripts/vision.js`)
- Model fallback → API requests: calls model (evidence: `skills/vision/scripts/vision.js`)
- Model discovery → API requests: fetches models (evidence: `skills/vision/scripts/vision.js`)
- API requests → Vision API: sends API requests (evidence: `skills/vision/scripts/vision.js`)
- MCP client → MCP server: calls tools (evidence: `README.md`)
- MCP server → Image tool: registers tool (evidence: `mcp/server.js`)
- Image tool → Model fallback: uses shared logic (evidence: `mcp/server.js`)
- MCP server → PDF input: loads PDF (evidence: `mcp/server.js`)
- PDF input → PDF classification: classifies bytes (evidence: `mcp/server.js`)
- PDF input → PDF text extraction: extracts text (evidence: `mcp/server.js`)
- PDF input → PDF to Markdown: converts PDF (evidence: `mcp/server.js`)
- PDF input → Page rendering: renders pages (evidence: `mcp/server.js`)
- PDF classification → PDF inspector: classifies PDF (evidence: `mcp/server.js`)
- PDF text extraction → PDF inspector: extracts layer (evidence: `mcp/server.js`)
- PDF to Markdown → PDF inspector: processes PDF (evidence: `mcp/server.js`)
- Page rendering → pdftoppm renderer: renders page (evidence: `mcp/server.js`)
- MCP server → Page descriptions: registers tool (evidence: `mcp/server.js`)
- Page descriptions → PDF input: loads PDF (evidence: `mcp/server.js`)
- Page descriptions → PDF classification: selects pages (evidence: `mcp/server.js`)
- Page descriptions → Page rendering: renders targets (evidence: `mcp/server.js`)
- Page descriptions → Image tool: describes pages (evidence: `mcp/server.js`)
