How dleungdl/vision-for-llm works
Vision lets an AI agent analyze images through an OpenAI-compatible vision API, either via a skill script or MCP tools. The main workflow accepts an image or PDF, selects the appropriate analysis or extraction path, and returns text to the agent. Image handling, model fallback, and model discovery are implemented in the shared script; the MCP server adds tool dispatch and PDF-specific workflows. The README documents additional skill modes and PDF capabilities; unsampled skill instructions and PDF library internals are represented at directory or external-service level.
Main entry points: vision.js Vision CLI · server.js PDF input · SKILL.md Skill instructions
Components 20
Agent skill
- Skill instructions
skills/vision/SKILL.md - Vision CLI
skills/vision/scripts/vision.js
MCP interface
- MCP server
mcp/server.js - Image tool
mcp/server.js
Vision processing
- Analysis modes
skills/vision/scripts/vision.js - Image preparation
skills/vision/scripts/vision.js - Model fallback
skills/vision/scripts/vision.js - Model discovery
skills/vision/scripts/vision.js - API requests
skills/vision/scripts/vision.js
PDF processing
- PDF input
mcp/server.js - PDF classification
mcp/server.js - PDF text extraction
mcp/server.js - PDF to Markdown
mcp/server.js - Page rendering
mcp/server.js - Page descriptions
mcp/server.js
Other components
- AI agent
- Vision API
- MCP client
- PDF inspector
- pdftoppm renderer
Connections 27
- AI agent → Skill instructions: uses skill
README.md - Skill instructions → Vision CLI: invokes script
- AI agent → Vision CLI: runs analysis
README.md - Vision CLI → Analysis modes: selects mode
skills/vision/scripts/vision.js - Vision CLI → Image preparation: resolves images
skills/vision/scripts/vision.js - Vision CLI → Model fallback: requests analysis
skills/vision/scripts/vision.js - Vision CLI → Model discovery: lists or checks
skills/vision/scripts/vision.js - Model fallback → API requests: calls model
skills/vision/scripts/vision.js - Model discovery → API requests: fetches models
skills/vision/scripts/vision.js - API requests → Vision API: sends API requests
skills/vision/scripts/vision.js - MCP client → MCP server: calls tools
README.md - MCP server → Image tool: registers tool
mcp/server.js - Image tool → Model fallback: uses shared logic
mcp/server.js - MCP server → PDF input: loads PDF
mcp/server.js - PDF input → PDF classification: classifies bytes
mcp/server.js - PDF input → PDF text extraction: extracts text
mcp/server.js - PDF input → PDF to Markdown: converts PDF
mcp/server.js - PDF input → Page rendering: renders pages
mcp/server.js - PDF classification → PDF inspector: classifies PDF
mcp/server.js - PDF text extraction → PDF inspector: extracts layer
mcp/server.js - PDF to Markdown → PDF inspector: processes PDF
mcp/server.js - Page rendering → pdftoppm renderer: renders page
mcp/server.js - MCP server → Page descriptions: registers tool
mcp/server.js - Page descriptions → PDF input: loads PDF
mcp/server.js - Page descriptions → PDF classification: selects pages
mcp/server.js - Page descriptions → Page rendering: renders targets
mcp/server.js - Page descriptions → Image tool: describes pages
mcp/server.js