How trehansalil/pageindex works
PageIndex MCP is a self-hosted, vectorless RAG service: clients upload documents, an asynchronous worker extracts and quality-checks content into hierarchical or flat indexes, and MCP tools retrieve relevant passages through LLM-guided tree search. Its main runtime layers are the server/API, worker and converter pipeline, search and table handling, and durable storage. The README documents the ingestion routes and shared infrastructure; sampled source confirms parts of search, conversion, and server composition, so some internal links are omitted where ownership is not evidenced.
Main entry points: rag.py Tree RAG search · job.py Async worker · server.py MCP server · pipeline.py Conversion pipeline
Components 21
Server and API
- MCP server
src/pageindex_mcp/server.py - Document tools
src/pageindex_mcp/tools/documents.py - Upload API
src/pageindex_mcp/upload_app.py
Ingestion pipeline
- Async worker
src/pageindex_mcp/worker/job.py - Job subprocess
src/pageindex_mcp/worker/subprocess_mgr.py - Page index client
src/pageindex_mcp/client/indexer.py - Conversion pipeline
src/pageindex_mcp/converters/pipeline.py - Docling service
services/docling-service/app.py - Tree quality gate
src/pageindex_mcp/helpers/tree_validation.py - OCR and recovery
src/pageindex_mcp/converters/pictures.py
Query and tables
- Table anchoring
src/pageindex_mcp/tables/anchor.py - Table search view
src/pageindex_mcp/tables/search_view.py - Tree RAG search
src/pageindex_mcp/helpers/rag.py
Persistence and registry
- MinIO objects
src/pageindex_mcp/storage/minio_ops.py - Redis cache and queue
src/pageindex_mcp/cache.py - Document registry
src/pageindex_mcp/registry/queries.py
External services
- LLM provider
- PostgreSQL
- Prometheus
Other components
- MCP client
- Upload client
Connections 25
- MCP client → MCP server: calls MCP
- Upload client → Upload API: submits files
- MCP server → Document tools: registers tools
- Document tools → Tree RAG search: queries documents
- MCP server → Upload API: mounts API
- Upload API → MinIO objects: stages files
- Upload API → Redis cache and queue: enqueues jobs
- Async worker → Redis cache and queue: consumes jobs
- Async worker → Job subprocess: runs isolated job
- Job subprocess → Page index client: starts indexing
- Page index client → Conversion pipeline: converts documents
- Conversion pipeline → Docling service: offloads conversion
- Conversion pipeline → Tree quality gate: validates tree
- Conversion pipeline → OCR and recovery: recovers content
- Tree quality gate → Table anchoring: anchors tables
- Async worker → MinIO objects: persists results
- Async worker → Document registry: writes catalog
- Tree RAG search → Redis cache and queue: reads cache
- Tree RAG search → MinIO objects: loads trees
- Tree RAG search → Document registry: narrows documents
- Tree RAG search → Table search view: renders search view
- Tree RAG search → LLM provider: requests reasoning
- Page index client → LLM provider: builds index tree
- Document registry → PostgreSQL: stores catalog
- MCP server → Prometheus: exposes metrics