How adithya-s-k/omniparse works
Project type and purpose
• OmniParse is primarily a backend-focused, monolithic Python service that ingests arbitrary unstructured data (documents, images, audio/video, web pages) and converts it into structured, GenAI-ready outputs via deep-learning models.
• It also provides a Python client SDK, a CLI for model download and server startup, and an ("interactive") Gradio UI.
File structure
• Top-level artifacts:
• server.py – main FastAPI (or similar) HTTP server entrypoint
• download.py – CLI to preload models
• Dockerfile – containerization
• omniparse/ – core modules by domain (chunking, documents, extraction, image, media, web, models, utils)
• python-sdk/ – client library that wraps HTTP calls to the server
• docs/ – user/developer documentation & examples
• examples/ – Colab notebook for end-to-end demo
• The omniparse package follows a “router + handler + model” pattern: each subfolder (e.g., omniparse/documents/router.py, omniparse/image/router.py, omniparse/media/router.py, omniparse/web/router.py) registers endpoints, delegates to domain-specific processing modules, which in turn invoke preloaded DL models.
README for additional insights
• CLI flags --documents, --media, --web control which modules (and therefore which heavy models) are loaded into GPU memory at startup.
• Exposed REST endpoints under /parse_document, /parse_media, /parse_image, /parse_website correspond one-to-one to the routers.
• No persistent database is used – data flows in (file or URL), is processed in memory via models, and results are returned immediately.
Main components and interactions
a. Clients:
• Python SDK (python-sdk/)
• REST/HTTP clients (curl, Postman)
• CLI (download.py, server.py flags)
• Gradio UI (embedded or launched by server)
b. API Server Layer (server.py + FastAPI/Starlette):
• API Gateway accepts HTTP requests → dispatches to domain routers
c. Domain Routers & Handlers:
• Document Router
• Image Router
• Media Router
• Web Router
• (Optional) Sheets Router
d. Processing Modules:
• Chunking (omniparse/chunking/)
• Extraction (omniparse/extraction/)
• OCR & Layout (Surya OCR series, Florence-2)
• Whisper for audio/video transcription
• Web crawler (Selenium-based)
e. Model Manager & GPU Resource:
• Central loader orchestrates model fetch (via download.py) and maintains them in GPU memory (with caching)
f. File System & External Services:
• Local disk for temporary file storage
• Selenium / headless browser for web crawling
g. Output Layer:
• Structured Markdown / JSON returned over HTTP
• (Optionally) streamed back to Gradio UI or Python SDK
Architectural patterns & design principles
• Modular router pattern: clearly separated REST endpoints match to domain routers, simplifying adding new data-type support.
• Plugin-style model loading: CLI flags selectively load heavy models, reducing startup footprint.
• Separation of concerns: API layer vs. processing layer vs. model management.
• Stateless request handling: no database–the service is horizontally scalable (multiple container instances behind a load balancer).
Elements
• Clear labels: “API Gateway (FastAPI)”, “Document Router”, “Image Router”, “Model Manager”, “GPU Memory”, “Python SDK (client)”, “CLI”
• Client → API Gateway → Domain Router → Processing Module → Model Manager → GPU
• Model Manager → Processing Module → Router → Response → Client
• Blue rectangles for server internals (modules, routers)
• Orange for external clients (SDK, CLI, UI)
• Grey for external services (Selenium, local FS)