How s-emanuilov/litepali works
Project Type and Purpose
• LitePali is an open-source Python library (packaged via pyproject.toml) for lightweight, image-based document retrieval using Vision-Language Models (specifically the ColPali engine). It is optimized for cloud deployment and batch processing of images, rather than PDF parsing.
File Structure Analysis
• Top-level files:
• README.md, LICENSE, pyproject.toml, .gitignore, .pre-commit-config.yaml – standard packaging, CI, and documentation.
• _config.yml, CNAME – GitHub Pages / docs site configuration.
• examples/: Jupyter notebooks showing usage patterns (e.g., converting PDFs to images externally, RAG example).
• src/litepali/: the core library code. model.py contains the LitePali class and ImageFile data structure.
README Insights
• Lists core features: direct image processing, batch and deterministic processing, minimal dependencies, optimized for cloud.
• Describes the underlying model: late interaction mechanism, multi‐vector representations, visual & textual understanding, efficient indexing.
• Usage section outlines the library’s public API: add images, process, search, save/load index.
Main Components
a. Client Application / User Code
• The Python script or notebook that instantiates LitePali, adds ImageFile objects, invokes process() and search().
b. API Layer (LitePali Class)
• Methods: add(ImageFile), process(), search(query), save_index(path), load_index(path).
c. Image Preprocessing Module
• Loads and optionally normalizes images, attaches metadata (document_id, page_id).
d. Feature Extraction (VLM Engine)
• Integration with colpali-engine (vision‐language model) using PyTorch to compute multi‐vector embeddings for each image and textual query.
e. Indexing Engine
• Builds and stores a vector index (in-memory or file‐based) for fast similarity search.
f. Retrieval Module
• Executes k-NN search on the index, returns top-k results with scores and metadata.
g. Persistence / Storage
• Serializes the index (and optionally image metadata) to disk via save_index/load_index.
h. External Dependencies
• PyTorch (for model), colpali-engine, any serialization libraries (e.g., pickle, JSON), cloud storage if used.
Relationships & Data Flow
• User code → (calls) LitePali API → (passes) ImageFile → Preprocessing → Feature Extraction → Indexing → Stored Index
• User search query → (calls) LitePali.search() → Query embedding (Feature Extraction) → Retrieval from Index → Results back to User
• save_index/load_index interact with Persistence block.
Architectural Patterns & Principles
• Modular design: clear separation between API, preprocessing, embedding, indexing, and retrieval.
• Plug-and-play model engine: the colpali-engine interface can be swapped/updated.
• Stateless processing steps, except for the in-memory index.
• Batch processing support for throughput and deterministic ordering.