How superlinear-ai/raglite works
Main Components
• Core RAG Engine: At the heart of the project lies the orchestration of retrieval and generation flows. Modules such as _rag.py, _embed.py, _search.py, _split_chunks.py, and _split_sentences.py make up the core pipeline that performs document ingestion, search (hybrid, keyword, vector), reranking, and generation.
• Database & Storage: The components in _database.py manage interactions with either PostgreSQL or SQLite. This part is responsible for storing documents, indexes, and evaluations.
• LLM & Embedding Integrations: With support for multiple LLM providers through integration with LiteLLM and specific modules like _lazy_llama.py and _litellm.py, as well as embedding utilities, these layers manage the communication with external language models and embedding tools.
• Configuration & CLI: Module _config.py centralizes configurable parameters (database URL, LLM identifiers, embedder details) while _cli.py exposes command-line interface capabilities for both running pipelines and server options.
• External Interfaces and Extensions:
• Chat & Frontend Integration: Modules like _chainlit.py support a chainlit powered ChatGPT-like front end.
• Model Context Protocol (MCP) Server: Implemented via _mcp.py, this enables connections for MCP clients like Claude desktop.
• Utility and Conversion Tools: Modules such as _markdown.py (for PDF-to-Markdown conversion) facilitate document preprocessing.
• Query Adapter: The _query_adapter.py encapsulates the optimized query adapter functionality, solving a closed-form problem to adjust retrieval requests.
• Testing and Validation: The tests directory contains scripts that validate the functionality of these components.
Mapping Relationships and Interactions
• User interaction (CLI, Chat frontend, or MCP calls) will trigger the configuration loader and then push commands into the RAG pipeline.
• The RAG pipeline begins with document insertion and conversion (via _extract.py, _split_chunks.py, etc.), where documents are ingested, processed, and stored into the database.
• For retrieval, components perform different types of search (hybrid, keyword, vector) using _search.py combined with reranking logic (with support from external packages).
• Once relevant chunks are retrieved, the LLM integration layer (including _litellm.py and _lazy_llama.py) generates responses, possibly after applying any query adaptations computed in _query_adapter.py.
• Optional modules (chainlit and MCP) demonstrate how the toolkit can serve responses via a frontend or expose tool endpoints for LLMs.
Architectural Patterns and Design Principles
• Modularity and Layering: Each functional responsibility is encapsulated in its own module. This separation supports extensibility (e.g., adding new LLM providers or reranker implementations).
• Configurable Pipeline: The design allows for both “adaptive” and “programmable” RAG flows, making it flexible for different use cases.
• Extensibility and Integration: By providing options such as Chainlit integration for ChatGPT-like frontends and an MCP server, the toolkit is built to be easily integrated into larger stacks or to serve as a building block for more complex systems.
Key Technologies and Libraries to Note
• Python’s standard ecosystem for packaging and dependency management with pyproject.toml.
• Docker, Dev Containers, and GitHub workflows for development, testing, and deployment.
• External tools like LiteLLM, rerankers, PDF conversion libraries (pdftext, pypdfium2), and the Chainlit frontend.