How rsrini7/rag-hybrid-inference-pipeline works
Project Type and Purpose
• This is a hybrid Retrieval-Augmented Generation (RAG) application written in Python.
• It provides three “front ends” for user interaction:
• A Streamlit web UI (app.py)
• A CLI (cli.py)
• A RESTful API via LitServe/FastAPI (api.py + api_client.py)
• Under the covers, it implements a hybrid retrieval strategy (BM25 keyword + dense embedding) plus cross-encoder reranking, then conditions an LLM to generate answers.
File Structure
• Top level:
• api.py, api_client.py – API server and client
• app.py – Streamlit UI
• cli.py – Command‐line interface
• src/dspy_rag_app/ – core pipeline implementation:
– config.py – environment‐driven configuration loader
– data.py – default document corpus
– bm25_utils.py, nltk_utils.py – preprocessing & BM25 support
– retrievers.py – ChromaRetriever & BM25Retriever classes
– rag_pipeline.py – RAGHybridFusedRerank pipeline module
– utils.py – helpers for indexing, loading models, wiring components
• Additional: .streamlit/config.toml, .env.example, pyproject.toml for build/deps.
Extract Architectural Insights from README
• The README outlines a six-step flow:
1) Query input (from UI, CLI or API)
2) Hybrid retrieval (vector via SentenceTransformer+ChromaDB, keyword via BM25)
3) Fusion & deduplication
4) Cross-Encoder reranking
5) LLM generation (via OpenRouter)
6) Answer output
Main Components
a. Clients / Entry Points (Frontend Layer)
• Streamlit Web App (app.py)
• CLI Tool (cli.py)
• REST API (LitServe/FastAPI) (api.py)
b. Configuration & Environment
• .env/.env.example → loaded by config.py
c. Core Pipeline (Backend Layer)
1. Preprocessing & Indexing
– NLTK utils (tokenization, stopwords)
– BM25 index builder (bm25_utils.py)
– ChromaDB vector store client (chromadb.PersistentClient)
2. Retrievers
– Vector Retriever (ChromaRetriever + SentenceTransformer)
– Keyword Retriever (BM25Retriever + rank_bm25)
3. Fusion & Deduplication (part of RAGHybridFusedRerank)
4. Reranker (CrossEncoder)
5. LLM Generator (dspy.LM via OpenRouter)
d. Data Stores & External Services
• ChromaDB (three separate persistent paths: CLI, Streamlit, API)
• OpenRouter LLM endpoint
• NLTK data downloads (punkt, stopwords)
Component Relationships & Data Flow
• From each client → Core Pipeline entry point
• Pipeline: Query → [Vector Retriever + BM25 Retriever] → Fusion → Reranker → LLM → Response
• Pipeline modules depend on config.py for parameters
• Indexing modules depend on data.py or user-supplied documents
• That the API and client communicate via HTTP POST /predict.
• That both CLI and Streamlit can trigger indexing flows (Chroma + BM25).
Architectural Patterns & Principles
• Layered/multi-tier architecture: Frontend clients, service layer, model layer, data layer
• Modular design: separate retriever modules, pipeline abstraction (RAGHybridFusedRerank), utility functions
• Environment-driven configuration (12-factor principle)