# rsrini7/rag-hybrid-inference-pipeline architecture

> Architecture diagram of the GitHub repository rsrini7/rag-hybrid-inference-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/rsrini7/rag-hybrid-inference-pipeline
- Repository: https://github.com/rsrini7/rag-hybrid-inference-pipeline

## Overview

##### Project Type and Purpose
- This is a hybrid Retrieval-Augmented Generation (RAG) application written in Python.
- It provides three “front ends” for user interaction:
- A Streamlit web UI (`app.py`)
- A CLI (`cli.py`)
- A RESTful API via LitServe/FastAPI (`api.py` + `api_client.py`)
- Under the covers, it implements a hybrid retrieval strategy (BM25 keyword + dense embedding) plus cross-encoder reranking, then conditions an LLM to generate answers.

##### File Structure
- Top level:
- `api.py`, `api_client.py` – API server and client
- `app.py` – Streamlit UI
- `cli.py` – Command‐line interface
- `src/dspy_rag_app/` – core pipeline implementation:
– `config.py` – environment‐driven configuration loader
– `data.py` – default document corpus
– `bm25_utils.py`, `nltk_utils.py` – preprocessing & BM25 support
– `retrievers.py` – ChromaRetriever & BM25Retriever classes
– `rag_pipeline.py` – RAGHybridFusedRerank pipeline module
– `utils.py` – helpers for indexing, loading models, wiring components
- Additional: `.streamlit/config.toml`, `.env.example`, `pyproject.toml` for build/deps.

##### Extract Architectural Insights from README
- The README outlines a six-step flow:
1) Query input (from UI, CLI or API)
2) Hybrid retrieval (vector via SentenceTransformer+ChromaDB, keyword via BM25)
3) Fusion & deduplication
4) Cross-Encoder reranking
5) LLM generation (via OpenRouter)
6) Answer output

##### Main Components
a. Clients / Entry Points (Frontend Layer)
- Streamlit Web App (`app.py`)
- CLI Tool (`cli.py`)
- REST API (LitServe/FastAPI) (`api.py`)
b. Configuration & Environment
- `.env`/.env.example → loaded by `config.py`
c. Core Pipeline (Backend Layer)
1. Preprocessing & Indexing
– NLTK utils (tokenization, stopwords)
– BM25 index builder (`bm25_utils.py`)
– ChromaDB vector store client (`chromadb.PersistentClient`)
2. Retrievers
– Vector Retriever (ChromaRetriever + SentenceTransformer)
– Keyword Retriever (BM25Retriever + rank_bm25)
3. Fusion & Deduplication (part of `RAGHybridFusedRerank`)
4. Reranker (CrossEncoder)
5. LLM Generator (`dspy.LM` via OpenRouter)
d. Data Stores & External Services
- ChromaDB (three separate persistent paths: CLI, Streamlit, API)
- OpenRouter LLM endpoint
- NLTK data downloads (punkt, stopwords)

##### Component Relationships & Data Flow
- From each client → Core Pipeline entry point
- Pipeline: Query → [Vector Retriever + BM25 Retriever] → Fusion → Reranker → LLM → Response
- Pipeline modules depend on `config.py` for parameters
- Indexing modules depend on `data.py` or user-supplied documents
- That the API and client communicate via HTTP POST `/predict`.
- That both CLI and Streamlit can trigger indexing flows (Chroma + BM25).

##### Architectural Patterns & Principles
- Layered/multi-tier architecture: Frontend clients, service layer, model layer, data layer
- Modular design: separate retriever modules, pipeline abstraction (`RAGHybridFusedRerank`), utility functions
- Environment-driven configuration (12-factor principle)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Clients Layer
    subgraph "Clients" 
        WS["Streamlit Web UI"]:::client
        CLI["CLI Tool"]:::client
        APIClient["REST API Client"]:::client
        API["REST API Server"]:::client
    end

    %% Configuration Layer
    subgraph "Configuration" 
        Env[".env/.env.example"]:::config
        StreamConf[".streamlit/config.toml"]:::config
        Build["pyproject.toml"]:::config
        ConfigLoader["Configuration Loader"]:::config
    end

    %% Core Pipeline Layer
    subgraph "Backend Pipeline"
        DataSource["Default Document Corpus"]:::util
        BM25Utils["BM25 Preprocessing & Indexing Utilities"]:::bm25
        NLTKUtils["NLTK Preprocessing Utilities"]:::bm25
        Utils["General Helpers"]:::util
        Retrievers["Retriever Implementations"]:::bm25
        Fusion["Fusion & Deduplication"]:::fusion
        Reranker["Cross-Encoder Reranker"]:::fusion
        Pipeline["Hybrid RAG Pipeline"]:::fusion
        LLM["LLM Generator"]:::llm
    end

    %% Data Stores & External Services
    subgraph "Data Stores & Services"
        ChromaCLI[(ChromaDB CLI)]:::db
        ChromaWeb[(ChromaDB Web UI)]:::db
        ChromaAPI[(ChromaDB API)]:::db
        OpenRouter[/OpenRouter LLM API/]:::external
        NLTKRes[/NLTK Resources/]:::external
    end

    %% Connections - Client to Pipeline
    WS -->|"query text"| Pipeline
    CLI -->|"query text"| Pipeline
    API -->|"HTTP POST /predict"| Pipeline
    APIClient -- bidirectional --> API

    %% Configuration dependencies
    Env --> ConfigLoader
    StreamConf --> ConfigLoader
    Build --> ConfigLoader

    ConfigLoader -->|configs| DataSource
    ConfigLoader -->|configs| BM25Utils
    ConfigLoader -->|configs| NLTKUtils
    ConfigLoader -->|configs| Utils
    ConfigLoader -->|configs| Retrievers
    ConfigLoader -->|configs| Fusion
    ConfigLoader -->|configs| Reranker
    ConfigLoader -->|configs| LLM

    %% Preprocessing & Indexing
    DataSource --> BM25Utils
    DataSource --> NLTKUtils
    BM25Utils --> ChromaCLI
    NLTKUtils --> NLTKRes
    Utils --> ChromaWeb
    Utils --> ChromaAPI

    %% Retrieval
    BM25Utils --> Retrievers
    ChromaCLI --> Retrievers
    ChromaWeb --> Retrievers
    ChromaAPI --> Retrievers

    %% Pipeline flow
    Pipeline -->|"keyword candidates"| Retrievers
    Pipeline -->|"vector candidates"| Retrievers
    Retrievers -->|"candidate docs"| Fusion
    Fusion -->|"fused docs"| Reranker
    Reranker -->|"reranked docs"| LLM
    LLM -->|"generated answer"| WS
    LLM -->|"generated answer"| CLI
    LLM -->|"generated answer"| API

    %% External LLM and resources
    LLM --> OpenRouter
    NLTKUtils --> NLTKRes

    %% Click events
    click WS "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/app.py"
    click CLI "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/cli.py"
    click API "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/api.py"
    click APIClient "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/api_client.py"
    click Env "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/.env.example"
    click StreamConf "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/.streamlit/config.toml"
    click Build "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/pyproject.toml"
    click ConfigLoader "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/src/dspy_rag_app/config.py"
    click DataSource "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/src/dspy_rag_app/data.py"
    click BM25Utils "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/src/dspy_rag_app/bm25_utils.py"
    click NLTKUtils "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/src/dspy_rag_app/nltk_utils.py"
    click Retrievers "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/src/dspy_rag_app/retrievers.py"
    click Pipeline "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/src/dspy_rag_app/rag_pipeline.py"
    click Utils "https://github.com/rsrini7/rag-hybrid-inference-pipeline/blob/main/src/dspy_rag_app/utils.py"

    %% Styles
    classDef client fill:#E5D3FF,stroke:#7D3C98,color:#000
    classDef config fill:#D3D3D3,stroke:#888,color:#000
    classDef util fill:#CCCCCC,stroke:#666,color:#000
    classDef bm25 fill:#AED6F1,stroke:#2980B9,color:#000
    classDef fusion fill:#ABEBC6,stroke:#27AE60,color:#000
    classDef llm fill:#F9E79F,stroke:#D68910,color:#000
    classDef db fill:#F5B7B1,stroke:#C0392B,color:#000
    classDef external fill:#FADBD8,stroke:#C0392B,color:#000
```
