How dhruva-31/multimodel-rag works
This repository is a multimodal RAG application: users upload files to build a searchable knowledge base, then ask questions and receive answers grounded in retrieved passages. Ingestion converts supported formats into chunks, embeddings, and dense and BM25 indexes; querying applies input checks, hybrid retrieval, optional reranking, context construction, generation, and citation validation. The README documents the full workflow; connections among modules are included where the source index or excerpts support them.
Main entry points: query_pipeline.py Query Pipeline · ingestion_pipeline.py Ingestion Pipeline · ingestion Format Loaders · hybrid_retriever.py Hybrid Retriever
Components 20
Document Ingestion
- Ingestion Pipeline
pipelines/ingestion_pipeline.py - Loader Factory
ingestion/loader_factory.py - Format Loaders
ingestion - Document Chunker
preprocessing/chunker.py
Retrieval and Ranking
- Dense Embedder
embeddings/dense_embeddings.py - BM25 Index
sparse/bm25_index.py - Hybrid Retriever
retrieval/hybrid_retriever.py - Dense Retriever
retrieval/dense_retriever.py - Cross-Encoder Reranker
retrieval/reranker.py
Grounded Answering
- Query Pipeline
pipelines/query_pipeline.py - Input Guardrail
guardrails/input_guardrail.py - Context Builder
generation/context_builder.py - LLM Generator
generation/generator.py - Output Guardrail
guardrails/output_guardrail.py
Data and Storage
- Document Model
models/document.py - Metadata Model
models/metadata.py - Chroma Store
vectorstore/chroma_store.py - Persistent Vector Store
Other components
- User
- Streamlit UI
streamlit_app.py
Connections 23
- User → Streamlit UI: uploads and asks
README.md - Streamlit UI → Ingestion Pipeline: builds knowledge base
streamlit_app.py - Streamlit UI → Query Pipeline: submits questions
streamlit_app.py - Ingestion Pipeline → Loader Factory: selects loader
pipelines/ingestion_pipeline.py - Loader Factory → Format Loaders: dispatches formats
ingestion/loader_factory.py - Format Loaders → Document Model: creates documents
ingestion/docx_loader.py - Format Loaders → Metadata Model: creates metadata
ingestion/docx_loader.py - Ingestion Pipeline → Document Chunker: chunks documents
pipelines/ingestion_pipeline.py - Ingestion Pipeline → Dense Embedder: embeds chunks
pipelines/ingestion_pipeline.py - Ingestion Pipeline → Chroma Store: stores vectors
pipelines/ingestion_pipeline.py - Ingestion Pipeline → BM25 Index: builds index
pipelines/ingestion_pipeline.py - Chroma Store → Persistent Vector Store: persists vectors
vectorstore/chroma_store.py - Query Pipeline → Input Guardrail: checks query
pipelines/query_pipeline.py - Query Pipeline → Hybrid Retriever: retrieves candidates
pipelines/query_pipeline.py - Hybrid Retriever → Dense Retriever: runs dense search
retrieval/hybrid_retriever.py - Hybrid Retriever → BM25 Index: runs sparse search
retrieval/hybrid_retriever.py - Dense Retriever → Dense Embedder: embeds query
retrieval/dense_retriever.py - Dense Retriever → Chroma Store: searches vectors
retrieval/dense_retriever.py - Query Pipeline → Cross-Encoder Reranker: reranks candidates
pipelines/query_pipeline.py - Query Pipeline → Context Builder: builds context
pipelines/query_pipeline.py - Query Pipeline → LLM Generator: generates answer
pipelines/query_pipeline.py - Query Pipeline → Output Guardrail: checks citations
pipelines/query_pipeline.py - Output Guardrail → User: returns grounded answer
README.md