# dhruva-31/multimodel-rag architecture

> Architecture diagram of the GitHub repository dhruva-31/multimodel-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dhruva-31/multimodel-rag
- Repository: https://github.com/dhruva-31/multimodel-rag

## Overview

This repository is a multimodal RAG application: users upload files to build a searchable knowledge base, then ask questions and receive answers grounded in retrieved passages. Ingestion converts supported formats into chunks, embeddings, and dense and BM25 indexes; querying applies input checks, hybrid retrieval, optional reranking, context construction, generation, and citation validation. The README documents the full workflow; connections among modules are included where the source index or excerpts support them.

Main entry points: [`pipelines/query_pipeline.py`](https://github.com/dhruva-31/multimodel-rag/blob/main/pipelines/query_pipeline.py) (Query Pipeline), [`pipelines/ingestion_pipeline.py`](https://github.com/dhruva-31/multimodel-rag/blob/main/pipelines/ingestion_pipeline.py) (Ingestion Pipeline), [`ingestion`](https://github.com/dhruva-31/multimodel-rag/tree/main/ingestion) (Format Loaders), [`retrieval/hybrid_retriever.py`](https://github.com/dhruva-31/multimodel-rag/blob/main/retrieval/hybrid_retriever.py) (Hybrid Retriever)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Document Ingestion"]
  node_ingest_pipeline["Ingestion Pipeline"]
  node_loader_factory["Loader Factory<br/>[loader_factory.py]"]
  node_format_loaders["Format Loaders"]
  node_chunker["Document Chunker<br/>[chunker.py]"]
end

subgraph group_search["Retrieval and Ranking"]
  node_embedder["Dense Embedder"]
  node_bm25_index["BM25 Index<br/>[bm25_index.py]"]
  node_hybrid_retriever["Hybrid Retriever"]
  node_dense_retriever["Dense Retriever<br/>[dense_retriever.py]"]
  node_reranker["Cross-Encoder Reranker<br/>[reranker.py]"]
end

subgraph group_answer["Grounded Answering"]
  node_query_pipeline["Query Pipeline<br/>[query_pipeline.py]"]
  node_input_guardrail["Input Guardrail<br/>[input_guardrail.py]"]
  node_context_builder["Context Builder<br/>[context_builder.py]"]
  node_generator["LLM Generator<br/>[generator.py]"]
  node_output_guardrail["Output Guardrail"]
end

subgraph group_models["Data and Storage"]
  node_document_model["Document Model<br/>[document.py]"]
  node_metadata_model["Metadata Model<br/>[metadata.py]"]
  node_chroma_store["Chroma Store<br/>[chroma_store.py]"]
  node_vector_db[("Persistent Vector Store")]
end

node_user(("User"))
node_ui["Streamlit UI<br/>[streamlit_app.py]"]

node_user -->|"uploads and asks"| node_ui
node_ui -->|"builds knowledge base"| node_ingest_pipeline
node_ui -->|"submits questions"| node_query_pipeline
node_ingest_pipeline -->|"selects loader"| node_loader_factory
node_loader_factory -->|"dispatches formats"| node_format_loaders
node_format_loaders -->|"creates documents"| node_document_model
node_format_loaders -->|"creates metadata"| node_metadata_model
node_ingest_pipeline -->|"chunks documents"| node_chunker
node_ingest_pipeline -->|"embeds chunks"| node_embedder
node_ingest_pipeline -->|"stores vectors"| node_chroma_store
node_ingest_pipeline -->|"builds index"| node_bm25_index
node_chroma_store -->|"persists vectors"| node_vector_db
node_query_pipeline -->|"checks query"| node_input_guardrail
node_query_pipeline -->|"retrieves candidates"| node_hybrid_retriever
node_hybrid_retriever -->|"runs dense search"| node_dense_retriever
node_hybrid_retriever -->|"runs sparse search"| node_bm25_index
node_dense_retriever -->|"embeds query"| node_embedder
node_dense_retriever -->|"searches vectors"| node_chroma_store
node_query_pipeline -->|"reranks candidates"| node_reranker
node_query_pipeline -->|"builds context"| node_context_builder
node_query_pipeline -->|"generates answer"| node_generator
node_query_pipeline -->|"checks citations"| node_output_guardrail
node_output_guardrail -->|"returns grounded answer"| node_user

click node_ui "https://github.com/dhruva-31/multimodel-rag/blob/main/streamlit_app.py"
click node_ingest_pipeline "https://github.com/dhruva-31/multimodel-rag/blob/main/pipelines/ingestion_pipeline.py"
click node_loader_factory "https://github.com/dhruva-31/multimodel-rag/blob/main/ingestion/loader_factory.py"
click node_format_loaders "https://github.com/dhruva-31/multimodel-rag/tree/main/ingestion"
click node_chunker "https://github.com/dhruva-31/multimodel-rag/blob/main/preprocessing/chunker.py"
click node_embedder "https://github.com/dhruva-31/multimodel-rag/blob/main/embeddings/dense_embeddings.py"
click node_document_model "https://github.com/dhruva-31/multimodel-rag/blob/main/models/document.py"
click node_metadata_model "https://github.com/dhruva-31/multimodel-rag/blob/main/models/metadata.py"
click node_chroma_store "https://github.com/dhruva-31/multimodel-rag/blob/main/vectorstore/chroma_store.py"
click node_bm25_index "https://github.com/dhruva-31/multimodel-rag/blob/main/sparse/bm25_index.py"
click node_query_pipeline "https://github.com/dhruva-31/multimodel-rag/blob/main/pipelines/query_pipeline.py"
click node_input_guardrail "https://github.com/dhruva-31/multimodel-rag/blob/main/guardrails/input_guardrail.py"
click node_hybrid_retriever "https://github.com/dhruva-31/multimodel-rag/blob/main/retrieval/hybrid_retriever.py"
click node_dense_retriever "https://github.com/dhruva-31/multimodel-rag/blob/main/retrieval/dense_retriever.py"
click node_reranker "https://github.com/dhruva-31/multimodel-rag/blob/main/retrieval/reranker.py"
click node_context_builder "https://github.com/dhruva-31/multimodel-rag/blob/main/generation/context_builder.py"
click node_generator "https://github.com/dhruva-31/multimodel-rag/blob/main/generation/generator.py"
click node_output_guardrail "https://github.com/dhruva-31/multimodel-rag/blob/main/guardrails/output_guardrail.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest_pipeline,node_loader_factory,node_format_loaders,node_chunker,node_user,node_ui toneBlue
class node_embedder,node_bm25_index,node_hybrid_retriever,node_dense_retriever,node_reranker toneAmber
class node_query_pipeline,node_input_guardrail,node_context_builder,node_generator,node_output_guardrail toneMint
class node_document_model,node_metadata_model,node_chroma_store,node_vector_db toneRose
```

## Components

### Document Ingestion

- [Ingestion Pipeline](https://github.com/dhruva-31/multimodel-rag/blob/main/pipelines/ingestion_pipeline.py): `pipelines/ingestion_pipeline.py`
- [Loader Factory](https://github.com/dhruva-31/multimodel-rag/blob/main/ingestion/loader_factory.py): `ingestion/loader_factory.py`
- [Format Loaders](https://github.com/dhruva-31/multimodel-rag/tree/main/ingestion): `ingestion`
- [Document Chunker](https://github.com/dhruva-31/multimodel-rag/blob/main/preprocessing/chunker.py): `preprocessing/chunker.py`

### Retrieval and Ranking

- [Dense Embedder](https://github.com/dhruva-31/multimodel-rag/blob/main/embeddings/dense_embeddings.py): `embeddings/dense_embeddings.py`
- [BM25 Index](https://github.com/dhruva-31/multimodel-rag/blob/main/sparse/bm25_index.py): `sparse/bm25_index.py`
- [Hybrid Retriever](https://github.com/dhruva-31/multimodel-rag/blob/main/retrieval/hybrid_retriever.py): `retrieval/hybrid_retriever.py`
- [Dense Retriever](https://github.com/dhruva-31/multimodel-rag/blob/main/retrieval/dense_retriever.py): `retrieval/dense_retriever.py`
- [Cross-Encoder Reranker](https://github.com/dhruva-31/multimodel-rag/blob/main/retrieval/reranker.py): `retrieval/reranker.py`

### Grounded Answering

- [Query Pipeline](https://github.com/dhruva-31/multimodel-rag/blob/main/pipelines/query_pipeline.py): `pipelines/query_pipeline.py`
- [Input Guardrail](https://github.com/dhruva-31/multimodel-rag/blob/main/guardrails/input_guardrail.py): `guardrails/input_guardrail.py`
- [Context Builder](https://github.com/dhruva-31/multimodel-rag/blob/main/generation/context_builder.py): `generation/context_builder.py`
- [LLM Generator](https://github.com/dhruva-31/multimodel-rag/blob/main/generation/generator.py): `generation/generator.py`
- [Output Guardrail](https://github.com/dhruva-31/multimodel-rag/blob/main/guardrails/output_guardrail.py): `guardrails/output_guardrail.py`

### Data and Storage

- [Document Model](https://github.com/dhruva-31/multimodel-rag/blob/main/models/document.py): `models/document.py`
- [Metadata Model](https://github.com/dhruva-31/multimodel-rag/blob/main/models/metadata.py): `models/metadata.py`
- [Chroma Store](https://github.com/dhruva-31/multimodel-rag/blob/main/vectorstore/chroma_store.py): `vectorstore/chroma_store.py`
- **Persistent Vector Store**

### Other components

- **User**
- [Streamlit UI](https://github.com/dhruva-31/multimodel-rag/blob/main/streamlit_app.py): `streamlit_app.py`

## Connections

- User → Streamlit UI: uploads and asks (evidence: `README.md`)
- Streamlit UI → Ingestion Pipeline: builds knowledge base (evidence: `streamlit_app.py`)
- Streamlit UI → Query Pipeline: submits questions (evidence: `streamlit_app.py`)
- Ingestion Pipeline → Loader Factory: selects loader (evidence: `pipelines/ingestion_pipeline.py`)
- Loader Factory → Format Loaders: dispatches formats (evidence: `ingestion/loader_factory.py`)
- Format Loaders → Document Model: creates documents (evidence: `ingestion/docx_loader.py`)
- Format Loaders → Metadata Model: creates metadata (evidence: `ingestion/docx_loader.py`)
- Ingestion Pipeline → Document Chunker: chunks documents (evidence: `pipelines/ingestion_pipeline.py`)
- Ingestion Pipeline → Dense Embedder: embeds chunks (evidence: `pipelines/ingestion_pipeline.py`)
- Ingestion Pipeline → Chroma Store: stores vectors (evidence: `pipelines/ingestion_pipeline.py`)
- Ingestion Pipeline → BM25 Index: builds index (evidence: `pipelines/ingestion_pipeline.py`)
- Chroma Store → Persistent Vector Store: persists vectors (evidence: `vectorstore/chroma_store.py`)
- Query Pipeline → Input Guardrail: checks query (evidence: `pipelines/query_pipeline.py`)
- Query Pipeline → Hybrid Retriever: retrieves candidates (evidence: `pipelines/query_pipeline.py`)
- Hybrid Retriever → Dense Retriever: runs dense search (evidence: `retrieval/hybrid_retriever.py`)
- Hybrid Retriever → BM25 Index: runs sparse search (evidence: `retrieval/hybrid_retriever.py`)
- Dense Retriever → Dense Embedder: embeds query (evidence: `retrieval/dense_retriever.py`)
- Dense Retriever → Chroma Store: searches vectors (evidence: `retrieval/dense_retriever.py`)
- Query Pipeline → Cross-Encoder Reranker: reranks candidates (evidence: `pipelines/query_pipeline.py`)
- Query Pipeline → Context Builder: builds context (evidence: `pipelines/query_pipeline.py`)
- Query Pipeline → LLM Generator: generates answer (evidence: `pipelines/query_pipeline.py`)
- Query Pipeline → Output Guardrail: checks citations (evidence: `pipelines/query_pipeline.py`)
- Output Guardrail → User: returns grounded answer (evidence: `README.md`)
