# neelsshah2006/omnirag architecture

> Architecture diagram of the GitHub repository neelsshah2006/omnirag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/neelsshah2006/omnirag
- Repository: https://github.com/neelsshah2006/omnirag

## Overview

1. **Repository shape**
   - This is a single backend service, not a multi-service monorepo.
   - The runtime entrypoint is `backend/app/main.py`, which wires FastAPI, middleware, logging, config, and startup/shutdown behavior.

2. **External system boundaries**
   - The backend depends on four main infrastructure services:
     - **Redis** for caching and request-adjacent state
     - **Qdrant** for vector search
     - **MinIO** for raw document/object storage
     - **SQLite or PostgreSQL** for relational metadata
   - It also calls **Groq APIs** for chat, vision, and query rewriting.

3. **API layer**
   - `backend/app/api/v1/endpoints/` exposes two functional surfaces:
     - document upload/ingestion
     - question answering via RAG
   - The API layer is thin; it delegates orchestration to service and RAG modules rather than embedding business logic.

4. **Document ingestion path**
   - Uploads flow through `DocumentService`, then into the ingestion pipeline under `backend/app/rag/ingestion/`.
   - The ingestion pipeline loads the file, enriches content, chunks it, generates embeddings, and writes both metadata and vectors.
   - Uploads are asynchronous: the HTTP request returns quickly while background tasks continue processing.

5. **Document lifecycle and audit trail**
   - Documents have a lifecycle tracked in relational storage: `uploaded → processing → ready / failed`.
   - There is an immutable event log model/repository pair for recording ingestion stages.
   - SHA-256 hashing is used to deduplicate identical re-uploads and avoid repeated processing.

6. **Content loading and enrichment**
   - Loaders in `backend/app/rag/loaders/` parse supported file types such as PDF, DOCX, PPTX, TXT, and Markdown.
   - Processors in `backend/app/rag/processors/` add multimodal enrichment:
     - images are described by a vision model
     - tables are summarized by an LLM
   - These enrichments are merged back into chunk text so non-text content becomes retrievable.

7. **Chunking strategy**
   - Chunking is selected by document type.
   - Structured documents use a title-aware / structure-preserving chunker.
   - Plain text and Markdown fall back to recursive character chunking.
   - This boundary matters because chunking directly determines retrieval quality and citation granularity.

8. **Embedding and indexing layer**
   - Dense embeddings are produced by a SentenceTransformers-based service.
   - Sparse embeddings use a FastEmbed/BM25 provider.
   - Both are persisted into Qdrant through the vector storage abstraction, which supports named dense and sparse vectors.
   - The vector layer is separate from relational storage, so metadata and retrieval data are decoupled.

9. **Retrieval pipeline**
   - Query-time retrieval is organized under `backend/app/rag/retrieval/`.
   - The flow is:
     - rewrite the user question
     - retrieve candidate chunks using dense or hybrid search
     - optionally fuse dense and sparse scores with reciprocal rank fusion
     - rerank candidates with a cross-encoder
     - build final context for generation
   - Redis-backed embedding caching reduces repeated embedding cost on identical text.

10. **Answer generation**
    - The final RAG chain composes the retrieved context and sends it to a Groq LLM.
    - Prompts are centralized in `backend/app/ai/prompts/` for query, RAG, table, and vision tasks.
    - Responses return grounded answers plus source chunks, vector scores, rerank scores, token usage, and latency.

11. **AI provider abstraction**
    - The `backend/app/ai/` tree is a provider boundary.
    - Each AI capability has its own service and provider interface:
      - embeddings
      - LLM chat
      - vision
      - reranking
      - sparse embeddings
    - This keeps vendor-specific behavior isolated from the ingestion and retrieval pipelines.

12. **Persistence and repositories**
    - ORM models live in `backend/app/models/`.
    - Repositories in `backend/app/repositories/` provide the data-access boundary for documents and document events.
    - Storage is split by concern:
      - database for relational state
      - object store for source files
      - vector store for retrieval data

13. **Cross-cutting runtime concerns**
    - `backend/app/core/` holds config, exceptions, logging, Redis wiring, and security helpers.
    - `backend/app/middleware/request.py` adds per-request tracing via request IDs and latency logging.
    - `backend/app/utils/` contains shared operational helpers such as hashing, batching, retries, concurrency, and time utilities.

14. **Deployment and tooling**
    - The service is containerized with a backend Dockerfile and a Docker Compose setup for infrastructure dependencies.
    - Development uses Python 3.12, `uv`, FastAPI/Uvicorn, Ruff, mypy, and pytest.
    - The architecture is production-oriented but still self-contained: one API service orchestrating document ingestion and RAG over external stores and model providers.

Main entry points: [`backend/app/rag/ingestion/pipeline.py`](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/ingestion/pipeline.py) (Ingestion pipeline), [`backend/app/rag/retrieval/pipeline.py`](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/retrieval/pipeline.py) (Retrieval pipeline), [`backend/app/services/document_service.py`](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/services/document_service.py) (Document service), [`backend/app/rag/query/service.py`](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/query/service.py) (Query service)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_main(("Main app<br/>FastAPI entrypoint<br/>[main.py]"))
  node_request_mw["Request middleware<br/>Tracing middleware<br/>[request.py]"]
  node_core["Core<br/>Config & infra"]
end

subgraph group_api["API"]
  node_api_router["API router<br/>v1 router<br/>[router.py]"]
  node_documents_api["Documents API<br/>HTTP endpoints<br/>[documents.py]"]
  node_query_api["Query API<br/>HTTP endpoints<br/>[query.py]"]
  node_document_service["Document service<br/>Orchestrator"]
end

subgraph group_rag["RAG"]
  node_ingestion_pipeline["Ingestion pipeline<br/>RAG pipeline<br/>[pipeline.py]"]
  node_loaders["Loaders<br/>[service.py]"]
  node_processors["Processors<br/>Multimodal enrichers<br/>[service.py]"]
  node_chunking["Chunking<br/>Chunker service<br/>[service.py]"]
  node_chain["RAG chain<br/>Answer pipeline<br/>[service.py]"]
  node_query_service["Query service<br/>RAG orchestrator<br/>[service.py]"]
  node_retrieval["Retrieval pipeline<br/>Search pipeline<br/>[pipeline.py]"]
end

subgraph group_ai["AI"]
  node_embeddings_ai["Embedding AI<br/>Embedding service<br/>[service.py]"]
  node_llm_ai["LLM AI<br/>Chat service<br/>[service.py]"]
  node_vision_ai["Vision AI<br/>Vision service<br/>[service.py]"]
  node_reranker_ai["Reranker AI<br/>Cross-encoder service<br/>[service.py]"]
  node_sparse_ai["Sparse AI<br/>Sparse embedding service<br/>[service.py]"]
end

subgraph group_storage["Storage"]
  node_database[("SQL database<br/>Relational store")]
  node_object_store[("Object store<br/>MinIO adapter<br/>[minio.py]")]
  node_vector_store[("Vector store<br/>Qdrant adapter<br/>[qdrant.py]")]
  node_redis[("Redis<br/>Cache backend")]
  node_groq(("Groq APIs<br/>Model provider"))
end

subgraph group_data["Data"]
end

node_main -->|"configures"| node_core
node_main -->|"installs"| node_request_mw
node_main -->|"mounts"| node_api_router
node_api_router -->|"routes"| node_documents_api
node_api_router -->|"routes"| node_query_api
node_documents_api -->|"delegates"| node_document_service
node_query_api -->|"delegates"| node_query_service
node_document_service -->|"stores file"| node_object_store
node_document_service -->|"updates state"| node_database
node_document_service -->|"starts"| node_ingestion_pipeline
node_ingestion_pipeline -->|"loads"| node_loaders
node_ingestion_pipeline -->|"enriches"| node_processors
node_ingestion_pipeline -->|"chunks"| node_chunking
node_ingestion_pipeline -->|"embeds"| node_embeddings_ai
node_ingestion_pipeline -->|"sparse indexes"| node_sparse_ai
node_ingestion_pipeline -->|"indexes"| node_vector_store
node_ingestion_pipeline -->|"persists metadata"| node_database
node_query_service -->|"retrieves"| node_retrieval
node_query_service -->|"generates"| node_chain
node_retrieval -->|"embeds query"| node_embeddings_ai
node_retrieval -->|"sparse query"| node_sparse_ai
node_retrieval -->|"searches"| node_vector_store
node_retrieval -->|"reranks"| node_reranker_ai
node_query_service -->|"rewrites prompt"| node_llm_ai
node_chain -->|"calls"| node_llm_ai
node_processors -->|"describes images"| node_vision_ai
node_processors -->|"summarizes tables"| node_llm_ai
node_llm_ai -->|"uses"| node_groq
node_vision_ai -->|"uses"| node_groq
node_core -->|"connects"| node_redis
node_retrieval -->|"caches embeddings"| node_redis

click node_main "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/main.py"
click node_request_mw "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/middleware/request.py"
click node_core "https://github.com/neelsshah2006/omnirag/tree/main/backend/app/core"
click node_api_router "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/api/v1/router.py"
click node_documents_api "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/api/v1/endpoints/documents.py"
click node_query_api "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/api/v1/endpoints/query.py"
click node_document_service "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/services/document_service.py"
click node_ingestion_pipeline "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/ingestion/pipeline.py"
click node_loaders "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/loaders/service.py"
click node_processors "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/processors/service.py"
click node_chunking "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/chunking/service.py"
click node_chain "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/chain/service.py"
click node_query_service "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/query/service.py"
click node_retrieval "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/retrieval/pipeline.py"
click node_embeddings_ai "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/embeddings/service.py"
click node_llm_ai "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/llm/service.py"
click node_vision_ai "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/vision/service.py"
click node_reranker_ai "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/reranker/service.py"
click node_sparse_ai "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/sparse/service.py"
click node_database "https://github.com/neelsshah2006/omnirag/tree/main/backend/app/storage/database"
click node_object_store "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/storage/object/minio.py"
click node_vector_store "https://github.com/neelsshah2006/omnirag/blob/main/backend/app/storage/vector/qdrant.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_main,node_request_mw,node_core toneBlue
class node_api_router,node_documents_api,node_query_api,node_document_service toneAmber
class node_ingestion_pipeline,node_loaders,node_processors,node_chunking,node_chain,node_query_service,node_retrieval toneMint
class node_embeddings_ai,node_llm_ai,node_vision_ai,node_reranker_ai,node_sparse_ai toneRose
class node_database,node_object_store,node_vector_store,node_redis,node_groq toneIndigo
```

## Components

### Runtime

Application entrypoint and request handling

- [Main app](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/main.py): FastAPI entrypoint · Wires the API, middleware, config, logging, and lifecycle hooks · `backend/app/main.py`
- [Request middleware](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/middleware/request.py): Tracing middleware · Adds request IDs and latency logging · `backend/app/middleware/request.py`
- [Core](https://github.com/neelsshah2006/omnirag/tree/main/backend/app/core): Config & infra · Configuration, exceptions, logging, Redis wiring, and security helpers · `backend/app/core`

### API

HTTP endpoints and orchestration

- [API router](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/api/v1/router.py): v1 router · Mounts the versioned HTTP surface · `backend/app/api/v1/router.py`
- [Documents API](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/api/v1/endpoints/documents.py): HTTP endpoints · Handles upload and ingestion requests · `backend/app/api/v1/endpoints/documents.py`
- [Query API](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/api/v1/endpoints/query.py): HTTP endpoints · Handles question answering requests · `backend/app/api/v1/endpoints/query.py`
- [Document service](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/services/document_service.py): Orchestrator · Coordinates document lifecycle, deduplication, and async ingestion · `backend/app/services/document_service.py`

### RAG

Ingestion, retrieval, and answer generation

- [Ingestion pipeline](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/ingestion/pipeline.py): RAG pipeline · Loads, enriches, chunks, embeds, and indexes documents · `backend/app/rag/ingestion/pipeline.py`
- [Loaders](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/loaders/service.py): File loaders · Parses PDF, DOCX, PPTX, TXT, and Markdown sources · `backend/app/rag/loaders/service.py`
- [Processors](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/processors/service.py): Multimodal enrichers · Adds image descriptions and table summaries to document content · `backend/app/rag/processors/service.py`
- [Chunking](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/chunking/service.py): Chunker service · Selects structure-aware or recursive chunking by document type · `backend/app/rag/chunking/service.py`
- [RAG chain](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/chain/service.py): Answer pipeline · Builds grounded responses from retrieved context and prompts · `backend/app/rag/chain/service.py`
- [Query service](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/query/service.py): RAG orchestrator · Rewrites queries and drives retrieval plus generation · `backend/app/rag/query/service.py`
- [Retrieval pipeline](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/rag/retrieval/pipeline.py): Search pipeline · Dense or hybrid retrieval, fusion, reranking, and context building · `backend/app/rag/retrieval/pipeline.py`

### AI

Provider abstractions for model capabilities

- [Embedding AI](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/embeddings/service.py): Embedding service · Dense embedding abstraction over sentence-transformer providers · `backend/app/ai/embeddings/service.py`
- [LLM AI](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/llm/service.py): Chat service · Chat/completion abstraction backed by Groq · `backend/app/ai/llm/service.py`
- [Vision AI](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/vision/service.py): Vision service · Image understanding abstraction backed by Groq · `backend/app/ai/vision/service.py`
- [Reranker AI](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/reranker/service.py): Cross-encoder service · Ranks retrieved candidates by relevance · `backend/app/ai/reranker/service.py`
- [Sparse AI](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/ai/sparse/service.py): Sparse embedding service · Produces sparse/BM25-style vectors for hybrid search · `backend/app/ai/sparse/service.py`

### Storage

Persistence and external infrastructure

- [SQL database](https://github.com/neelsshah2006/omnirag/tree/main/backend/app/storage/database): Relational store · Stores document state, metadata, and event logs · `backend/app/storage/database`
- [Object store](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/storage/object/minio.py): MinIO adapter · Persists raw uploaded files and document objects · `backend/app/storage/object/minio.py`
- [Vector store](https://github.com/neelsshah2006/omnirag/blob/main/backend/app/storage/vector/qdrant.py): Qdrant adapter · Stores dense and sparse retrieval vectors · `backend/app/storage/vector/qdrant.py`
- **Redis**: Cache backend · Caches embeddings and request-adjacent state
- **Groq APIs**: Model provider · External chat, vision, and query rewriting provider

## Connections

- Main app → Core: configures · Loads runtime config, logging, security, and Redis wiring
- Main app → Request middleware: installs · Adds per-request tracing and latency logging
- Main app → API router: mounts · Registers versioned HTTP routes
- API router → Documents API: routes · Exposes upload and ingestion endpoints
- API router → Query API: routes · Exposes question answering endpoints
- Documents API → Document service: delegates · Hands off upload and lifecycle operations
- Query API → Query service: delegates · Hands off RAG query orchestration
- Document service → Object store: stores file · Persists original uploaded content
- Document service → SQL database: updates state · Tracks document lifecycle and audit events
- Document service → Ingestion pipeline: starts · Schedules asynchronous document processing
- Ingestion pipeline → Loaders: loads · Parses source documents into content
- Ingestion pipeline → Processors: enriches · Adds image and table understanding
- Ingestion pipeline → Chunking: chunks · Splits content for retrieval granularity
- Ingestion pipeline → Embedding AI: embeds · Produces dense embeddings for chunks
- Ingestion pipeline → Sparse AI: sparse indexes · Produces sparse vectors for hybrid search
- Ingestion pipeline → Vector store: indexes · Writes retrieval vectors to Qdrant
- Ingestion pipeline → SQL database: persists metadata · Stores document and event metadata
- Query service → Retrieval pipeline: retrieves · Runs query-time candidate gathering
- Query service → RAG chain: generates · Builds the final grounded answer
- Retrieval pipeline → Embedding AI: embeds query · Creates dense query vectors, with cache support
- Retrieval pipeline → Sparse AI: sparse query · Creates sparse query vectors for hybrid search
- Retrieval pipeline → Vector store: searches · Fetches dense or hybrid candidates from Qdrant
- Retrieval pipeline → Reranker AI: reranks · Reorders candidates with cross-encoder scoring
- Query service → LLM AI: rewrites prompt · Uses LLM-backed query rewriting when needed
- RAG chain → LLM AI: calls · Sends context and prompts for answer generation
- Processors → Vision AI: describes images · Uses vision models for multimodal enrichment
- Processors → LLM AI: summarizes tables · Uses LLMs to summarize tabular content
- LLM AI → Groq APIs: uses · Invokes Groq chat/completion APIs
- Vision AI → Groq APIs: uses · Invokes Groq vision APIs
- Core → Redis: connects · Provides cache and request-adjacent state
- Retrieval pipeline → Redis: caches embeddings · Reuses embeddings for repeated text and queries