How neelsshah2006/omnirag works
1. Repository shape
- This is a single backend service, not a multi-service monorepo.
- The runtime entrypoint is backend/app/main.py, which wires FastAPI, middleware, logging, config, and startup/shutdown behavior.
2. External system boundaries
- The backend depends on four main infrastructure services:
- Redis for caching and request-adjacent state
- Qdrant for vector search
- MinIO for raw document/object storage
- SQLite or PostgreSQL for relational metadata
- It also calls Groq APIs for chat, vision, and query rewriting.
3. API layer
- backend/app/api/v1/endpoints/ exposes two functional surfaces:
- document upload/ingestion
- question answering via RAG
- The API layer is thin; it delegates orchestration to service and RAG modules rather than embedding business logic.
4. Document ingestion path
- Uploads flow through DocumentService, then into the ingestion pipeline under backend/app/rag/ingestion/.
- The ingestion pipeline loads the file, enriches content, chunks it, generates embeddings, and writes both metadata and vectors.
- Uploads are asynchronous: the HTTP request returns quickly while background tasks continue processing.
5. Document lifecycle and audit trail
- Documents have a lifecycle tracked in relational storage: uploaded → processing → ready / failed.
- There is an immutable event log model/repository pair for recording ingestion stages.
- SHA-256 hashing is used to deduplicate identical re-uploads and avoid repeated processing.
6. Content loading and enrichment
- Loaders in backend/app/rag/loaders/ parse supported file types such as PDF, DOCX, PPTX, TXT, and Markdown.
- Processors in backend/app/rag/processors/ add multimodal enrichment:
- images are described by a vision model
- tables are summarized by an LLM
- These enrichments are merged back into chunk text so non-text content becomes retrievable.
7. Chunking strategy
- Chunking is selected by document type.
- Structured documents use a title-aware / structure-preserving chunker.
- Plain text and Markdown fall back to recursive character chunking.
- This boundary matters because chunking directly determines retrieval quality and citation granularity.
8. Embedding and indexing layer
- Dense embeddings are produced by a SentenceTransformers-based service.
- Sparse embeddings use a FastEmbed/BM25 provider.
- Both are persisted into Qdrant through the vector storage abstraction, which supports named dense and sparse vectors.
- The vector layer is separate from relational storage, so metadata and retrieval data are decoupled.
9. Retrieval pipeline
- Query-time retrieval is organized under backend/app/rag/retrieval/.
- The flow is:
- rewrite the user question
- retrieve candidate chunks using dense or hybrid search
- optionally fuse dense and sparse scores with reciprocal rank fusion
- rerank candidates with a cross-encoder
- build final context for generation
- Redis-backed embedding caching reduces repeated embedding cost on identical text.
10. Answer generation
- The final RAG chain composes the retrieved context and sends it to a Groq LLM.
- Prompts are centralized in backend/app/ai/prompts/ for query, RAG, table, and vision tasks.
- Responses return grounded answers plus source chunks, vector scores, rerank scores, token usage, and latency.
11. AI provider abstraction
- The backend/app/ai/ tree is a provider boundary.
- Each AI capability has its own service and provider interface:
- embeddings
- LLM chat
- vision
- reranking
- sparse embeddings
- This keeps vendor-specific behavior isolated from the ingestion and retrieval pipelines.
12. Persistence and repositories
- ORM models live in backend/app/models/.
- Repositories in backend/app/repositories/ provide the data-access boundary for documents and document events.
- Storage is split by concern:
- database for relational state
- object store for source files
- vector store for retrieval data
13. Cross-cutting runtime concerns
- backend/app/core/ holds config, exceptions, logging, Redis wiring, and security helpers.
- backend/app/middleware/request.py adds per-request tracing via request IDs and latency logging.
- backend/app/utils/ contains shared operational helpers such as hashing, batching, retries, concurrency, and time utilities.
14. Deployment and tooling
- The service is containerized with a backend Dockerfile and a Docker Compose setup for infrastructure dependencies.
- Development uses Python 3.12, uv, FastAPI/Uvicorn, Ruff, mypy, and pytest.
- The architecture is production-oriented but still self-contained: one API service orchestrating document ingestion and RAG over external stores and model providers.
Main entry points: ingestion/pipeline.py Ingestion pipeline · retrieval/pipeline.py Retrieval pipeline · document_service.py Document service · service.py Query service
Components 24
Runtime
Application entrypoint and request handling
- Main app · Wires the API, middleware, config, logging, and lifecycle hooks
backend/app/main.py - Request middleware · Adds request IDs and latency logging
backend/app/middleware/request.py - Core · Configuration, exceptions, logging, Redis wiring, and security helpers
backend/app/core
API
HTTP endpoints and orchestration
- API router · Mounts the versioned HTTP surface
backend/app/api/v1/router.py - Documents API · Handles upload and ingestion requests
backend/app/api/v1/endpoints/documents.py - Query API · Handles question answering requests
backend/app/api/v1/endpoints/query.py - Document service · Coordinates document lifecycle, deduplication, and async ingestion
backend/app/services/document_service.py
RAG
Ingestion, retrieval, and answer generation
- Ingestion pipeline · Loads, enriches, chunks, embeds, and indexes documents
backend/app/rag/ingestion/pipeline.py - Loaders · Parses PDF, DOCX, PPTX, TXT, and Markdown sources
backend/app/rag/loaders/service.py - Processors · Adds image descriptions and table summaries to document content
backend/app/rag/processors/service.py - Chunking · Selects structure-aware or recursive chunking by document type
backend/app/rag/chunking/service.py - RAG chain · Builds grounded responses from retrieved context and prompts
backend/app/rag/chain/service.py - Query service · Rewrites queries and drives retrieval plus generation
backend/app/rag/query/service.py - Retrieval pipeline · Dense or hybrid retrieval, fusion, reranking, and context building
backend/app/rag/retrieval/pipeline.py
AI
Provider abstractions for model capabilities
- Embedding AI · Dense embedding abstraction over sentence-transformer providers
backend/app/ai/embeddings/service.py - LLM AI · Chat/completion abstraction backed by Groq
backend/app/ai/llm/service.py - Vision AI · Image understanding abstraction backed by Groq
backend/app/ai/vision/service.py - Reranker AI · Ranks retrieved candidates by relevance
backend/app/ai/reranker/service.py - Sparse AI · Produces sparse/BM25-style vectors for hybrid search
backend/app/ai/sparse/service.py
Storage
Persistence and external infrastructure
- SQL database · Stores document state, metadata, and event logs
backend/app/storage/database - Object store · Persists raw uploaded files and document objects
backend/app/storage/object/minio.py - Vector store · Stores dense and sparse retrieval vectors
backend/app/storage/vector/qdrant.py - Redis · Caches embeddings and request-adjacent state
- Groq APIs · External chat, vision, and query rewriting provider
Connections 31
- Main app → Core: configures · Loads runtime config, logging, security, and Redis wiring
- Main app → Request middleware: installs · Adds per-request tracing and latency logging
- Main app → API router: mounts · Registers versioned HTTP routes
- API router → Documents API: routes · Exposes upload and ingestion endpoints
- API router → Query API: routes · Exposes question answering endpoints
- Documents API → Document service: delegates · Hands off upload and lifecycle operations
- Query API → Query service: delegates · Hands off RAG query orchestration
- Document service → Object store: stores file · Persists original uploaded content
- Document service → SQL database: updates state · Tracks document lifecycle and audit events
- Document service → Ingestion pipeline: starts · Schedules asynchronous document processing
- Ingestion pipeline → Loaders: loads · Parses source documents into content
- Ingestion pipeline → Processors: enriches · Adds image and table understanding
- Ingestion pipeline → Chunking: chunks · Splits content for retrieval granularity
- Ingestion pipeline → Embedding AI: embeds · Produces dense embeddings for chunks
- Ingestion pipeline → Sparse AI: sparse indexes · Produces sparse vectors for hybrid search
- Ingestion pipeline → Vector store: indexes · Writes retrieval vectors to Qdrant
- Ingestion pipeline → SQL database: persists metadata · Stores document and event metadata
- Query service → Retrieval pipeline: retrieves · Runs query-time candidate gathering
- Query service → RAG chain: generates · Builds the final grounded answer
- Retrieval pipeline → Embedding AI: embeds query · Creates dense query vectors, with cache support
- Retrieval pipeline → Sparse AI: sparse query · Creates sparse query vectors for hybrid search
- Retrieval pipeline → Vector store: searches · Fetches dense or hybrid candidates from Qdrant
- Retrieval pipeline → Reranker AI: reranks · Reorders candidates with cross-encoder scoring
- Query service → LLM AI: rewrites prompt · Uses LLM-backed query rewriting when needed
- RAG chain → LLM AI: calls · Sends context and prompts for answer generation
- Processors → Vision AI: describes images · Uses vision models for multimodal enrichment
- Processors → LLM AI: summarizes tables · Uses LLMs to summarize tabular content
- LLM AI → Groq APIs: uses · Invokes Groq chat/completion APIs
- Vision AI → Groq APIs: uses · Invokes Groq vision APIs
- Core → Redis: connects · Provides cache and request-adjacent state
- Retrieval pipeline → Redis: caches embeddings · Reuses embeddings for repeated text and queries