How mazleon/omni-rag works
OmniRAG is a monorepo for a production RAG platform, organized around a few clear runtime boundaries: an HTTP API, background workers, a web frontend, and shared service/domain code. The repository is mostly scaffolded here, but the directory layout and README describe the intended system architecture clearly.
1. Top-level shape: a multi-app monorepo
The repo separates deployable surfaces under apps/, shared business capabilities under services/, shared contracts under core/, and environment-specific concerns under infra/. That suggests the architecture is split by runtime role rather than by framework layer.
2. User-facing application
apps/frontend is the Next.js 15 frontend. It is the interactive client for document upload, query/answer flows, and likely evaluation or admin views. Because the README emphasizes SSE streaming and grounded answers, the frontend is not just a static UI; it is expected to consume streaming responses from the backend API.
3. API boundary
apps/api is the FastAPI service, the main synchronous entry point for the system. It likely handles authentication, document upload, query execution, and orchestration endpoints. The apps/api/routers folder indicates a router-based API surface, which is the main integration boundary for the frontend and any external clients.
4. Background execution
apps/worker is the asynchronous processing tier. The README says ingestion uses Arq worker fleets via Redis Streams, so this app is where long-running tasks are executed: document processing, chunking, embedding, indexing, and possibly evaluation jobs. This keeps heavy ingestion work off the request path.
5. Service decomposition
The services/ directory is split into ingestion, retrieval, orchestrator, and evaluation. This is the clearest indication of the core domain boundaries:
- ingestion handles turning raw uploaded files into structured, chunked content.
- retrieval handles dense/sparse search, structured filtering, and reranking.
- orchestrator likely coordinates the agent/tool loop and query execution pipeline.
- evaluation contains offline quality checks and the golden dataset used by CI gates.
6. Shared schemas and contracts
core/schemas is the shared contract layer. In a system like this, that is typically where request/response models, document schemas, chunk representations, retrieval result shapes, and evaluation artifacts live. This layer is important because it keeps the API, worker, and service modules aligned on the same data model.
7. Ingestion data flow
The ingestion pipeline described in the README is: upload document → async task queued → extract structure from PDFs/spreadsheets/images/slides → semantic chunking with overlap → embeddings → persistence/indexing. The key architectural point is that ingestion is asynchronous and idempotent, which allows large volumes of heterogeneous documents to be processed safely and repeatedly.
8. Retrieval and answer generation flow
Query-time processing is a multi-stage pipeline: dense retrieval from Qdrant, sparse BM25 lookup, structured filtering, fusion via RRF, reranking with Cohere rerank-v3.5, then answer generation via Claude/GPT-4o through OpenRouter. This is the core online path and explains why retrieval, orchestration, and model access are separated into distinct service layers.
9. Multi-tenancy boundary
The README explicitly calls out Postgres RLS and per-organization Qdrant collections. That means tenant isolation is enforced at both the transactional metadata layer and the vector-store layer. This is an important architectural boundary for any diagram: org context is not just an application concern, it is part of storage layout and query filtering.
10. Persistence and external systems
The system uses PostgreSQL 17 for OLTP/state, Qdrant for vector search, Redis Streams for task coordination, and Supabase Storage for raw files and derived artifacts. These services are the main infrastructure dependencies and define the durable data flow:
raw assets in object storage, metadata in Postgres, embeddings/chunks in Qdrant, and async work through Redis.
11. Observability and evaluation
OpenTelemetry, Langfuse, and ClickHouse are listed for observability, which implies tracing/metrics/logging are first-class across API, workers, and model calls. Evaluation is also first-class: services/evaluation, tests/eval, and .github/workflows/eval-gate.yml indicate offline RAGAS-style quality checks are part of the release process, not an afterthought.
12. Infrastructure and deployment
docker-compose.yml, infra/docker, infra/migrations, and infra/fly show three infrastructure layers: local dev composition, schema migrations, and Fly.io deployment. This suggests the repo is designed to run locally in a full stack, evolve via migrations, and eventually deploy to managed hosting.
13. CI/CD and quality gates
The GitHub workflows in .github/workflows/ci.yml and eval-gate.yml indicate two separate gates: general test/build checks and evaluation-specific merge protection. That is consistent with an RAG system where functional correctness alone is insufficient; retrieval/answer quality must also be validated before merging to main.
14. Repository maturity note
Many directories contain only .gitkeep files, so this repo is currently more of an architecture scaffold than a fully populated implementation. Still, the layout is explicit enough to infer the intended runtime graph: frontend and API at the edge, worker tier behind Redis, domain services in the middle, and Postgres/Qdrant/object storage as the durable backend.
Main entry points: api API · fly Deploy · worker Worker · ingestion Ingestion
Components 18
Apps
Deployable user-facing and background runtimes.
Services
Shared domain capabilities and pipelines.
- Ingestion · Turns raw documents into structured chunks and embeddings.
services/ingestion - Retrieval · Performs dense/sparse retrieval, filtering, fusion, and reranking.
services/retrieval - Orchestrator · Coordinates model/tool execution and answer generation.
services/orchestrator - Evaluation · Offline scoring, golden datasets, and release-time assessment.
services/evaluation
Core
Shared contracts and schemas.
- Schemas · Common request, response, and domain data models.
core/schemas
Infra
Runtime infrastructure, deployment, and persistence concerns.
- Postgres · Primary transactional database and tenant-aware metadata store.
infra/migrations - Qdrant · Per-tenant vector collections for semantic search.
- Redis · Stream-based coordination for asynchronous worker jobs.
- Object Store · Raw uploads and derived artifacts kept in external storage.
- Observability · Tracing, logging, and operational analytics across runtimes.
- Deploy · Docker, migrations, and Fly.io deployment assets.
infra/fly
Quality
Testing and evaluation gates.
- CI · Build/test automation and evaluation gate workflows.
.github/workflows/ci.yml - Eval Gate · Quality gate for retrieval and answer quality regression checks.
.github/workflows/eval-gate.yml - Integration Tests · Cross-component validation for service and runtime boundaries.
tests/integration - Unit Tests · Focused component and domain-level verification.
tests/unit
Connections 27
- Frontend → API: HTTP/SSE · Browser requests and streamed responses flow through the API.
- API → Schemas: uses · API requests and responses are typed by shared contracts.
- Worker → Schemas: uses · Jobs and persisted records share the same models.
- API → Redis: enqueue · Requests hand off long-running work to the async queue.
- Redis → Worker: dispatch · Queued jobs are delivered to background workers.
- Worker → Ingestion: runs · Workers execute the document processing pipeline.
- Ingestion → Object Store: read/write · Raw files and derived artifacts are persisted externally.
- Ingestion → Postgres: metadata · Ingestion persists document state and processing records.
- Ingestion → Qdrant: index · Embeddings and chunks are written to the vector store.
- API → Orchestrator: query flow · Online requests are handed to the answer orchestration layer.
- Orchestrator → Retrieval: retrieve · The orchestrator asks retrieval for candidate context.
- Retrieval → Qdrant: dense search · Semantic retrieval reads from vector collections.
- Retrieval → Postgres: filters · Structured metadata and tenant constraints come from Postgres.
- Orchestrator → Postgres: state · Query/session state and tenancy checks are stored centrally.
- API → Object Store: serve files · Uploads and derived assets are exposed through storage.
- API → Observability: telemetry · Request traces and metrics are emitted from the API.
- Worker → Observability: telemetry · Background jobs emit traces and operational signals.
- Orchestrator → Observability: telemetry · Model and retrieval spans are traced across the query path.
- Evaluation → Schemas: uses · Evaluation artifacts are defined against shared domain models.
- Evaluation → CI: gates · Quality checks are wired into automated CI workflows.
- Evaluation → Eval Gate: protects · Model and retrieval quality gates block regressions.
- CI → Integration Tests: runs · Continuous integration executes cross-system tests.
- CI → Unit Tests: runs · Continuous integration executes focused unit tests.
- Deploy → Postgres: migrate · Deployment includes schema evolution for the database.
- Deploy → Redis: provision · Infrastructure supports async coordination services.
- Deploy → Qdrant: provision · Infrastructure includes the vector search backend.
- Deploy → Object Store: provision · Infrastructure depends on external file storage.