# olvenint/multi-format-rag architecture

> Architecture diagram of the GitHub repository olvenint/multi-format-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/olvenint/multi-format-rag
- Repository: https://github.com/olvenint/multi-format-rag

## Overview

This is a multi-format document RAG system. Users upload Word, PDF, TXT, or Excel files, which are parsed into chunks and indexed; they then ask questions through the web UI or REST API and receive document-grounded, optionally streamed answers. The main stages are ingestion, hybrid retrieval and ranking, answer generation, and session memory. README capabilities include image descriptions, incremental updates, Redis caching, and multi-session isolation. Some subsystem links are documented in the README; source excerpts are partial, so uncertain calls are omitted.

Main entry points: [`knowledge_base.py`](https://github.com/olvenint/multi-format-rag/blob/main/knowledge_base.py) (Knowledge base ingestion), [`qa_service.py`](https://github.com/olvenint/multi-format-rag/blob/main/qa_service.py) (Hybrid retrieval), [`loader_factory.py`](https://github.com/olvenint/multi-format-rag/blob/main/loader_factory.py) (Loader factory), [`app_api.py`](https://github.com/olvenint/multi-format-rag/blob/main/app_api.py) (REST and SSE API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_service["Service interface"]
  node_web["Chat and upload UI<br/>[index.html]"]
  node_api["REST and SSE API<br/>[app_api.py]"]
end

subgraph group_ingest["Document ingestion"]
  node_factory["Loader factory<br/>[loader_factory.py]"]
  node_docx["Word parser<br/>[docx_loader.py]"]
  node_pdf["PDF parser<br/>[pdf_loader.py]"]
  node_text["TXT parser<br/>[txt_loader.py]"]
  node_excel["Excel parser<br/>[excel_loader.py]"]
  node_base["Loader contract<br/>[base_loader.py]"]
  node_kb["Knowledge base ingestion<br/>[knowledge_base.py]"]
end

subgraph group_retrieve["Retrieval and ranking"]
  node_terms["Term extraction<br/>[term_extractor.py]"]
  node_rewriter["Query rewriting<br/>[query_rewriter.py]"]
  node_retriever["Hybrid retrieval<br/>[qa_service.py]"]
  node_bm25["BM25 index<br/>[bm25_index.py]"]
  node_reranker["Rerank service<br/>[reranker.py]"]
end

subgraph group_answer["Answer and memory"]
  node_qa["Answer orchestration<br/>[qa_service.py]"]
  node_memory["Conversation memory<br/>[memory.py]"]
end

subgraph group_infra["Persistence and integrations"]
  node_chroma[("Chroma vector store<br/>[knowledge_base.py]")]
  node_redis[("Redis cache and sessions<br/>[redis_cache.py]")]
  node_deepseek["DeepSeek model"]
  node_vision["Vision model"]
  node_config["Runtime configuration<br/>[config_data.py]"]
end

node_user(("User"))

node_user -->|"uses"| node_web
node_web -->|"calls"| node_api
node_user -->|"calls"| node_api
node_api -->|"dispatches uploads"| node_factory
node_api -->|"ingests documents"| node_kb
node_api -->|"submits questions"| node_qa
node_factory -->|"creates"| node_docx
node_factory -->|"creates"| node_pdf
node_factory -->|"creates"| node_text
node_factory -->|"creates"| node_excel
node_docx -->|"implements"| node_base
node_pdf -->|"implements"| node_base
node_text -->|"implements"| node_base
node_excel -->|"implements"| node_base
node_kb -->|"writes chunks"| node_chroma
node_kb -->|"describes images"| node_vision
node_kb -->|"updates terms"| node_terms
node_kb -->|"clears caches"| node_redis
node_qa -->|"loads and saves"| node_memory
node_qa -->|"requests passages"| node_retriever
node_retriever -->|"rewrites query"| node_rewriter
node_retriever -->|"vector searches"| node_chroma
node_retriever -->|"lexical searches"| node_bm25
node_retriever -->|"reranks candidates"| node_reranker
node_retriever -->|"reads and writes cache"| node_redis
node_qa -->|"generates answer"| node_deepseek
node_qa -->|"checks answer cache"| node_redis
node_terms -->|"supplies vocabulary"| node_rewriter
node_memory -->|"stores sessions"| node_redis
node_memory -->|"reads settings"| node_config
node_kb -->|"reads settings"| node_config
node_qa -->|"reads settings"| node_config

click node_web "https://github.com/olvenint/multi-format-rag/blob/main/static/index.html"
click node_api "https://github.com/olvenint/multi-format-rag/blob/main/app_api.py"
click node_factory "https://github.com/olvenint/multi-format-rag/blob/main/loader_factory.py"
click node_docx "https://github.com/olvenint/multi-format-rag/blob/main/loaders/docx_loader.py"
click node_pdf "https://github.com/olvenint/multi-format-rag/blob/main/loaders/pdf_loader.py"
click node_text "https://github.com/olvenint/multi-format-rag/blob/main/loaders/txt_loader.py"
click node_excel "https://github.com/olvenint/multi-format-rag/blob/main/loaders/excel_loader.py"
click node_base "https://github.com/olvenint/multi-format-rag/blob/main/loaders/base_loader.py"
click node_kb "https://github.com/olvenint/multi-format-rag/blob/main/knowledge_base.py"
click node_terms "https://github.com/olvenint/multi-format-rag/blob/main/term_extractor.py"
click node_rewriter "https://github.com/olvenint/multi-format-rag/blob/main/query_rewriter.py"
click node_retriever "https://github.com/olvenint/multi-format-rag/blob/main/qa_service.py"
click node_bm25 "https://github.com/olvenint/multi-format-rag/blob/main/bm25_index.py"
click node_reranker "https://github.com/olvenint/multi-format-rag/blob/main/reranker.py"
click node_qa "https://github.com/olvenint/multi-format-rag/blob/main/qa_service.py"
click node_memory "https://github.com/olvenint/multi-format-rag/blob/main/memory.py"
click node_chroma "https://github.com/olvenint/multi-format-rag/blob/main/knowledge_base.py"
click node_redis "https://github.com/olvenint/multi-format-rag/blob/main/redis_cache.py"
click node_config "https://github.com/olvenint/multi-format-rag/blob/main/config_data.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web,node_api,node_user toneBlue
class node_factory,node_docx,node_pdf,node_text,node_excel,node_base,node_kb toneAmber
class node_terms,node_rewriter,node_retriever,node_bm25,node_reranker toneMint
class node_qa,node_memory toneRose
class node_chroma,node_redis,node_deepseek,node_vision,node_config toneIndigo
```

## Components

### Service interface

- [Chat and upload UI](https://github.com/olvenint/multi-format-rag/blob/main/static/index.html): `static/index.html`
- [REST and SSE API](https://github.com/olvenint/multi-format-rag/blob/main/app_api.py): `app_api.py`

### Document ingestion

- [Loader factory](https://github.com/olvenint/multi-format-rag/blob/main/loader_factory.py): `loader_factory.py`
- [Word parser](https://github.com/olvenint/multi-format-rag/blob/main/loaders/docx_loader.py): `loaders/docx_loader.py`
- [PDF parser](https://github.com/olvenint/multi-format-rag/blob/main/loaders/pdf_loader.py): `loaders/pdf_loader.py`
- [TXT parser](https://github.com/olvenint/multi-format-rag/blob/main/loaders/txt_loader.py): `loaders/txt_loader.py`
- [Excel parser](https://github.com/olvenint/multi-format-rag/blob/main/loaders/excel_loader.py): `loaders/excel_loader.py`
- [Loader contract](https://github.com/olvenint/multi-format-rag/blob/main/loaders/base_loader.py): `loaders/base_loader.py`
- [Knowledge base ingestion](https://github.com/olvenint/multi-format-rag/blob/main/knowledge_base.py): `knowledge_base.py`

### Retrieval and ranking

- [Term extraction](https://github.com/olvenint/multi-format-rag/blob/main/term_extractor.py): `term_extractor.py`
- [Query rewriting](https://github.com/olvenint/multi-format-rag/blob/main/query_rewriter.py): `query_rewriter.py`
- [Hybrid retrieval](https://github.com/olvenint/multi-format-rag/blob/main/qa_service.py): `qa_service.py`
- [BM25 index](https://github.com/olvenint/multi-format-rag/blob/main/bm25_index.py): `bm25_index.py`
- [Rerank service](https://github.com/olvenint/multi-format-rag/blob/main/reranker.py): `reranker.py`

### Answer and memory

- [Answer orchestration](https://github.com/olvenint/multi-format-rag/blob/main/qa_service.py): `qa_service.py`
- [Conversation memory](https://github.com/olvenint/multi-format-rag/blob/main/memory.py): `memory.py`

### Persistence and integrations

- [Chroma vector store](https://github.com/olvenint/multi-format-rag/blob/main/knowledge_base.py): `knowledge_base.py`
- [Redis cache and sessions](https://github.com/olvenint/multi-format-rag/blob/main/redis_cache.py): `redis_cache.py`
- **DeepSeek model**
- **Vision model**
- [Runtime configuration](https://github.com/olvenint/multi-format-rag/blob/main/config_data.py): `config_data.py`

### Other components

- **User**

## Connections

- User → Chat and upload UI: uses (evidence: `README.md`)
- Chat and upload UI → REST and SSE API: calls (evidence: `README.md`)
- User → REST and SSE API: calls (evidence: `README.md`)
- REST and SSE API → Loader factory: dispatches uploads (evidence: `README.md`)
- REST and SSE API → Knowledge base ingestion: ingests documents (evidence: `README.md`)
- REST and SSE API → Answer orchestration: submits questions (evidence: `README.md`)
- Loader factory → Word parser: creates (evidence: `loader_factory.py`)
- Loader factory → PDF parser: creates (evidence: `loader_factory.py`)
- Loader factory → TXT parser: creates (evidence: `loader_factory.py`)
- Loader factory → Excel parser: creates (evidence: `loader_factory.py`)
- Word parser → Loader contract: implements (evidence: `loaders/docx_loader.py`)
- PDF parser → Loader contract: implements (evidence: `loaders/pdf_loader.py`)
- TXT parser → Loader contract: implements (evidence: `loaders/txt_loader.py`)
- Excel parser → Loader contract: implements (evidence: `loaders/excel_loader.py`)
- Knowledge base ingestion → Chroma vector store: writes chunks (evidence: `README.md`)
- Knowledge base ingestion → Vision model: describes images (evidence: `README.md`)
- Knowledge base ingestion → Term extraction: updates terms (evidence: `term_extractor.py`)
- Knowledge base ingestion → Redis cache and sessions: clears caches (evidence: `knowledge_base.py`)
- Answer orchestration → Conversation memory: loads and saves (evidence: `qa_service.py`)
- Answer orchestration → Hybrid retrieval: requests passages (evidence: `qa_service.py`)
- Hybrid retrieval → Query rewriting: rewrites query (evidence: `qa_service.py`)
- Hybrid retrieval → Chroma vector store: vector searches (evidence: `README.md`)
- Hybrid retrieval → BM25 index: lexical searches (evidence: `README.md`)
- Hybrid retrieval → Rerank service: reranks candidates (evidence: `README.md`)
- Hybrid retrieval → Redis cache and sessions: reads and writes cache (evidence: `qa_service.py`)
- Answer orchestration → DeepSeek model: generates answer (evidence: `README.md`)
- Answer orchestration → Redis cache and sessions: checks answer cache (evidence: `README.md`)
- Term extraction → Query rewriting: supplies vocabulary (evidence: `query_rewriter.py`)
- Conversation memory → Redis cache and sessions: stores sessions (evidence: `README.md`)
- Conversation memory → Runtime configuration: reads settings (evidence: `memory.py`)
- Knowledge base ingestion → Runtime configuration: reads settings (evidence: `knowledge_base.py`)
- Answer orchestration → Runtime configuration: reads settings (evidence: `qa_service.py`)
