# thakshil/rag-intelligence architecture

> Architecture diagram of the GitHub repository thakshil/rag-intelligence, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thakshil/rag-intelligence
- Repository: https://github.com/thakshil/rag-intelligence

## Overview

RAG Document Intelligence turns uploaded PDFs into document-scoped, searchable chunks and answers questions with grounded citations. Users enter through the Streamlit interface or REST API; ingestion extracts text with conditional OCR, chunks and embeds it, then indexes it. Queries are embedded, retrieved from the active document, and passed to grounded generation with provider fallback. The graph preserves the documented telemetry and status capabilities. Some UI and ingestion-stage links rely on the README; unsampled implementation details are not asserted.

Main entry points: [`src/rag/pipeline.py`](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/pipeline.py) (RAG pipeline), [`api/main.py`](https://github.com/thakshil/rag-intelligence/blob/main/api/main.py) (REST API), [`src/rag/retrieve.py`](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/retrieve.py) (Dense retrieval), [`src/rag/embed.py`](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/embed.py) (Embedding model)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_experience["User experience"]
  node_ui["Streamlit UI<br/>[app.py]"]
  node_api["REST API<br/>[main.py]"]
  node_status[("Document status<br/>[status_store.py]")]
end

subgraph group_ingestion["Document ingestion"]
  node_extract["PDF extraction<br/>[extract.py]"]
  node_ocr["Tesseract OCR"]
  node_chunk["Text chunking<br/>[chunk.py]"]
  node_tokenizer["Token counting<br/>[tokenizer.py]"]
  node_embed["Embedding model<br/>[embed.py]"]
end

subgraph group_query["Retrieval and answers"]
  node_pipeline["RAG pipeline<br/>[pipeline.py]"]
  node_retriever["Dense retrieval<br/>[retrieve.py]"]
  node_generate["Grounded generation<br/>[generate.py]"]
  node_llm["LLM providers<br/>[llm.py]"]
end

subgraph group_state["State and configuration"]
  node_cache[("Embedding cache<br/>[embed.py]")]
  node_store["Vector store<br/>[store.py]"]
  node_qdrant[("Qdrant")]
  node_manifest["Ingestion manifest<br/>[pipeline.py]"]
  node_config["Settings<br/>[config.py]"]
end

node_user(("User"))

node_user -->|"uses"| node_ui
node_user -->|"uploads or queries"| node_api
node_ui -->|"uploads documents"| node_pipeline
node_api -->|"ingests and queries"| node_pipeline
node_api -->|"updates status"| node_status
node_pipeline -->|"extracts text"| node_extract
node_extract -.->|"uses when needed"| node_ocr
node_pipeline -->|"chunks text"| node_chunk
node_chunk -->|"counts tokens"| node_tokenizer
node_pipeline -->|"encodes chunks"| node_embed
node_embed -->|"reads and writes"| node_cache
node_pipeline -->|"indexes chunks"| node_store
node_store -->|"reads and writes"| node_qdrant
node_pipeline -->|"retrieves passages"| node_retriever
node_retriever -->|"embeds query"| node_embed
node_retriever -->|"searches vectors"| node_store
node_pipeline -->|"generates answer"| node_generate
node_generate -->|"invokes provider"| node_llm
node_pipeline -->|"reads document stats"| node_manifest
node_api -->|"loads settings"| node_config
node_pipeline -->|"uses settings"| node_config

click node_ui "https://github.com/thakshil/rag-intelligence/blob/main/ui/app.py"
click node_api "https://github.com/thakshil/rag-intelligence/blob/main/api/main.py"
click node_status "https://github.com/thakshil/rag-intelligence/blob/main/api/status_store.py"
click node_pipeline "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/pipeline.py"
click node_extract "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/extract.py"
click node_chunk "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/chunk.py"
click node_tokenizer "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/tokenizer.py"
click node_embed "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/embed.py"
click node_cache "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/embed.py"
click node_retriever "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/retrieve.py"
click node_store "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/store.py"
click node_generate "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/generate.py"
click node_llm "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/llm.py"
click node_manifest "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/pipeline.py"
click node_config "https://github.com/thakshil/rag-intelligence/blob/main/src/rag/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui,node_api,node_status,node_user toneBlue
class node_extract,node_ocr,node_chunk,node_tokenizer,node_embed toneAmber
class node_pipeline,node_retriever,node_generate,node_llm toneMint
class node_cache,node_store,node_qdrant,node_manifest,node_config toneRose
```

## Components

### User experience

- [Streamlit UI](https://github.com/thakshil/rag-intelligence/blob/main/ui/app.py): `ui/app.py`
- [REST API](https://github.com/thakshil/rag-intelligence/blob/main/api/main.py): `api/main.py`
- [Document status](https://github.com/thakshil/rag-intelligence/blob/main/api/status_store.py): `api/status_store.py`

### Document ingestion

- [PDF extraction](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/extract.py): `src/rag/extract.py`
- **Tesseract OCR**
- [Text chunking](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/chunk.py): `src/rag/chunk.py`
- [Token counting](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/tokenizer.py): `src/rag/tokenizer.py`
- [Embedding model](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/embed.py): `src/rag/embed.py`

### Retrieval and answers

- [RAG pipeline](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/pipeline.py): `src/rag/pipeline.py`
- [Dense retrieval](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/retrieve.py): `src/rag/retrieve.py`
- [Grounded generation](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/generate.py): `src/rag/generate.py`
- [LLM providers](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/llm.py): `src/rag/llm.py`

### State and configuration

- [Embedding cache](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/embed.py): `src/rag/embed.py`
- [Vector store](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/store.py): `src/rag/store.py`
- **Qdrant**
- [Ingestion manifest](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/pipeline.py): `src/rag/pipeline.py`
- [Settings](https://github.com/thakshil/rag-intelligence/blob/main/src/rag/config.py): `src/rag/config.py`

### Other components

- **User**

## Connections

- User → Streamlit UI: uses (evidence: `README.md`)
- User → REST API: uploads or queries (evidence: `README.md`)
- Streamlit UI → RAG pipeline: uploads documents (evidence: `README.md`)
- REST API → RAG pipeline: ingests and queries (evidence: `api/main.py`)
- REST API → Document status: updates status (evidence: `api/main.py`)
- RAG pipeline → PDF extraction: extracts text (evidence: `README.md`)
- PDF extraction → Tesseract OCR: uses when needed (evidence: `README.md`)
- RAG pipeline → Text chunking: chunks text (evidence: `README.md`)
- Text chunking → Token counting: counts tokens (evidence: `src/rag/chunk.py`)
- RAG pipeline → Embedding model: encodes chunks (evidence: `README.md`)
- Embedding model → Embedding cache: reads and writes (evidence: `src/rag/embed.py`)
- RAG pipeline → Vector store: indexes chunks (evidence: `README.md`)
- Vector store → Qdrant: reads and writes (evidence: `src/rag/store.py`)
- RAG pipeline → Dense retrieval: retrieves passages (evidence: `src/rag/pipeline.py`)
- Dense retrieval → Embedding model: embeds query (evidence: `src/rag/retrieve.py`)
- Dense retrieval → Vector store: searches vectors (evidence: `src/rag/retrieve.py`)
- RAG pipeline → Grounded generation: generates answer (evidence: `src/rag/pipeline.py`)
- Grounded generation → LLM providers: invokes provider (evidence: `src/rag/generate.py`)
- RAG pipeline → Ingestion manifest: reads document stats (evidence: `src/rag/pipeline.py`)
- REST API → Settings: loads settings (evidence: `api/main.py`)
- RAG pipeline → Settings: uses settings (evidence: `src/rag/pipeline.py`)