# ndricca/ocr-rag-qa architecture

> Architecture diagram of the GitHub repository ndricca/ocr-rag-qa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ndricca/ocr-rag-qa
- Repository: https://github.com/ndricca/ocr-rag-qa

## Overview

##### Project type and overall purpose
- This is a full-stack, two-phase system:
– Offline Document Processing pipeline (OCR → chunk → embed → store).
– Online Question-Answering pipeline (User → Websocket API → Agentic RAG workflow → LLM & Tools → Response).

##### Main architectural components
A. Client (Frontend)
- Web browser or any Websocket client that connects to FastAPI `/ws`.
B. API Layer (FastAPI server)
- `src/app.py` with Websocket endpoint.
- Hosts the ConversationHandler, routes messages to agents.
C. Conversation Handler & Routing Agent
- `src/utils/conversation_handler.py` implements async orchestration.
- The Routing Agent decides:
– If question needs retrieval → invokes Get Context Tool.
– If math reasoning required → invokes Math Reasoning Tool.
– Else returns an answer directly via Azure OpenAI.
D. Tool Clients (Agents)
1. Get Context Agent
- `src/utils/tool_client.py` → `get_context()`
- Queries Qdrant vector DB.
2. Math Reasoning Agent
- `src/utils/tool_client.py` → `math_reasoning()`
- Performs reasoning via Azure OpenAI.
E. LLM Handlers
- `src/llm_handlers/*_handler.py`
- Handle low-level API calls to: Azure OpenAI, Jina Embeddings, Mistral OCR.
F. Preprocessing & OCR
- `src/utils/ocr.py` calls Mistral OCR.
- `src/preprocessing/chunking.py` splits text into chunks.
G. Vector Database
- Qdrant (embedded SQLite or remote) storing embeddings + metadata.
H. Data Storage
- `data/raw`: original documents.
- `data/processed`: intermediate OCR/chunk/embedding JSON & MD.
- `data/qdrant`: persisted Qdrant files.
I. Scripts & Development
- `scripts/load_embeddings.py`: standalone loader for offline phase.
- `research/`: exploratory notebooks (excluded from prod).
J. Containerization & Deployment
- Dockerfile / Podmanfile for building an image.
- Env vars via `.env` (sample.env) for API keys and Qdrant config.

##### Interactions and data flows
Offline Document Processing:
1. `scripts/load_embeddings.py` (or equivalent) reads `data/raw/*.pdf`.
2. Calls Mistral OCR → outputs text → saved in `data/processed`.
3. Chunking → produce smaller sections.
4. Jina AI Embeddings → vector representations.
5. Store embeddings + chunks in Qdrant (via LLM handler → tool_client).

Online Question Answering:
1. Client connects via Websocket → sends user question.
2. FastAPI → ConversationHandler.main → Routing Agent.
3. Routing Agent may:
a. Call Get Context → Qdrant → retrieve relevant chunks.
b. Call Math Reasoning → LLM reasoning on numerical text.
4. Routing Agent calls Azure OpenAI LLM for final answer.
5. Server streams response events over WS → Client.

##### Architectural patterns & principles
- Separation of concerns: preprocessing vs QA; API vs business logic vs data.
- Agentic pattern: small “tool” agents invoked by a routing agent.
- Modular handlers: each LLM or service has its own handler class.
- Asynchronous flow: FastAPI + Websockets + async agent calls.
- Data versioning: raw, processed, Qdrant snapshots under `data/`.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Offline Ingestion Pipeline
    subgraph "Offline Document Processing" 
        direction TB
        RawData["Raw Documents"]:::datastore
        LoadEmbeddings["Load Embeddings Script"]:::script
        OCRUtil["OCR Utility"]:::internal
        Chunking["Chunking Module"]:::internal
        JinaEmbeddings["Jina Embeddings Handler"]:::external
        QdrantDB["Qdrant Vector DB"]:::datastore
        RawData -->|"reads PDFs"| LoadEmbeddings
        LoadEmbeddings -->|"OCR request → text"| OCRUtil
        OCRUtil -->|"text → chunks"| Chunking
        Chunking -->|"chunks → embeddings"| JinaEmbeddings
        JinaEmbeddings -->|"store vectors"| QdrantDB
    end

    %% Online QA Pipeline
    subgraph "Online Question Answering"
        direction TB
        Client["Websocket Client"]:::external
        FastAPIServer["FastAPI Server"]:::internal
        ConversationHandler["Conversation Handler"]:::internal
        RoutingAgent["Routing Agent"]:::internal
        GetContextAgent["Get Context Agent"]:::internal
        MathReasoningAgent["Math Reasoning Agent"]:::internal
        AzureHandler["Azure OpenAI Handler"]:::external
        QdrantDB2["Qdrant Vector DB"]:::datastore
        Client -->|"Websocket connect /ws"| FastAPIServer
        FastAPIServer -->|"route message"| ConversationHandler
        ConversationHandler -->|"orchestrate"| RoutingAgent
        RoutingAgent -->|"invoke get_context()"| GetContextAgent
        RoutingAgent -->|"invoke math_reasoning()"| MathReasoningAgent
        GetContextAgent -->|"query top-k vectors"| QdrantDB2
        MathReasoningAgent -->|"LLM reasoning"| AzureHandler
        RoutingAgent -->|"LLM call → final answer"| AzureHandler
        AzureHandler -->|"stream response"| FastAPIServer
        FastAPIServer -->|"stream events"| Client
    end

    %% Containerization
    subgraph "Container Boundary (Docker)" 
        direction TB
        FastAPIServer
        ConversationHandler
        RoutingAgent
        GetContextAgent
        MathReasoningAgent
        OCRUtil
        Chunking
        AzureHandler
        JinaEmbeddings
    end

    %% Other Components
    ProcessedData["Processed Data Store"]:::datastore
    Research["Exploratory Notebooks"]:::script
    PyProject["pyproject.toml"]:::config
    PythonVersion[".python-version"]:::config
    EnvSample["Environment Template"]:::config
    Dockerfile["Dockerfile"]:::config

    %% Relationships to config and data
    OCRUtil --> ProcessedData
    Chunking --> ProcessedData
    LoadEmbeddings --> ProcessedData
    FastAPIServer --> EnvSample
    FastAPIServer --> PyProject
    FastAPIServer --> PythonVersion
    FastAPIServer --> Dockerfile

    %% Click Events
    click FastAPIServer "https://github.com/ndricca/ocr-rag-qa/blob/main/src/app.py"
    click ConversationHandler "https://github.com/ndricca/ocr-rag-qa/blob/main/src/utils/conversation_handler.py"
    click RoutingAgent "https://github.com/ndricca/ocr-rag-qa/blob/main/src/prompts/routing_agent.py"
    click ToolClient "https://github.com/ndricca/ocr-rag-qa/blob/main/src/utils/tool_client.py"
    click AzureHandler "https://github.com/ndricca/ocr-rag-qa/blob/main/src/llm_handlers/azure_openai_handler.py"
    click JinaHandler "https://github.com/ndricca/ocr-rag-qa/blob/main/src/llm_handlers/jina_handler.py"
    click MistralHandler "https://github.com/ndricca/ocr-rag-qa/blob/main/src/llm_handlers/mistral_handler.py"
    click BaseHandler "https://github.com/ndricca/ocr-rag-qa/blob/main/src/llm_handlers/base_handler.py"
    click OCRUtil "https://github.com/ndricca/ocr-rag-qa/blob/main/src/utils/ocr.py"
    click Chunking "https://github.com/ndricca/ocr-rag-qa/blob/main/src/preprocessing/chunking.py"
    click ToolAgentPrompts "https://github.com/ndricca/ocr-rag-qa/blob/main/src/prompts/tool_agents.py"
    click QdrantData "https://github.com/ndricca/ocr-rag-qa/tree/main/data/qdrant"
    click QdrantCollections "https://github.com/ndricca/ocr-rag-qa/tree/main/data/qdrant/collection"
    click RawData "https://github.com/ndricca/ocr-rag-qa/tree/main/data/raw"
    click ProcessedData "https://github.com/ndricca/ocr-rag-qa/tree/main/data/processed"
    click LoadEmbeddings "https://github.com/ndricca/ocr-rag-qa/blob/main/scripts/load_embeddings.py"
    click Research "https://github.com/ndricca/ocr-rag-qa/tree/main/research/"
    click Dockerfile "https://github.com/ndricca/ocr-rag-qa/tree/main/Dockerfile"
    click EnvSample "https://github.com/ndricca/ocr-rag-qa/blob/main/sample.env"
    click PyProject "https://github.com/ndricca/ocr-rag-qa/blob/main/pyproject.toml"
    click PythonVersion "https://github.com/ndricca/ocr-rag-qa/blob/main/.python-version"

    %% Styles
    classDef external fill:#b3cde0,stroke:#08519c,color:#000;
    classDef internal fill:#c7e9c0,stroke:#238b45,color:#000;
    classDef datastore fill:#fdae6b,stroke:#e6550d,color:#000;
    classDef script fill:#decbe4,stroke:#7b3294,color:#000;
    classDef config fill:#fed976,stroke:#b15928,color:#000;
```
