How ndricca/ocr-rag-qa works
Project type and overall purpose
• This is a full-stack, two-phase system:
– Offline Document Processing pipeline (OCR → chunk → embed → store).
– Online Question-Answering pipeline (User → Websocket API → Agentic RAG workflow → LLM & Tools → Response).
Main architectural components
A. Client (Frontend)
• Web browser or any Websocket client that connects to FastAPI /ws.
B. API Layer (FastAPI server)
• src/app.py with Websocket endpoint.
• Hosts the ConversationHandler, routes messages to agents.
C. Conversation Handler & Routing Agent
• src/utils/conversation_handler.py implements async orchestration.
• The Routing Agent decides:
– If question needs retrieval → invokes Get Context Tool.
– If math reasoning required → invokes Math Reasoning Tool.
– Else returns an answer directly via Azure OpenAI.
D. Tool Clients (Agents)
1. Get Context Agent
• src/utils/tool_client.py → get_context()
• Queries Qdrant vector DB.
2. Math Reasoning Agent
• src/utils/tool_client.py → math_reasoning()
• Performs reasoning via Azure OpenAI.
E. LLM Handlers
• src/llm_handlers/*_handler.py
• Handle low-level API calls to: Azure OpenAI, Jina Embeddings, Mistral OCR.
F. Preprocessing & OCR
• src/utils/ocr.py calls Mistral OCR.
• src/preprocessing/chunking.py splits text into chunks.
G. Vector Database
• Qdrant (embedded SQLite or remote) storing embeddings + metadata.
H. Data Storage
• data/raw: original documents.
• data/processed: intermediate OCR/chunk/embedding JSON & MD.
• data/qdrant: persisted Qdrant files.
I. Scripts & Development
• scripts/load_embeddings.py: standalone loader for offline phase.
• research/: exploratory notebooks (excluded from prod).
J. Containerization & Deployment
• Dockerfile / Podmanfile for building an image.
• Env vars via .env (sample.env) for API keys and Qdrant config.
Interactions and data flows
Offline Document Processing:
1. scripts/load_embeddings.py (or equivalent) reads data/raw/*.pdf.
2. Calls Mistral OCR → outputs text → saved in data/processed.
3. Chunking → produce smaller sections.
4. Jina AI Embeddings → vector representations.
5. Store embeddings + chunks in Qdrant (via LLM handler → tool_client).
Online Question Answering:
1. Client connects via Websocket → sends user question.
2. FastAPI → ConversationHandler.main → Routing Agent.
3. Routing Agent may:
a. Call Get Context → Qdrant → retrieve relevant chunks.
b. Call Math Reasoning → LLM reasoning on numerical text.
4. Routing Agent calls Azure OpenAI LLM for final answer.
5. Server streams response events over WS → Client.
Architectural patterns & principles
• Separation of concerns: preprocessing vs QA; API vs business logic vs data.
• Agentic pattern: small “tool” agents invoked by a routing agent.
• Modular handlers: each LLM or service has its own handler class.
• Asynchronous flow: FastAPI + Websockets + async agent calls.
• Data versioning: raw, processed, Qdrant snapshots under data/.