# jnandanavanam-byte/oracle-rag-comparison-lab architecture

> Architecture diagram of the GitHub repository jnandanavanam-byte/oracle-rag-comparison-lab, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jnandanavanam-byte/oracle-rag-comparison-lab
- Repository: https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab

## Overview

Oracle RAG Comparison Lab is a notebook-based framework for comparing two Oracle ERP document RAG pipelines. The workflow loads PDF, DOCX, and spreadsheet documents, applies recursive or section-based chunking, embeds chunks with MiniLM or BGE-M3, indexes them in ChromaDB or FAISS, compares retrieval, and generates answers with Qwen3:14B. The README documents these stages and pairings; sampled source confirms document loading and configuration, but notebook implementations were not inspected, so notebook-stage links cite the README. The semantic chunking notebook is an archived evaluation, not a final pipeline.

Main entry points: [`src/loaders/document_loader.py`](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/src/loaders/document_loader.py) (Document Loader), [`notebooks/09_Retrieval_Comparison.ipynb`](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/09_Retrieval_Comparison.ipynb) (Retrieval Comparison), [`notebooks/02_Recursive_Chunking.ipynb`](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/02_Recursive_Chunking.ipynb) (Recursive Chunking), [`notebooks/03_Section_Based_Chunking.ipynb`](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/03_Section_Based_Chunking.ipynb) (Section Chunking)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Document Ingestion"]
  node_loader["Document Loader<br/>[document_loader.py]"]
end

subgraph group_preparation["Chunking and Embeddings"]
  node_recursive["Recursive Chunking"]
  node_section["Section Chunking"]
  node_semantic["Semantic Evaluation"]
  node_minilm["MiniLM Embeddings"]
  node_bgem3["BGE-M3 Embeddings"]
end

subgraph group_indexing["Vector Indexing"]
  node_chroma[("Chroma Index")]
  node_faiss[("FAISS Index")]
end

subgraph group_evaluation["Retrieval and Answers"]
  node_retrieval["Retrieval Comparison"]
  node_qwen["Qwen Inference"]
end

subgraph group_support["Runtime Support"]
  node_config["Pipeline Configuration<br/>[config.py]"]
  node_device["Device Detection<br/>[device_utils.py]"]
end

node_manuals(("Oracle Manuals"))
node_answers["Generated Answers"]

node_manuals -->|"provides files"| node_loader
node_loader -->|"loads documents"| node_recursive
node_loader -->|"loads documents"| node_section
node_recursive -->|"chunks for"| node_minilm
node_section -->|"chunks for"| node_bgem3
node_minilm -->|"indexes with"| node_chroma
node_bgem3 -->|"indexes with"| node_faiss
node_chroma -->|"retrieves from"| node_retrieval
node_faiss -->|"retrieves from"| node_retrieval
node_retrieval -->|"provides context"| node_qwen
node_qwen -->|"generates"| node_answers
node_manuals -.->|"evaluation corpus"| node_semantic

click node_loader "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/src/loaders/document_loader.py"
click node_recursive "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/02_Recursive_Chunking.ipynb"
click node_section "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/03_Section_Based_Chunking.ipynb"
click node_semantic "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/03_Semantic_Chunking.ipynb"
click node_minilm "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/05_MiniLM_Embeddings.ipynb"
click node_bgem3 "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/06_BGEM3_Embeddings.ipynb"
click node_chroma "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/tree/main/vectorstores/chroma_minilm"
click node_faiss "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/tree/main/vectorstores/faiss_bgem3"
click node_retrieval "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/09_Retrieval_Comparison.ipynb"
click node_qwen "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/10_Qwen_Inference.ipynb"
click node_config "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/src/config.py"
click node_device "https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/src/utils/device_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_loader toneBlue
class node_recursive,node_section,node_semantic,node_minilm,node_bgem3 toneAmber
class node_chroma,node_faiss toneMint
class node_retrieval,node_qwen toneRose
class node_config,node_device,node_manuals,node_answers toneIndigo
```

## Components

### Document Ingestion

- [Document Loader](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/src/loaders/document_loader.py): `src/loaders/document_loader.py`

### Chunking and Embeddings

- [Recursive Chunking](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/02_Recursive_Chunking.ipynb): `notebooks/02_Recursive_Chunking.ipynb`
- [Section Chunking](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/03_Section_Based_Chunking.ipynb): `notebooks/03_Section_Based_Chunking.ipynb`
- [Semantic Evaluation](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/03_Semantic_Chunking.ipynb): `notebooks/03_Semantic_Chunking.ipynb`
- [MiniLM Embeddings](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/05_MiniLM_Embeddings.ipynb): `notebooks/05_MiniLM_Embeddings.ipynb`
- [BGE-M3 Embeddings](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/06_BGEM3_Embeddings.ipynb): `notebooks/06_BGEM3_Embeddings.ipynb`

### Vector Indexing

- [Chroma Index](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/tree/main/vectorstores/chroma_minilm): `vectorstores/chroma_minilm`
- [FAISS Index](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/tree/main/vectorstores/faiss_bgem3): `vectorstores/faiss_bgem3`

### Retrieval and Answers

- [Retrieval Comparison](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/09_Retrieval_Comparison.ipynb): `notebooks/09_Retrieval_Comparison.ipynb`
- [Qwen Inference](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/notebooks/10_Qwen_Inference.ipynb): `notebooks/10_Qwen_Inference.ipynb`

### Runtime Support

- [Pipeline Configuration](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/src/config.py): `src/config.py`
- [Device Detection](https://github.com/jnandanavanam-byte/oracle-rag-comparison-lab/blob/main/src/utils/device_utils.py): `src/utils/device_utils.py`

### Other components

- **Oracle Manuals**
- **Generated Answers**

## Connections

- Oracle Manuals → Document Loader: provides files (evidence: `README.md`)
- Document Loader → Recursive Chunking: loads documents (evidence: `README.md`)
- Document Loader → Section Chunking: loads documents (evidence: `README.md`)
- Recursive Chunking → MiniLM Embeddings: chunks for (evidence: `README.md`)
- Section Chunking → BGE-M3 Embeddings: chunks for (evidence: `README.md`)
- MiniLM Embeddings → Chroma Index: indexes with (evidence: `README.md`)
- BGE-M3 Embeddings → FAISS Index: indexes with (evidence: `README.md`)
- Chroma Index → Retrieval Comparison: retrieves from (evidence: `README.md`)
- FAISS Index → Retrieval Comparison: retrieves from (evidence: `README.md`)
- Retrieval Comparison → Qwen Inference: provides context (evidence: `README.md`)
- Qwen Inference → Generated Answers: generates (evidence: `README.md`)
- Oracle Manuals → Semantic Evaluation: evaluation corpus (evidence: `README.md`)
