How jnandanavanam-byte/oracle-rag-comparison-lab works
Oracle RAG Comparison Lab is a notebook-based framework for comparing two Oracle ERP document RAG pipelines. The workflow loads PDF, DOCX, and spreadsheet documents, applies recursive or section-based chunking, embeds chunks with MiniLM or BGE-M3, indexes them in ChromaDB or FAISS, compares retrieval, and generates answers with Qwen3:14B. The README documents these stages and pairings; sampled source confirms document loading and configuration, but notebook implementations were not inspected, so notebook-stage links cite the README. The semantic chunking notebook is an archived evaluation, not a final pipeline.
Main entry points: document_loader.py Document Loader · 09_Retrieval_Comparison.ipynb Retrieval Comparison · 02_Recursive_Chunking.ipynb Recursive Chunking · 03_Section_Based_Chunking.ipynb Section Chunking
Components 14
Document Ingestion
- Document Loader
src/loaders/document_loader.py
Chunking and Embeddings
- Recursive Chunking
notebooks/02_Recursive_Chunking.ipynb - Section Chunking
notebooks/03_Section_Based_Chunking.ipynb - Semantic Evaluation
notebooks/03_Semantic_Chunking.ipynb - MiniLM Embeddings
notebooks/05_MiniLM_Embeddings.ipynb - BGE-M3 Embeddings
notebooks/06_BGEM3_Embeddings.ipynb
Vector Indexing
- Chroma Index
vectorstores/chroma_minilm - FAISS Index
vectorstores/faiss_bgem3
Retrieval and Answers
- Retrieval Comparison
notebooks/09_Retrieval_Comparison.ipynb - Qwen Inference
notebooks/10_Qwen_Inference.ipynb
Runtime Support
- Pipeline Configuration
src/config.py - Device Detection
src/utils/device_utils.py
Other components
- Oracle Manuals
- Generated Answers
Connections 12
- Oracle Manuals → Document Loader: provides files
README.md - Document Loader → Recursive Chunking: loads documents
README.md - Document Loader → Section Chunking: loads documents
README.md - Recursive Chunking → MiniLM Embeddings: chunks for
README.md - Section Chunking → BGE-M3 Embeddings: chunks for
README.md - MiniLM Embeddings → Chroma Index: indexes with
README.md - BGE-M3 Embeddings → FAISS Index: indexes with
README.md - Chroma Index → Retrieval Comparison: retrieves from
README.md - FAISS Index → Retrieval Comparison: retrieves from
README.md - Retrieval Comparison → Qwen Inference: provides context
README.md - Qwen Inference → Generated Answers: generates
README.md - Oracle Manuals → Semantic Evaluation: evaluation corpus
README.md