How yeswanthupputuri/rag-chunking-strategy-evaluation works
This repository is an experimental RAG evaluation system: users provide a document and Q&A set, build three chunking-strategy indexes, then compare generated answers against references. The main runtime is the Streamlit app; Python entry points also support single-question and dataset evaluation. The graph preserves retrieval, generation, answer and retrieval metrics, and aggregation. The README documents separate Chroma collections, but the corresponding vector-store implementation is absent from the supplied tree; its node is therefore unpathed. Some app wiring is shown only in partial excerpts, so documented relationships use README evidence.
Main entry points: app.py Streamlit interface · rag_pipeline.py RAG pipeline · evaluate_single_question.py Single-question runner · evaluate_dataset.py Dataset runner
Components 21
Document preparation
- Document loader
src/loaders/document_loader.py - Fixed chunking
src/chunking/fixed_chunking.py - Recursive chunking
src/chunking/recursive_chunking.py - Semantic chunking
src/chunking/semantic_chunking.py
RAG answering
- Embedding model
src/embeddings/embedding_model.py - Chroma collections
- Retriever
src/retrieval/retriever.py - Document reranker
src/reranking/reranker.py - RAG pipeline
src/rag/rag_pipeline.py - Local language model
src/llm/huggingface_llm.py
Evaluation
- Q&A loader
src/loaders/qa_loader.py - Answer metrics
src/evaluation/answer_metrics.py - Retrieval metrics
src/evaluation/retrieval_metrics.py - Evaluation coordinator
src/evaluation/evaluator.py - Result aggregation
src/evaluation/aggregate_results.py
User workflows
- Streamlit interface
app.py - Single-question runner
src/evaluate_single_question.py - Dataset runner
src/evaluate_dataset.py
Other components
- User
- Document input
- Q&A dataset
Connections 26
- User → Streamlit interface: uses
README.md - Document input → Streamlit interface: uploads
README.md - Q&A dataset → Streamlit interface: uploads
README.md - Streamlit interface → Document loader: loads document
app.py - Streamlit interface → Q&A loader: loads Q&A
app.py - Streamlit interface → Fixed chunking: creates chunks
app.py - Streamlit interface → Recursive chunking: creates chunks
app.py - Streamlit interface → Semantic chunking: creates chunks
app.py - Streamlit interface → Chroma collections: indexes chunks
README.md - Semantic chunking → Embedding model: uses embeddings
src/chunking/semantic_chunking.py - Streamlit interface → Embedding model: creates model
app.py - Chroma collections → Retriever: searches collection
README.md - Retriever → Document reranker: reranks candidates
src/retrieval/retriever.py - RAG pipeline → Retriever: retrieves results
src/rag/rag_pipeline.py - RAG pipeline → Local language model: generates answer
src/rag/rag_pipeline.py - Streamlit interface → RAG pipeline: requests answers
app.py - Dataset runner → Q&A loader: loads dataset
src/evaluate_dataset.py - Dataset runner → RAG pipeline: requests answers
src/evaluate_dataset.py - Single-question runner → Q&A loader: loads dataset
src/evaluate_single_question.py - Single-question runner → RAG pipeline: requests answers
src/evaluate_single_question.py - Streamlit interface → Answer metrics: scores answers
app.py - Single-question runner → Answer metrics: scores answers
src/evaluate_single_question.py - Dataset runner → Evaluation coordinator: evaluates results
src/evaluate_dataset.py - Evaluation coordinator → Answer metrics: scores answers
src/evaluation/evaluator.py - Evaluation coordinator → Retrieval metrics: scores retrieval
src/evaluation/evaluator.py - Streamlit interface → Result aggregation: summarizes strategies
app.py