How aayushbokde/decision_guided_rag works
This is a Streamlit research demo and evaluation codebase for comparing standard RAG with decision-guided context filtering and accumulated-evidence early exit. The user selects or enters a question, chooses a pipeline, and receives a generated answer with retrieval, decision, and timing details. Shared embedding and similarity-retrieval code supports the pipelines; separate evaluation modules assess answer quality and compare results. The README identifies KeywordOracle as a development proxy, not JEV. Some app and pipeline wiring is unsampled, so uncertain internal links are omitted.
Main entry points: app.py Streamlit demo · rag.py Filtered RAG · embeddings.py Text embedding · early_exit.py Early-exit RAG
Components 23
Interactive demo
- Streamlit demo
app.py - Question data
processed/questions.json - Chunk data
processed/chunks.json - Chunked questions
processed/chunked_questions.json - Chunk embeddings
embeddings/chunk_embeddings.json
RAG pipelines
- Normal RAG
src/jev_rag_research/normal_rag/rag.py - Filtered RAG
src/jev_rag_research/jev_rag/rag.py - Early-exit RAG
src/jev_rag_research/jev_rag/early_exit.py - Keyword oracle
src/jev_rag_research/jev_rag/oracle.py - Decision contract
src/jev_rag_research/jev_rag/decision.py
Shared retrieval
- Text embedding
src/jev_rag_research/shared/embeddings.py - Similarity retrieval
src/jev_rag_research/shared/retrieval.py - Dataset preparation
src/jev_rag_research/shared/prepare_dataset.py - Dataset chunking
src/jev_rag_research/shared/chunk_dataset.py - Dataset embedding
src/jev_rag_research/shared/embed_dataset.py
Research evaluation
- Retrieval evaluation
src/jev_rag_research/shared/evaluate_retrieval.py - Threshold sweep
src/jev_rag_research/evaluation/threshold_sweep.py - Answer evaluation
src/jev_rag_research/evaluation/evaluate_answers.py - Early-exit evaluation
src/jev_rag_research/evaluation/evaluate_early_exit_answers.py - Filtered evaluation
src/jev_rag_research/evaluation/evaluate_filtered_answers.py - Result comparison
src/jev_rag_research/evaluation/compare_rag_results.py
Other components
- Questioner
- Ollama service
Connections 35
- Questioner → Streamlit demo: asks question
- Streamlit demo → Question data: loads examples
- Streamlit demo → Chunk data: loads chunks
- Streamlit demo → Chunk embeddings: loads embeddings
- Streamlit demo → Normal RAG: runs selected mode
- Streamlit demo → Filtered RAG: runs selected mode
- Streamlit demo → Early-exit RAG: runs selected mode
- Normal RAG → Text embedding: embeds query
- Normal RAG → Similarity retrieval: retrieves chunks
- Normal RAG → Ollama service: generates answer
- Filtered RAG → Text embedding: embeds query
- Filtered RAG → Similarity retrieval: retrieves candidates
- Filtered RAG → Keyword oracle: scores relevance
- Filtered RAG → Ollama service: generates answer
- Early-exit RAG → Text embedding: embeds query
- Early-exit RAG → Similarity retrieval: retrieves candidates
- Early-exit RAG → Keyword oracle: checks evidence
- Early-exit RAG → Ollama service: generates answer
- Keyword oracle → Decision contract: returns decision
- Text embedding → Ollama service: requests embeddings
- Dataset preparation → Question data: prepares questions
- Dataset chunking → Chunk data: writes chunks
- Dataset chunking → Chunked questions: writes questions
- Dataset embedding → Chunk embeddings: writes embeddings
- Dataset embedding → Text embedding: embeds dataset
- Retrieval evaluation → Text embedding: embeds queries
- Retrieval evaluation → Similarity retrieval: measures recall
- Threshold sweep → Text embedding: embeds queries
- Threshold sweep → Similarity retrieval: retrieves chunks
- Threshold sweep → Keyword oracle: sweeps decisions
- Answer evaluation → Normal RAG: evaluates results
- Early-exit evaluation → Early-exit RAG: evaluates results
- Filtered evaluation → Filtered RAG: evaluates results
- Result comparison → Normal RAG: compares results
- Result comparison → Filtered RAG: compares results