prajwal25xd/rag-failure-diagnosis-tool

How prajwal25xd/rag-failure-diagnosis-tool works

This is a local Streamlit tool for answering tender questions while diagnosing possible RAG failures. A user submits a question and mode; the orchestrator plans a SQL, RAG, or hybrid route, gathers metadata and/or document evidence, produces an answer, and returns evidence and diagnostics for display. The documented diagnostic capabilities cover retrieval, chunk quality, conflicting evidence, and answer reasoning. Some connections below rely on the README’s architecture description because relevant source was not sampled; unsampled implementation details are not inferred.

Main entry points: orchestrator.py Query Orchestrator · main.py RAG Pipeline · config.py Runtime Configuration · hybrid_search.py Hybrid Ranking

Components 21

Question Interface

Streamlit Dashboard app.py

Query Workflow

Query Orchestrator agent/orchestrator.py
Route Planner agent/query_planner.py
Metadata Executor agent/sql_executor.py
Answer Merger agent/evidence_merger.py

Evidence Retrieval

RAG Pipeline main.py
Hybrid Ranking retrieval/hybrid_search.py
Vector Search retrieval/vector_search.py
Keyword Search retrieval/keyword_search.py

RAG Diagnostics

Retrieval Diagnosis diagnostics/retrieval_diagnosis.py
Chunking Diagnosis diagnostics/chunking_diagnosis.py
Conflict Diagnosis diagnostics/conflict_diagnosis.py
Reasoning Diagnosis diagnostics/reasoning_diagnosis.py
Diagnostic Summary diagnostics/diagnostic_summary.py

Data and Model Services

Tender Metadata
Indexed Chunks
HF Embeddings
LM Studio
LM Client agent/lm_studio_client.py
Runtime Configuration config.py

Other components

Questioner

Connections 27

Questioner → Streamlit Dashboard: submits question README.md
Streamlit Dashboard → Query Orchestrator: starts query README.md
Query Orchestrator → Route Planner: plans route agent/orchestrator.py
Route Planner → LM Client: requests plan
LM Client → LM Studio: calls model
Query Orchestrator → Metadata Executor: runs metadata query agent/orchestrator.py
Metadata Executor → Tender Metadata: reads metadata README.md
Query Orchestrator → RAG Pipeline: runs RAG agent/orchestrator.py
RAG Pipeline → Hybrid Ranking: retrieves evidence README.md
Hybrid Ranking → Vector Search: gets vector ranks retrieval/hybrid_search.py
Hybrid Ranking → Keyword Search: gets keyword ranks retrieval/hybrid_search.py
Vector Search → Indexed Chunks: searches vectors README.md
Keyword Search → Indexed Chunks: searches text README.md
Vector Search → HF Embeddings: embeds question README.md
Query Orchestrator → Answer Merger: synthesizes answer agent/orchestrator.py
Answer Merger → LM Studio: generates answer README.md
RAG Pipeline → Retrieval Diagnosis: checks retrieval main.py
RAG Pipeline → Chunking Diagnosis: checks chunks main.py
RAG Pipeline → Conflict Diagnosis: checks conflicts main.py
RAG Pipeline → Reasoning Diagnosis: checks reasoning main.py
Query Orchestrator → Diagnostic Summary: builds summary agent/orchestrator.py
Streamlit Dashboard → Questioner: displays report README.md
Runtime Configuration → Vector Search: configures search retrieval/vector_search.py
Runtime Configuration → Hybrid Ranking: configures ranking retrieval/hybrid_search.py
Runtime Configuration → Chunking Diagnosis: configures checks diagnostics/chunking_diagnosis.py
Runtime Configuration → Conflict Diagnosis: configures checks diagnostics/conflict_diagnosis.py
Runtime Configuration → Reasoning Diagnosis: configures checks diagnostics/reasoning_diagnosis.py

Drawn by GitDiagram on November 12, 2026 from the repository's file tree, README and source files. Read as Markdown ·