How ziqiwang0908/ragrouter-bench works
RAGRouter-Bench is a benchmark for comparing retrieval-augmented generation strategies across query–corpus pairs. Its primary workflow processes corpus documents into chunks and embeddings/graphs, runs a selected retrieval paradigm to produce answers, and evaluates answer quality and corpus properties. It also includes query generation and query classification. Relationships shown are limited to README-documented capabilities and references in the SOURCE INDEX; unsampled implementations may have additional wiring not represented here.
Main entry points: main.py CLI · PromptTemplate.py Prompt templates · IterativeRAGDo.py Iterative RAG · NaiveRAGDo.py Naive RAG
Components 22
Entry and orchestration
- CLI
main.py
Corpus preparation
- Chunking
RAGCore/Chunk/ChunkDo.py - Embedding
RAGCore/Embedding/EmbeddingDo.py - Embedding store
RAGCore/Embedding/EmbeddingSave.py - Graph construction
RAGCore/Graph/GraphDo.py - Graph store
RAGCore/Graph/GraphSave.py - Indexing
RAGCore/Index/IndexDo.py
Retrieval paradigms
- Naive RAG
RAGCore/Retriever/NaiveRAG/NaiveRAGDo.py - Graph RAG
RAGCore/Retriever/GraphRAG/GraphRAGDo.py - Hybrid RAG
RAGCore/Retriever/HybridRAG/HybridRAGDo.py - Iterative RAG
RAGCore/Retriever/IterativeRAG/IterativeRAGDo.py - LLM-only
RAGCore/Retriever/LLMDirect/LLMDirectDo.py - Prompt templates
RAGCore/Prompt/PromptTemplate.py
Benchmark and analysis
- Answer evaluation
BenchCore/Evaluation/ResultEvaluation/EvaluationDo.py - Semantic corpus evaluation
BenchCore/Evaluation/CorpusEvaluation/SemanticEvaluation/EvaluationDo.py - Structure corpus evaluation
BenchCore/Evaluation/CorpusEvaluation/StructureEvaluation/EvaluationDo.py
Query preparation
- Query generation
BenchCore/QueryGeneration/GenerateDo.py - Query preprocessing
BenchCore/QueryGeneration/Preprocess/ChainBuilder.py - Query validation
BenchCore/QueryGeneration/ValidateDo.py - Query classification
RAGCore/Query/QueryClassifyDo.py
Other components
- Benchmark user
- LLM provider
Connections 41
- Benchmark user → CLI: runs stages
README.md - CLI → Chunking: dispatches to
main.py - CLI → Embedding: dispatches to
main.py - CLI → Embedding store: dispatches to
main.py - CLI → Graph construction: dispatches to
main.py - CLI → Graph store: dispatches to
main.py - CLI → Indexing: dispatches to
main.py - CLI → Naive RAG: dispatches to
main.py - CLI → Graph RAG: dispatches to
main.py - CLI → Hybrid RAG: dispatches to
main.py - CLI → Iterative RAG: dispatches to
main.py - CLI → LLM-only: dispatches to
main.py - CLI → Answer evaluation: dispatches to
main.py - CLI → Semantic corpus evaluation: dispatches to
main.py - CLI → Structure corpus evaluation: dispatches to
main.py - CLI → Query classification: dispatches to
main.py - Embedding → Chunking: embeds chunks
RAGCore/Embedding/EmbeddingDo.py - Embedding → Embedding store: persists vectors
RAGCore/Embedding/EmbeddingDo.py - Graph construction → Chunking: uses chunks
RAGCore/Graph/GraphDo.py - Graph construction → Graph store: persists graph data
RAGCore/Graph/GraphDo.py - Indexing → Embedding store: loads embeddings
RAGCore/Index/IndexSave.py - Naive RAG → Embedding: uses embeddings
RAGCore/Retriever/NaiveRAG/NaiveRAGDo.py - Graph RAG → Embedding: uses embeddings
RAGCore/Retriever/GraphRAG/GraphRAGDo.py - Iterative RAG → Naive RAG: may retrieve via
RAGCore/Retriever/IterativeRAG/IterativeRAGDo.py - Iterative RAG → Graph RAG: may retrieve via
RAGCore/Retriever/IterativeRAG/IterativeRAGDo.py - Naive RAG → Prompt templates: gets messages
RAGCore/Retriever/NaiveRAG/NaiveRAGDo.py - Graph RAG → Prompt templates: uses prompts
RAGCore/Retriever/GraphRAG/GraphRAGDo.py - Hybrid RAG → Prompt templates: uses prompts
RAGCore/Retriever/HybridRAG/HybridRAGDo.py - Iterative RAG → Prompt templates: uses prompts
RAGCore/Retriever/IterativeRAG/IterativeRAGDo.py - LLM-only → Prompt templates: uses prompts
RAGCore/Retriever/LLMDirect/LLMDirectDo.py - Naive RAG → LLM provider: requests generation
- Graph RAG → LLM provider: requests generation
- Hybrid RAG → LLM provider: requests generation
- Iterative RAG → LLM provider: requests generation
- LLM-only → LLM provider: requests generation
- Answer evaluation → Prompt templates: uses prompts
BenchCore/Evaluation/ResultEvaluation/EvaluationDo.py - Semantic corpus evaluation → Embedding store: reads embeddings
BenchCore/Evaluation/CorpusEvaluation/SemanticEvaluation/EvaluationDo.py - Structure corpus evaluation → Graph store: reads graph data
BenchCore/Evaluation/CorpusEvaluation/StructureEvaluation/EvaluationDo.py - Query preprocessing → Query generation: builds context for
BenchCore/QueryGeneration/Preprocess/ChainBuilder.py - CLI → Query generation: runs generation
- CLI → Query validation: runs validation