How prajitha-j/re-ranker-rag-pipeline works
1. What this system is
- This repository implements an audit-focused RAG pipeline that answers natural-language questions over audit reports, IR paragraphs, compliance rules, and tabular evidence.
- The architecture is not a simple retriever-plus-LLM; it is a staged planning/execution system built around BAG (Bag-of-Actions-Graph).
2. Primary runtime entrypoints
- server_v3.py is the API-facing runtime, and streamlit_app.py is the interactive UI.
- Both frontends feed requests into pipeline.py, which is the main orchestrator for the end-to-end query flow.
3. End-to-end query flow
- A user query moves through three sequential stages: Interpret → Strategize → Synthesize.
- Between Strategize and Synthesize, retrieved evidence is re-ranked by evidence_reranker.py using a BGE re-ranker before the final LLM response is produced.
4. Interpret stage boundary
- interpret_bag.py handles query understanding: extracting entities like state, department, domain, and financial years.
- Supporting modules such as query_parser.py, schema_loader.py, and filter_normalizer*.py shape the user request into structured constraints that downstream tools can consume.
5. Strategize stage boundary
- strategize_bag.py decides which tools to invoke and in what order based on query intent.
- The pipeline explicitly supports three query types: count, content, and calculate, which drive different retrieval and computation behaviors.
6. Dynamic tool system
- tool_registry.py and execution.py form the dynamic dispatch layer.
- tools.py contains the tool implementations, while schemas/ contains JSON schemas that expose those tools to the LLM in a structured way.
- The key architectural point is that tools are discovered at runtime rather than hardcoded into prompt logic or execution branches.
7. Tool groups and responsibilities
- The toolset is organized around four functional groups: query understanding, filter planning, search execution, and calculation.
- table_agent.py and observation_agent.py suggest specialized internal helpers for table-heavy analysis and evidence handling, which fit into the tool-driven execution model.
8. Retrieval and data access layer
- audit_db_client.py is the main interface to the audit database, which the README indicates is PostgreSQL with pgvector used for retrieval.
- milvus_client.py and Milvus_metadata_helper.py provide Milvus support, likely as a secondary or legacy vector-search path.
- This makes the retrieval boundary a distinct subsystem from the LLM layer: evidence is fetched, filtered, and ranked before synthesis.
9. LLM and re-ranking services
- llm_client.py abstracts calls to the model serving backend, which the README describes as vLLM-hosted Mistral/Qwen models.
- Embeddings and ranking are externalized: BGE-M3 is used for embeddings, and TEI serves the BGE re-ranker.
- The final answer is produced only after evidence has been reordered by relevance.
10. Synthesis stage
- synthesize_bag.py combines the structured plan, ranked evidence, and query context into the final response.
- conv.py likely supports conversational state or response assembly, keeping the synthesis layer separate from retrieval and planning.
11. Configuration and service boundaries
- config.py and .env.example define the environment-driven integration points for databases, model servers, and other infrastructure.
- The README also names MinIO as document storage, so the repo sits in front of several external services rather than owning the full data plane itself.
12. Operational and diagnostic code
- The debug_*.py, health_check.py, Server_check.py, Log_analyzer.py, and Token_tracker.py files are support tooling for validation, observability, and troubleshooting.
- qa_packets/ and test_dynamic_tools.py indicate an offline evaluation and regression-testing layer for the tool-discovery and retrieval behavior.
13. What to draw in an architecture diagram
- Show two clients, UI and API, feeding into pipeline.py.
- From there, draw the three BAG stages, with a tool-dispatch branch in the middle and a reranking step before synthesis.
- Separate the internal orchestration layer from the external services: PostgreSQL/pgvector, Milvus, LLM serving via vLLM, embedding/rerank services, and document storage.
Main entry points: pipeline.py Pipeline · tools.py Tools · interpret_bag.py Interpret · config.py Config
Components 19
Clients
User-facing entrypoints into the pipeline
- API server · API-facing runtime entrypoint
server_v3.py - UI app · Interactive user interface entrypoint
streamlit_app.py
Orchestration
Query planning and execution flow
- Pipeline · Main end-to-end query coordinator
pipeline.py - Interpret · Extracts entities and structured constraints from queries
interpret_bag.py - Strategize · Chooses tools and execution order from query intent
strategize_bag.py - Reranker · Reorders retrieved evidence before synthesis
evidence_reranker.py - Synthesize · Builds the final answer from plan and evidence
synthesize_bag.py
Tooling
Dynamic tools and stage-specific helpers
- Parser · Parses user intent and query structure
query_parser.py - Schema loader · Loads structured schemas for constraints and tools
schema_loader.py - Filter norm · Normalizes filters into downstream-friendly form
filter_normalizer.py - Registry · Discovers and exposes tools at runtime
tool_registry.py - Execution · Invokes registered tools dynamically
execution.py - Tools · Core tool implementations for retrieval and computation
tools.py - Table agent · Specialized helper for table-heavy analysis
table_agent.py - Observation agent · Helper for evidence handling and observations
observation_agent.py
Data & Services
External retrieval, model, and storage services
- Audit DB · Primary audit database access layer
audit_db_client.py - Milvus · Secondary or legacy vector search client
milvus_client.py - LLM client · Abstraction over hosted LLM serving
llm_client.py - Config · Environment-driven service and integration settings
config.py
Connections 22
- API server → Pipeline: requests · API requests enter the main pipeline
- UI app → Pipeline: requests · UI queries enter the main pipeline
- Pipeline → Interpret: stage 1 · Query understanding and constraint extraction
- Interpret → Parser: parse · Interpretation uses query parsing helpers
- Interpret → Schema loader: load schema · Structured schemas guide interpretation
- Interpret → Filter norm: normalize · Filters are normalized for downstream use
- Pipeline → Strategize: stage 2 · Strategy selection and tool planning
- Strategize → Registry: discover tools · Tool availability is resolved at runtime
- Registry → Execution: dispatch · Registry feeds the dynamic execution layer
- Execution → Tools: invoke · Execution calls into tool implementations
- Tools → Audit DB: retrieve · Tools fetch audit evidence from the main database
- Tools → Milvus: vector search · Tools can query vector storage when needed
- Tools → Table agent: assist · Table-heavy tasks are delegated to a helper
- Tools → Observation agent: assist · Evidence and observation handling is delegated to a helper
- Pipeline → Reranker: rerank · Retrieved evidence is reordered before synthesis
- Reranker → LLM client: rank model · Reranking relies on external model serving
- Pipeline → Synthesize: stage 3 · Final response synthesis
- Synthesize → LLM client: generate · LLM client produces the final answer
- Pipeline → Config: settings · Runtime behavior is driven by environment configuration
- Audit DB → Config: connect · Database connectivity is configured externally
- Milvus → Config: connect · Vector store connectivity is configured externally
- LLM client → Config: connect · Model serving endpoints come from configuration