How 0xankitsingh/corrective-rag works
1. What this repo is
- This is a Python command-line RAG system, not a web service.
- Its core purpose is to answer questions against a Bedrock Knowledge Base, using a “corrective” retrieval loop when the first retrieval is weak.
2. Top-level architecture
- The repository splits into three main runtime paths:
- Knowledge base provisioning for onboarding documents.
- Interactive/query-time agents for answering questions.
- Support tooling for retrieval scoring and web fallback.
- The main boundary is between offline setup and online inference.
3. CLI and application entry
- src/rag/main.py is the user-facing entry point for the agent CLI.
- It supports interactive mode, one-shot queries, switching between a simple agent and a corrective agent, and selecting the LLM provider/model from the command line or environment.
4. Configuration layer
- src/rag/config.py centralizes environment-driven settings using Pydantic-style config.
- This layer is the control plane for the whole repo: Bedrock Knowledge Base ID, AWS region, Tavily key, model IDs, provider selection, and thresholds such as retrieval and relevance cutoffs.
- .env.example and the README indicate the system is designed to be configured externally rather than hard-coded.
5. AWS client boundary
- src/rag/clients.py is the integration point for model and AWS service clients.
- The architecture depends on pluggable LLM backends, with Bedrock as the default and alternatives such as Anthropic, OpenAI-compatible endpoints, and Ollama.
- This means the agent logic is separated from the specific model provider implementation.
6. Agent subsystem
- src/rag/agents/simple.py appears to be the baseline RAG flow: retrieve context and answer.
- src/rag/agents/corrective.py adds the corrective decision loop: retrieve, evaluate chunk relevance, and optionally fall back to web search.
- src/rag/agents/prompts.py holds system prompts, keeping prompting concerns separate from orchestration code.
7. Retrieval and correction flow
- The primary data flow is:
1. User query enters the CLI.
2. The agent queries the Bedrock Knowledge Base.
3. Retrieved chunks are scored for relevance.
4. If the score is acceptable, the answer is generated from KB context.
5. If not, web search is used to supplement or replace weak retrieval.
- This is the central architectural pattern of the repo.
8. Relevance evaluation tool
- src/rag/tools/relevance.py implements chunk relevance checking using RAGAS, specifically a context-precision style metric without a reference answer.
- This tool acts as the “corrective” gate in the system: it decides whether KB retrieval is good enough to trust.
9. Web fallback tool
- src/rag/tools/search.py integrates Tavily for external web search.
- This is the fallback knowledge source when internal retrieval is insufficient, creating a second information boundary outside AWS.
- The README makes clear this is only used in the corrective path, not the baseline simple agent.
10. Knowledge base provisioning path
- src/rag/knowledge_base/provisioner.py is an offline bootstrapper for the ingestion pipeline.
- It creates the S3 bucket, uploads documents from onboarding_files/, builds the Bedrock Knowledge Base, and triggers ingestion.
- It also writes the resulting KNOWLEDGE_BASE_ID back into .env, tying provisioning output directly into runtime configuration.
11. Content and data sources
- onboarding_files/ contains the sample source documents that become the knowledge base corpus.
- The repo therefore has a clear document lifecycle: local files → S3 upload → Bedrock Knowledge Base indexing → retrieval at query time.
12. External services and infrastructure
- The architecture depends materially on AWS Bedrock, S3, and OpenSearch Serverless for KB storage/indexing/retrieval.
- It also requires an LLM provider for generation/evaluation and Tavily for fallback search.
- The default runtime model is Claude via Bedrock, but the system is intentionally provider-agnostic.
13. Build and execution tooling
- pyproject.toml and uv.lock show the project is managed with UV and follows modern PEP 621 packaging.
- The README suggests two operational commands matter most: provisioning the KB and running the agent.
- That separation reinforces the architecture: setup once, query many times.
14. How to draw the diagram
- A good diagram should show:
- User/CLI at the edge.
- Agent orchestration in the middle.
- Bedrock KB retrieval as the primary knowledge path.
- RAGAS relevance evaluation as the decision gate.
- Tavily web search as the fallback path.
- Provisioner + S3 + Bedrock KB ingestion as the offline ingestion pipeline.
- The key design idea is a retrieval-then-validate-then-fallback loop, with model/provider choice kept behind a client boundary.
Main entry points: clients.py Client layer · provisioner.py Provisioner · corrective.py Corrective agent · main.py CLI entry
Components 16
Online inference
Query-time CLI, agents, and supporting tools.
- CLI entry · User-facing command line entry for interactive and one-shot question answering.
src/rag/main.py - Settings · Environment-driven configuration for models, AWS, thresholds, and fallback search.
src/rag/config.py - Client layer · Factory and boundary for model and AWS service clients.
src/rag/clients.py - Simple agent · Baseline retrieve-and-answer path over the knowledge base.
src/rag/agents/simple.py - Corrective agent · Retrieval flow that scores context and falls back when evidence is weak.
src/rag/agents/corrective.py - Prompts · System prompts and instruction templates for the agents.
src/rag/agents/prompts.py - Relevance check · RAGAS-based chunk relevance scoring used as the corrective gate.
src/rag/tools/relevance.py - Web search · Tavily-backed external search used when retrieval quality is insufficient.
src/rag/tools/search.py
Offline setup
Knowledge base provisioning and corpus bootstrapping.
- Provisioner · Offline setup flow that creates and ingests the knowledge base.
src/rag/knowledge_base/provisioner.py - Onboarding docs · Source PDFs uploaded into the knowledge base during provisioning.
onboarding_files
External systems
AWS services and third-party providers used by the repo.
- Bedrock KB · Primary retrieval store queried at runtime and created during provisioning.
- S3 bucket · Staging storage for uploaded onboarding documents.
- OpenSearch · Backend index used by the knowledge base for retrieval.
- LLM provider · Pluggable generation backend such as Bedrock, Anthropic, OpenAI-compatible endpoints, or Ollama.
- Tavily API · Third-party search service used only on the corrective fallback path.
- Bedrock service · AWS service boundary for knowledge base and model access.
Connections 21
- CLI entry → Settings: loads · CLI reads runtime settings and thresholds from environment-backed config.
- CLI entry → Client layer: builds · Entry point initializes provider and AWS clients before dispatching agents.
- CLI entry → Simple agent: selects · CLI can run the baseline retrieval flow.
- CLI entry → Corrective agent: selects · CLI can run the corrective retrieval loop.
- Simple agent → Client layer: uses · Simple agent depends on retrieval and model clients.
- Corrective agent → Client layer: uses · Corrective agent depends on retrieval, evaluation, and model clients.
- Simple agent → Prompts: prompts · Baseline answer generation is guided by shared prompt templates.
- Corrective agent → Prompts: prompts · Corrective orchestration uses shared system instructions.
- Corrective agent → Relevance check: checks · Retrieved chunks are scored before answering.
- Corrective agent → Web search: falls back · Weak retrieval triggers external search supplementation.
- Client layer → Bedrock KB: queries · Retrieval client talks to the knowledge base.
- Client layer → LLM provider: generates · Model client is abstracted behind a provider boundary.
- Client layer → Bedrock service: integrates · Default path targets AWS Bedrock services.
- Provisioner → Onboarding docs: uploads · Offline bootstrap takes local documents as input.
- Provisioner → S3 bucket: stages · Documents are first placed in S3.
- Provisioner → Bedrock KB: creates · Provisioner creates the knowledge base and records its id.
- Provisioner → OpenSearch: indexes · Knowledge base setup includes the retrieval index backend.
- Provisioner → Settings: writes · Provisioning updates runtime configuration with the new KB id.
- Bedrock KB → OpenSearch: backed by · Knowledge base retrieval depends on the search index.
- Web search → Tavily API: calls · Fallback search queries the external Tavily API.
- LLM provider → Bedrock service: may use · One supported provider is Bedrock-hosted generation.