How amandeva/agentic-rag works
This repo is the Python backend for a production Agentic RAG system deployed on AWS EC2. It exposes a FastAPI service (/ask, /ask_async) that runs a LangGraph-orchestrated hybrid retrieval + agentic validation + generation pipeline, with:
• Hybrid retrieval: Pinecone vector search + BM25 keyword search
• Re-ranking: Flashrank cross-encoder
• Agentic filtering: Llama-3 via Groq grades/filters retrieved chunks
• Final answer: Mistral via OpenRouter using *filtered* context
• Caching: Redis
• Async: AWS SQS + separate worker.py process
Top-level files indicate a small-but-complete backend service:
• app.py
FastAPI entrypoint. Owns HTTP endpoints (/ask, /ask_async), request validation, cache checks, calling the RAG agent, and (for async) SQS enqueue.
• rag_agent.py
• worker.py
Background worker that consumes SQS messages and runs the same (or similar) pipeline, then returns/persists results (how results are returned isn’t shown in tree—capture as “response channel TBD/implementation-specific”).
• upload_to_pinecone.py
Offline/ingestion job that builds embeddings (BAAI/bge-large-en-v1.5) and upserts into Pinecone index.
• requirements.txt
Defines runtime dependencies; use it only to confirm technologies (FastAPI, redis client, boto3, pinecone client, langgraph, etc.).
Client / Product surface (external to this repo, but essential)
• User
• MERN Chat UI (React)
• Node/Express API (JWT auth, chat history)
• MongoDB Atlas (user + chat logs)
• Axios/Proxy from Express → FastAPI on EC2
Core backend runtime (this repo on EC2)
• FastAPI app process (Uvicorn)
• RAG Agent (LangGraph) as a subcomponent called by FastAPI
• Redis (local) cache with TTL (24h)
• SQS producer (inside FastAPI) for /ask_async
• SQS worker process (worker.py) as a separate runtime unit on the same EC2 host
Retrieval / ranking / model providers (external services)
• Pinecone serverless index (cogni-compliance)
• Flashrank cross-encoder reranker (in-process library)
• Groq API (Llama-3 grading)
• OpenRouter API (Mistral generation)
Ingestion / indexing (offline path)
• Document source (legal/regulatory corpus; not in repo)
• upload_to_pinecone.py job
• Embedding model (BAAI/bge-large-en-v1.5)
• Pinecone upsert
FastAPI → Redis GET (cache key: prompt + params)
• Cache hit → return answer
RAG Agent → Hybrid Retrieval
• Pinecone query (vector)
• BM25 query (keyword)
• Merge candidates
Worker writes result to a result sink
• If implemented: Redis (job_id→result), DB, or callback endpoint
Architectural patterns to highlight (call these out visually)
• C4-style separation: Client apps vs backend service vs managed services.
• Agentic RAG: grading/validation step *between* reranking and generation.
• Hybrid retrieval: two retrieval sources merged (semantic + keyword).
• Async job pattern: HTTP enqueue → SQS → worker consumer.
• Cache-aside: Redis in front of expensive inference.
• Multi-provider LLM routing: Groq for grading, OpenRouter for final answer.
System Context (Level 0/1)
User → MERN → FastAPI(EC2) → Pinecone/Redis/SQS/Groq/OpenRouter, plus MongoDB Atlas.
Container/Deployment (Level 2)
EC2 instance boundary containing:
• Uvicorn/FastAPI process
• worker.py process
• Redis (local service)
Do one sequence for /ask and one for /ask_async (with SQS + worker).
• EC2 instance type: t3.large, OS Ubuntu 24.04 LTS
• API port: 8000
• Cache TTL: 24h
• Pinecone index: cogni-compliance
• Embeddings: BAAI/bge-large-en-v1.5
• Grader model: Llama-3 (Groq)
• Generator model: Mistral (OpenRouter)
• Re-ranking: Flashrank cross-encoder
• Async processing: AWS SQS