# amandeva/agentic-rag architecture

> Architecture diagram of the GitHub repository amandeva/agentic-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-03-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/amandeva/agentic-rag
- Repository: https://github.com/amandeva/agentic-rag

## Overview

This repo is the **Python backend for a production Agentic RAG system** deployed on **AWS EC2**. It exposes a **FastAPI** service (`/ask`, `/ask_async`) that runs a **LangGraph-orchestrated hybrid retrieval + agentic validation + generation** pipeline, with:
- **Hybrid retrieval**: Pinecone vector search + BM25 keyword search
- **Re-ranking**: Flashrank cross-encoder
- **Agentic filtering**: Llama-3 via Groq grades/filters retrieved chunks
- **Final answer**: Mistral via OpenRouter using *filtered* context
- **Caching**: Redis
- **Async**: AWS SQS + separate `worker.py` process

Top-level files indicate a small-but-complete backend service:

- `app.py`
**FastAPI entrypoint**. Owns HTTP endpoints (`/ask`, `/ask_async`), request validation, cache checks, calling the RAG agent, and (for async) SQS enqueue.
- `rag_agent.py`
- `worker.py`
**Background worker** that consumes SQS messages and runs the same (or similar) pipeline, then returns/persists results (how results are returned isn’t shown in tree—capture as “response channel TBD/implementation-specific”).
- `upload_to_pinecone.py`
**Offline/ingestion job** that builds embeddings (BAAI/bge-large-en-v1.5) and upserts into Pinecone index.
- `requirements.txt`
Defines runtime dependencies; use it only to confirm technologies (FastAPI, redis client, boto3, pinecone client, langgraph, etc.).

##### Client / Product surface (external to this repo, but essential)
- **User**
- **MERN Chat UI (React)**
- **Node/Express API (JWT auth, chat history)**
- **MongoDB Atlas** (user + chat logs)
- **Axios/Proxy** from Express → FastAPI on EC2

##### Core backend runtime (this repo on EC2)
- **FastAPI app process** (Uvicorn)
- **RAG Agent (LangGraph)** as a subcomponent called by FastAPI
- **Redis (local)** cache with TTL (24h)
- **SQS producer** (inside FastAPI) for `/ask_async`
- **SQS worker process** (`worker.py`) as a separate runtime unit on the same EC2 host

##### Retrieval / ranking / model providers (external services)
- **Pinecone serverless index** (`cogni-compliance`)
- **Flashrank cross-encoder reranker** (in-process library)
- **Groq API** (Llama-3 grading)
- **OpenRouter API** (Mistral generation)

##### Ingestion / indexing (offline path)
- **Document source** (legal/regulatory corpus; not in repo)
- `upload_to_pinecone.py` job
- **Embedding model** (BAAI/bge-large-en-v1.5)
- **Pinecone upsert**

##### FastAPI → Redis GET (cache key: prompt + params)
- Cache hit → return answer
##### RAG Agent → Hybrid Retrieval
- Pinecone query (vector)
- BM25 query (keyword)
- Merge candidates

##### Worker writes result to a result sink
- If implemented: Redis (job_id→result), DB, or callback endpoint

##### Architectural patterns to highlight (call these out visually)
- **C4-style separation**: Client apps vs backend service vs managed services.
- **Agentic RAG**: grading/validation step *between* reranking and generation.
- **Hybrid retrieval**: two retrieval sources merged (semantic + keyword).
- **Async job pattern**: HTTP enqueue → SQS → worker consumer.
- **Cache-aside**: Redis in front of expensive inference.
- **Multi-provider LLM routing**: Groq for grading, OpenRouter for final answer.

##### System Context (Level 0/1)
User → MERN → FastAPI(EC2) → Pinecone/Redis/SQS/Groq/OpenRouter, plus MongoDB Atlas.
##### Container/Deployment (Level 2)
EC2 instance boundary containing:
- Uvicorn/FastAPI process
- worker.py process
- Redis (local service)
Do one sequence for `/ask` and one for `/ask_async` (with SQS + worker).

- EC2 instance type: `t3.large`, OS `Ubuntu 24.04 LTS`
- API port: `8000`
- Cache TTL: `24h`
- Pinecone index: `cogni-compliance`
- Embeddings: `BAAI/bge-large-en-v1.5`
- Grader model: `Llama-3 (Groq)`
- Generator model: `Mistral (OpenRouter)`
- Re-ranking: `Flashrank cross-encoder`
- Async processing: `AWS SQS`

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% =========================================================
    %% Combined Diagram: System Context + Container/Deployment + Critical Flows
    %% (Merged into one flowchart to ensure single-diagram Mermaid validity)
    %% =========================================================

    %% =======================
    %% Diagram 1: System Context (C4 L0/L1)
    %% =======================

    User["User"]:::external

    subgraph D1["Diagram 1 — System Context (C4 L0/L1)"]
        direction TB

        subgraph ExtSurf["External Product Surface (Not in this repo)"]
            direction TB
            MERN_UI["MERN Chat UI (React)"]:::compute
            MERN_API["Node/Express API\n(JWT auth, chat history)"]:::compute
            Mongo["MongoDB Atlas\n(users + chat logs)"]:::datastore
            MERN_UI -->|"HTTPS\nchat UI"| MERN_API
            MERN_API -->|"MongoDB driver\nread/write"| Mongo
        end

        subgraph EC2A["AWS EC2 (t3.large) • Ubuntu 24.04 LTS"]
            direction TB
            FastAPI["FastAPI API Server\n(app.py)\nUvicorn • Port 8000\nEndpoints: /ask, /ask_async"]:::compute
            RAG["RAG Agent / LangGraph Orchestrator\n(rag_agent.py)\nHybrid Retrieval → Rerank → Grade → Generate"]:::compute
            Redis[("Redis (local)\nCache-aside • TTL 24h")]:::datastore
            Worker["Async SQS Worker\n(worker.py)\nConsumes jobs • Runs RAG pipeline"]:::compute
        end

        subgraph Managed["External Managed Services"]
            direction TB
            Pinecone[("Pinecone Serverless Index\ncogni-compliance")]:::datastore
            BM25[("BM25 Keyword Index / Search\n(local/in-process)")]:::datastore
            SQS["AWS SQS Queue"]:::queue
            Groq["Groq API\nLlama-3 (grading)"]:::ml
            OpenRouter["OpenRouter API\nMistral (generation)"]:::ml
            ResultSink[("Result Store / Response Channel\n(implementation-specific)")]:::datastore
        end

        %% Upstream -> FastAPI
        MERN_API -->|"HTTPS/REST (Axios proxy)\nPOST /ask, /ask_async"| FastAPI

        %% FastAPI -> cache -> RAG
        FastAPI -->|"Redis TCP\nGET cache (prompt+params)"| Redis
        Redis -->|"cache_hit → answer"| FastAPI
        FastAPI -->|"cache_miss → invoke"| RAG

        %% RAG hybrid retrieval + generation
        RAG -->|"Vector query\n(question embedding)"| Pinecone
        RAG -->|"Keyword query\nBM25 search"| BM25
        RAG -->|"Rerank Top10\nFlashrank cross-encoder"| RAG
        RAG -->|"Grade/filter docs\nLlama-3"| Groq
        Groq -->|"relevance scores / filtered set"| RAG
        RAG -->|"Generate grounded answer\nMistral"| OpenRouter
        OpenRouter -->|"final answer"| RAG
        RAG -->|"return answer"| FastAPI
        FastAPI -->|"Redis TCP\nSET cache (TTL 24h)"| Redis
        FastAPI -->|"HTTPS/REST\nanswer"| MERN_API
        MERN_API -->|"HTTPS\nrender answer"| MERN_UI
        MERN_UI -->|"UX"| User

        %% Async path
        FastAPI -->|"SQS SendMessage\n(job payload: question, params)"| SQS
        Worker -->|"SQS LongPoll/ReceiveMessage"| SQS
        Worker -->|"run same RAG pipeline"| RAG
        Worker -->|"persist result\n(job_id → result)"| ResultSink
        MERN_API -->|"poll/webhook (TBD)\nfetch result by job_id"| ResultSink

        %% Offline ingestion (context-level)
        subgraph IngestLane["Offline Ingestion / Indexing"]
            direction TB
            Corpus["Document Source\n(legal/regulatory corpus)"]:::external
            Ingest["Offline Ingestion Job\n(upload_to_pinecone.py)\nChunk/Parse → Embed → Upsert"]:::compute
            Embed["Embedding Model\nBAAI/bge-large-en-v1.5"]:::ml
            Corpus -->|"raw docs"| Ingest
            Ingest -->|"chunked text"| Embed
            Embed -->|"vectors"| Ingest
            Ingest -->|"upsert vectors + metadata"| Pinecone
        end

        %% Callouts
        Notes1["Callouts:\n• Cache TTL: 24h\n• Pinecone index: cogni-compliance\n• Re-ranking: Flashrank cross-encoder\n• Grader: Llama-3 (Groq)\n• Generator: Mistral (OpenRouter)\n• Async: AWS SQS"]:::note
        Notes1 -.-> FastAPI
        Notes1 -.-> RAG
        Notes1 -.-> SQS
    end

    %% =======================
    %% Diagram 2: Container / Deployment (C4 L2)
    %% =======================

    subgraph D2["Diagram 2 — Container / Deployment (C4 L2)"]
        direction TB

        UpUser["User"]:::external
        UpMERN["MERN (React + Node/Express)\n(External)"]:::compute
        UpMongo[("MongoDB Atlas\n(External)")]:::datastore

        UpUser -->|"HTTPS"| UpMERN
        UpMERN -->|"MongoDB driver"| UpMongo

        subgraph EC2B["AWS EC2 Instance Boundary\n(t3.large • Ubuntu 24.04 LTS)"]
            direction TB
            subgraph RepoSvc["Python Backend Service (this repo)"]
                direction TB
                ProcFastAPI["Process: Uvicorn/FastAPI\n(app.py)\nPort 8000\n/ask, /ask_async"]:::compute
                ProcWorker["Process: SQS Worker\n(worker.py)"]:::compute
                ModuleRAG["Module: LangGraph RAG Agent\n(rag_agent.py)"]:::compute
                LocalRedis[("Redis (local)\nCache-aside TTL 24h")]:::datastore

                ProcFastAPI -->|"in-process call"| ModuleRAG
                ProcWorker -->|"in-process call"| ModuleRAG

                ProcFastAPI -->|"Redis TCP\nGET/SET cache"| LocalRedis
                ProcWorker -->|"Redis TCP\n(optional: job result cache)"| LocalRedis
            end
        end

        subgraph Providers["AWS Managed / External Providers"]
            direction TB
            DepSQS["AWS SQS Queue"]:::queue
            DepPinecone[("Pinecone Serverless\nIndex: cogni-compliance")]:::datastore
            DepBM25[("BM25 Keyword Index/Search\n(local/in-process)")]:::datastore
            DepGroq["Groq API\nLlama-3 grader"]:::ml
            DepOpenRouter["OpenRouter API\nMistral generator"]:::ml
            DepResult[("Result Store / Response Channel\n(implementation-specific)")]:::datastore
        end

        UpMERN -->|"HTTPS/REST\nproxy to :8000"| ProcFastAPI

        %% Sync dependencies from RAG
        ModuleRAG -->|"Vector query"| DepPinecone
        ModuleRAG -->|"Keyword query"| DepBM25
        ModuleRAG -->|"Grade docs"| DepGroq
        ModuleRAG -->|"Generate answer"| DepOpenRouter

        %% Async queueing
        ProcFastAPI -->|"boto3 SQS\nSendMessage"| DepSQS
        ProcWorker -->|"boto3 SQS\nLongPoll/ReceiveMessage"| DepSQS
        ProcWorker -->|"persist result"| DepResult
        UpMERN -->|"poll/webhook (TBD)"| DepResult

        DepNotes["Deployment Notes:\n• FastAPI listens on 8000\n• Separate worker process on same EC2\n• Redis local service\n• External: Pinecone, SQS, Groq, OpenRouter"]:::note
        DepNotes -.-> ProcFastAPI
        DepNotes -.-> ProcWorker
        DepNotes -.-> LocalRedis
    end

    %% =======================
    %% Diagram 3: Critical Sequences / Pipelines (rendered as flowchart steps)
    %% =======================

    subgraph D3["Diagram 3 — Critical Sequences / Pipelines (Flow Representation)"]
        direction TB

        subgraph Sync["Sync: POST /ask"]
            direction TB
            S1["1) MERN → FastAPI: POST /ask (prompt, params)"]:::compute
            S2["2) FastAPI → Redis: GET cache (key=prompt+params)"]:::datastore
            S3["3) Cache hit → return answer"]:::note
            S4["4) Cache miss → invoke RAG Agent (LangGraph)"]:::compute
            S5["5) Hybrid retrieval: Pinecone + BM25"]:::datastore
            S6["6) Rerank Top10: Flashrank cross-encoder"]:::ml
            S7["7) Grade/filter: Llama-3 via Groq"]:::ml
            S8["8) Generate answer: Mistral via OpenRouter"]:::ml
            S9["9) FastAPI → Redis: SET cache (TTL 24h)"]:::datastore
            S10["10) FastAPI → MERN: 200 OK (answer)"]:::compute

            S1 --> S2
            S2 -->|"hit"| S3 --> S10
            S2 -->|"miss"| S4 --> S5 --> S6 --> S7 --> S8 --> S9 --> S10
        end

        subgraph Async["Async: POST /ask_async (enqueue → SQS → worker)"]
            direction TB
            A1["1) MERN → FastAPI: POST /ask_async (prompt, params)"]:::compute
            A2["2) Optional: FastAPI → Redis GET cache"]:::datastore
            A3["3) FastAPI → SQS: SendMessage (job payload)"]:::queue
            A4["4) FastAPI → MERN: 202 Accepted (job_id)"]:::compute
            A5["5) Worker ← SQS: LongPoll/ReceiveMessage"]:::queue
            A6["6) Worker → RAG Agent: run same pipeline"]:::compute
            A7["7) Worker → Result Store: persist (job_id → answer)"]:::datastore
            A8["8) MERN ↔ Result Store: poll/webhook (TBD)"]:::note

            A1 --> A2 --> A3 --> A4
            A5 --> A6 --> A7 --> A8
        end
    end

    %% Click events (repo anchors) — keep all original mappings
    click FastAPI "https://github.com/amandeva/agentic-rag/blob/main/app.py"
    click RAG "https://github.com/amandeva/agentic-rag/blob/main/rag_agent.py"
    click Worker "https://github.com/amandeva/agentic-rag/blob/main/worker.py"
    click Ingest "https://github.com/amandeva/agentic-rag/blob/main/upload_to_pinecone.py"

    click ProcFastAPI "https://github.com/amandeva/agentic-rag/blob/main/app.py"
    click ModuleRAG "https://github.com/amandeva/agentic-rag/blob/main/rag_agent.py"
    click ProcWorker "https://github.com/amandeva/agentic-rag/blob/main/worker.py"
    click DepNotes "https://github.com/amandeva/agentic-rag/blob/main/requirements.txt"

    %% Styles
    classDef compute fill:#d7ebff,stroke:#2b6cb0,stroke-width:1.2,color:#0b2545
    classDef datastore fill:#d9f7d9,stroke:#2f855a,stroke-width:1.2,color:#123524
    classDef ml fill:#ffe6bf,stroke:#c05621,stroke-width:1.2,color:#4a2c0a
    classDef queue fill:#eadcff,stroke:#6b46c1,stroke-width:1.2,color:#2a1a4a
    classDef external fill:#f2f2f2,stroke:#4a5568,stroke-width:1.2,color:#1a202c,stroke-dasharray: 4 3
    classDef note fill:#fff7cc,stroke:#b7791f,stroke-width:1.1,color:#3b2f0b
```
