# nikhil-agrawal123/explainable-rag-system architecture

> Architecture diagram of the GitHub repository nikhil-agrawal123/explainable-rag-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nikhil-agrawal123/explainable-rag-system
- Repository: https://github.com/nikhil-agrawal123/explainable-rag-system

## Overview

##### Project type and purpose
- This is a backend-only, explainable Retrieval-Augmented Generation (RAG) pipeline implemented as a FastAPI service.
- It ingests documents, chunks and embeds them, stores embeddings in ChromaDB, retrieves evidence, builds an explainable graph, and generates answers with trust scores.
- It uses local LLMs (Ollama), SpaCy for entity/relation extraction, LangChain for chunking, and LangSmith for tracing.

##### File structure
- app/main.py: FastAPI entry point.
- app/api: API router definitions (endpoints.py) and dependency injection (dependencies.py).
- app/core: configuration loader (config.py) and environment settings.
- app/db: ChromaDB client wrapper.
- app/models: Pydantic schemas for requests/responses.
- app/pipeline: Six pipeline stages (ingestion, decomposition, retrieval, graph construction, generation, scoring) and chunk/metadata utilities.
- app/utils: LLM wrappers (LLM calls to Ollama), visualizer for graph rendering.
- app/static: Swagger CSS customization (UI).
- test/: Unit tests for each pipeline module, database, embedding, graph, retrieval.

##### README for additional insights
- The pipeline is documented in DataForge_nikhil24380.pdf.
- Features highlight local embedding & domain classification, SpaCy extraction, LangChain splitters, ChromaDB persistence, LangSmith tracing, evidence graphs, fully local operation.
- Quickstart shows environment variables for Ollama & LangSmith connections.
- API endpoints list clarifies the surface area of the service.

## Diagram (Mermaid)

```mermaid
flowchart TD
    Client["Client/Consumer"]:::external
    subgraph "API Server"
        A["FastAPI Server<br/>(app/main.py)"]:::internal
        B["API Layer<br/>(endpoints & dependencies)"]:::internal
        C["Pydantic Schemas<br/>(app/models/schemas.py)"]:::internal
    end
    subgraph "Configuration"
        CFG["Configuration Store<br/>(app/core/config.py)"]:::config
    end
    subgraph "Pipeline Orchestrator"
        P1["Stage1: Ingestion<br/>(stage_1_ingestion.py)"]:::internal
        P2["Stage2: Decomposition<br/>(stage_2_decomposition.py)"]:::internal
        P2a["Chunking Utilities<br/>(chuncking.py)"]:::internal
        P2b["Metadata Utilities<br/>(metadata.py)"]:::internal
        P3["Stage3: Retrieval<br/>(stage_3_retrieval.py)"]:::internal
        P4["Stage4: Graph Construction<br/>(stage_4_local_graph.py)"]:::internal
        P5["Stage5: Generation<br/>(stage_5_generation.py)"]:::internal
        P6["Stage6: Scoring<br/>(stage_6_scoring.py)"]:::internal
    end
    subgraph "Data Store"
        DB["ChromaDB Client<br/>(chroma_client.py)"]:::datastore
        FS["File Storage (uploads/, chroma_storage)"]:::datastore
    end
    subgraph "External Services"
        LLM["Local LLM Engine (Ollama)"]:::external
        SPACY["SpaCy Extractor"]:::external
        TRACE["LangSmith Tracing"]:::external
    end
    subgraph "Utilities"
        LLMWRAP["LLM Wrapper<br/>(app/utils/llm.py)"]:::internal
        VIS["Visualizer<br/>(app/utils/visualizer.py)"]:::internal
    end

    Client -->|POST /full_pipeline| A
    A -->|"routes requests"| B
    B -->|"uses schemas"| C
    B -->|"invokes pipeline"| P1
    CFG -.-> A
    CFG -.-> P1

    P1 -->|"file ingestion"| P2
    P2 -->|"decompose"| P2a
    P2 -->|"extract metadata"| P2b
    P2a -->|"embed(text)"| DB
    P2b -->|"extract metadata"| DB
    P3 <-->|"query embeddings"| DB
    P3 -->|"domain classification"| SPACY
    P3 -->|"entity extraction"| SPACY
    P3 -->|"LLM calls"| LLMWRAP
    LLMWRAP -->|"infer"| LLM
    P4 -->|"build graph"| VIS
    P5 -->|"generate answer"| LLMWRAP
    P6 -->|"assign trust scores"| TRACE
    P6 -->|"return results"| B
    A -->|"serve static CSS"| FS
    B -->|"response"| Client

    click A "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/main.py"
    click B "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/api/endpoints.py"
    click B "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/api/dependencies.py"
    click C "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/models/schemas.py"
    click CFG "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/core/config.py"
    click DB "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/db/chroma_client.py"
    click P1 "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/stage_1_ingestion.py"
    click P2 "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/stage_2_decomposition.py"
    click P2a "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/chuncking.py"
    click P2b "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/metadata.py"
    click P3 "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/stage_3_retrieval.py"
    click P4 "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/stage_4_local_graph.py"
    click P5 "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/stage_5_generation.py"
    click P6 "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/pipeline/stage_6_scoring.py"
    click LLMWRAP "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/utils/llm.py"
    click VIS "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/utils/visualizer.py"
    click FS "https://github.com/nikhil-agrawal123/explainable-rag-system/blob/master/app/static/swagger-dark.css"

    classDef internal fill:#cce5ff,stroke:#004085,color:#000
    classDef datastore fill:#d4edda,stroke:#155724,color:#000
    classDef external fill:#ffeeba,stroke:#856404,color:#000
    classDef config stroke-dasharray: 5 5,fill:#f8f9fa,stroke:#6c757d,color:#000
```
