# nvidia/context-aware-rag architecture

> Architecture diagram of the GitHub repository nvidia/context-aware-rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-13). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/nvidia/context-aware-rag
- Repository: https://github.com/nvidia/context-aware-rag

## Overview

##### Project Type and Purpose
- This is a Python-based library and microservice suite (“Context Aware RAG”) that sits inside existing data pipelines to provide two HTTP services:
a. A Data Ingestion Service (FastAPI) that accepts documents, runs ingestion workflows, and stores embeddings/graphs.
b. A Data Retrieval Service (FastAPI) that accepts natural-language queries, runs retrieval workflows, and returns results.
- It is complemented by a plugin-style architecture for functions (RAG, summarization, notifications) and tools (storage, LLMs, health checks), and includes built-in observability via OpenTelemetry, Jaeger, and Prometheus.

##### File Structure
- Top-level dirs:
- service/ – FastAPI app definitions, models, globals.
- src/vss_ctx_rag/ – Core library modules: aiq workflows & registration, aiq_config YAMLs, base classes, context manager, functions, tools, and utils.
- docker/ – Dockerfiles, Compose manifests to spin up the two microservices plus Neo4j, Milvus, Cassandra, OTel collector, Jaeger, Prometheus.
- config/ – Default YAML configs for context and notifications.
- Patterns:
- Hexagonal/plug-in style: clear separation between API layer (service/), domain/workflows (aiq & context_manager), and infrastructure/tools (storage/llm/notification).
- Two separate FastAPI apps (ingestion vs. retrieval), each mounted on independent ports via Docker Compose.

##### README for Insights
- Confirms a two-service deployment with endpoints `/init`, `/add_doc` for ingestion and `/init`, `/call` for retrieval.
- External dependencies: Neo4j (graph store), Milvus (vector store), Cassandra (persistence?), OpenTelemetry collector + Jaeger + Prometheus for metrics/tracing.
- Plugin model: users can add custom “functions” and “tools.”

## Diagram (Mermaid)

```mermaid
flowchart TD
    Client["Client / Upstream Workflow"]:::external

    subgraph "API Layer"
        Ingest["Ingestion Service (FastAPI)"]:::api
        Retrieve["Retrieval Service (FastAPI)"]:::api
    end

    subgraph "Core Library / Execution Layer"
        CM["Context Manager / Orchestrator"]:::core
        subgraph "AIQ Workflows"
            AIQIn["Ingestion Workflow"]:::core
            AIQRet["Retrieval Workflow"]:::core
        end
        subgraph "Function Plugins"
            RAGGraph["RAG Graph Functions"]:::core
            RAGVector["RAG Vector Function"]:::core
            ChatFunc["Chat Function"]:::core
            Summ["Summarization"]:::core
            NotifyFunc["Notification Function"]:::core
        end
        subgraph "Tool Plugins"
            Storage["Storage Drivers"]:::core
            LLM["LLM Handler"]:::core
            NotifyTool["Notification Tools"]:::core
            Health["Health Checks"]:::core
        end
    end

    subgraph "Data Stores & External Systems"
        Milvus["Milvus Vector Store"]:::db
        Neo4j["Neo4j Graph Store"]:::db
        Cass["Cassandra Store"]:::db
        LLMProv["LLM Providers"]:::external
        NotifyEP["Notification Endpoints"]:::external
    end

    subgraph "Observability Stack"
        OTel["OTel Collector"]:::obs
        Jaeger["Jaeger (Traces)"]:::obs
        Prom["Prometheus (Metrics)"]:::obs
    end

    Client -->|"/init, /add_doc"| Ingest
    Client -->|"/init, /call"| Retrieve
    Ingest --> CM
    Retrieve --> CM
    CM --> AIQIn
    CM --> AIQRet
    AIQIn --> RAGGraph
    AIQIn --> RAGVector
    AIQIn --> Summ
    AIQIn --> NotifyFunc
    AIQRet --> RAGGraph
    AIQRet --> RAGVector
    AIQRet --> Summ
    AIQRet --> NotifyFunc
    RAGGraph --> Storage
    RAGVector --> Storage
    ChatFunc --> LLMProv
    Summ --> LLMProv
    NotifyFunc --> NotifyTool
    Storage --> Milvus
    Storage --> Neo4j
    Storage --> Cass
    LLM --> LLMProv
    NotifyTool --> NotifyEP
    Health -.-> Ingest
    Health -.-> Retrieve
    Ingest -->|traces/metrics| OTel
    Retrieve -->|traces/metrics| OTel
    OTel --> Jaeger
    OTel --> Prom

    click Ingest "https://github.com/nvidia/context-aware-rag/blob/main/service/service.py"
    click Retrieve "https://github.com/nvidia/context-aware-rag/blob/main/service/service.py"
    click CM "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/context_manager/context_manager.py"
    click AIQIn "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/aiq/register_in.py"
    click AIQRet "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/aiq/register_ret.py"
    click RAGGraph "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/functions/rag/graph_rag/graph_extraction.py"
    click RAGVector "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/functions/rag/vector_rag/vector_retrieval_func.py"
    click ChatFunc "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/functions/rag/chat_function.py"
    click Summ "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/functions/summarization/batch.py"
    click NotifyFunc "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/functions/notification/notifier.py"
    click Storage "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/tools/storage/milvus_db.py"
    click LLM "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/tools/llm/llm_handler.py"
    click NotifyTool "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/tools/notification/alert_sse_tool.py"
    click Health "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/tools/health/rag_health.py"
    click OTel "https://github.com/nvidia/context-aware-rag/blob/main/src/vss_ctx_rag/utils/otel.py"
    click Prom "https://github.com/nvidia/context-aware-rag/blob/main/docker/configs/prometheus.yml"

    classDef api fill:#ADD8E6,stroke:#333,stroke-width:1px;
    classDef core fill:#90EE90,stroke:#333,stroke-width:1px;
    classDef db fill:#FFA500,stroke:#333,stroke-width:1px;
    classDef obs fill:#D3D3D3,stroke:#333,stroke-width:1px;
    classDef external fill:#F0E68C,stroke:#333,stroke-width:1px;
```
