How nvidia/context-aware-rag works
Project Type and Purpose
• This is a Python-based library and microservice suite (“Context Aware RAG”) that sits inside existing data pipelines to provide two HTTP services:
a. A Data Ingestion Service (FastAPI) that accepts documents, runs ingestion workflows, and stores embeddings/graphs.
b. A Data Retrieval Service (FastAPI) that accepts natural-language queries, runs retrieval workflows, and returns results.
• It is complemented by a plugin-style architecture for functions (RAG, summarization, notifications) and tools (storage, LLMs, health checks), and includes built-in observability via OpenTelemetry, Jaeger, and Prometheus.
File Structure
• Top-level dirs:
• service/ – FastAPI app definitions, models, globals.
• src/vss_ctx_rag/ – Core library modules: aiq workflows & registration, aiq_config YAMLs, base classes, context manager, functions, tools, and utils.
• docker/ – Dockerfiles, Compose manifests to spin up the two microservices plus Neo4j, Milvus, Cassandra, OTel collector, Jaeger, Prometheus.
• config/ – Default YAML configs for context and notifications.
• Patterns:
• Hexagonal/plug-in style: clear separation between API layer (service/), domain/workflows (aiq & context_manager), and infrastructure/tools (storage/llm/notification).
• Two separate FastAPI apps (ingestion vs. retrieval), each mounted on independent ports via Docker Compose.
README for Insights
• Confirms a two-service deployment with endpoints /init, /add_doc for ingestion and /init, /call for retrieval.
• External dependencies: Neo4j (graph store), Milvus (vector store), Cassandra (persistence?), OpenTelemetry collector + Jaeger + Prometheus for metrics/tracing.
• Plugin model: users can add custom “functions” and “tools.”