How nvidia-ai-blueprints/ai-virtual-assistant works
Main Components
• Frontend/UI Service – Although the UI source code is not included, it is the customer-facing layer that interacts with the API gateway.
• API Gateway – Acts as the single entry point for external API calls and routes them to the underlying services.
• Agent Service – The “agent” microservice (under src/agent) handles the primary logic to process user queries, retrieve context, and generate intelligent responses.
• Analytics Service – (Under src/analytics) Provides conversation summaries, sentiment analysis, and other operational metrics.
• Ingestion Service – (Under src/ingest_service) Responsible for the ingestion of structured (CSV) and unstructured (PDF) data.
• Retrievers – There are two types:
• Structured Data Retriever – Handles structured data via PostgreSQL.
• Unstructured Data Retriever – Uses tools like Milvus (vector database) and NeMo Retriever for PDF/FAQ ingestion.
• Additional Support Microservices – Such as the “common” utilities, the “retrievers” package (with both unstructured and structured variants), and caching mechanisms (local_cache, redis_client).
• Databases and External Data Stores:
• PostgreSQL – For structured data (order histories, customer profiles).
• Milvus – For vector accelerated unstructured data search.
• Redis – Utilized for caching session data and intermediate storage.
Deployment and Orchestration Layers
• Deployment Artifacts – The “deploy” folder shows both Docker Compose and Helm chart configurations, indicating that containerized microservices are the de facto units.
• Helm Charts – There are charts for agent-services, aiva-ui (the UI service), analytics, API gateway, cache, etc. It is useful to show that each microservice is containerized and may have its own deployment, service, persistent volumes, secrets, and configuration maps.
• Docker Compose – In single node deployment mode, the Docker Compose file coordinates the startup for all containers (including databases, caches, and microservices).
Relationships and Interactions
• The UI sends customer queries to the API gateway.
• The API gateway forwards the query to the Agent service.
• The Agent service interacts with both the Structured and Unstructured retrievers:
• The Structured Retriever queries the PostgreSQL database (via Postgres clients).
• The Unstructured Retriever queries Milvus after fetching document embeddings (involving NeMo Retriever NIM for embedding generation).
• The Agent also invokes the LLM NIM endpoints (either NVIDIA hosted or self-hosted) to generate responses based on the retrieved context.
• The Analytics service gathers conversation data (from logs or via direct API calls) and optionally processes sentiment analysis and summarization.
• Caching is used by various services (via the redis clients and local cache modules) to improve performance.
• The Ingestion Service, triggered in either a standalone process or via scheduled jobs/cronjobs (as seen in helm templates), loads the sample CSV and PDF files into the respective databases.
Architectural Patterns and Technologies
• Microservices Architecture: Each component such as agent, analytics, API gateway, and retrievers is built as an independent, containerized service.
• Container Orchestration: The project supports both Docker Compose and Kubernetes (via Helm charts) for deployment.
• Dataflow Pattern: A typical query flows from the UI → API gateway → Agent → (retrievers and LLM inference endpoints) → response, with side-channel data collection for analytics and feedback.
• Component Reuse: Common libraries and utilities are encapsulated in the “src/common” directory.
• External Integrations: The architecture explicitly integrates with NVIDIA’s NIM services, and leverages containerized databases and vector storage (Milvus, PostgreSQL, MinIO, etc.).
Specific Elements
• Presentation layer (UI)
• Business logic / microservices (Agent, Analytics, API Gateway, Ingestion)
• Data stores (Structured DB, Vector DB, Cache)
• Infrastructure layer (deployment artifacts and orchestration components).
• Optionally, annotate container orchestration details (Docker Compose vs Helm) where relevant, as this is a key aspect of the blueprint’s operational design.