# confluentinc/mongodb-cflt-genai-quickstart architecture

> Architecture diagram of the GitHub repository confluentinc/mongodb-cflt-genai-quickstart, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/confluentinc/mongodb-cflt-genai-quickstart
- Repository: https://github.com/confluentinc/mongodb-cflt-genai-quickstart

## Overview

##### Project type and purpose
- This is a full-stack, cloud-native “GenAI Chatbot Quickstart” designed to help small-business-loan agents.
- It combines a single-page/frontend app (Astro + React + TypeScript), serverless back end (AWS Lambda + API Gateway WebSocket), event streaming (Confluent Cloud + Flink), a vector database (MongoDB Atlas), and AI/ML inference (Anthropic Claude via AWS Bedrock).

##### Main components
a. Frontend (frontend/)
- Astro pages + React components (Chat, Dashboard, Login, etc.)
- Deployed to S3 + CloudFront (via Terraform’s frontend module)
- Connects to the backend over a WebSocket API
b. API & Compute (AWS)
- API Gateway (WebSocket)
- Lambda functions:
- websocket_lambda – handles open/close/message on WS
- chat_input.py – pushes incoming user messages into Kafka topic “chat-input”
- chat_output.py – consumes enriched responses from “chat-output” topic
- kafkatrigger_lambda.py – auxiliary trigger for Kafka events if needed
- IAM, CloudWatch (logs/metrics), Secrets Manager
c. Event Stream / Processing (Confluent Cloud + Flink)
- Kafka topics: products, chat-input, chat-output
- Flink SQL jobs:
- Document indexing pipeline: ingest raw product data → summarize via Claude → embed via Titan Embed → sink connector → MongoDB Atlas vector store
- Query pipeline: embed end-user’s query → search in MongoDB → produce search results
d. Vector Database (MongoDB Atlas)
- Atlas vector search index (configured in main.tf under mongodbatlas_search_index)
- Stores embeddings & summarized documents for RAG
e. AI/ML Inference (AWS Bedrock)
- Anthropic Claude for summarization & conversational responses
- Amazon Titan Embed for vectorization
f. Infrastructure as Code (Terraform)
- Root modules under infrastructure/: backend, frontend, mongodb, confluent-cloud-cluster
- Variables, providers, outputs, scripts (deploy.sh, destroy.sh)

##### Relationships & data flows
- Document Indexing Flow:
1. Sample data injected into Kafka “products” topic via Flink SQL
2. Flink job reads “products” → Anthropic Claude summarization → Titan Embed → writes to “products” → Confluent Sink → MongoDB
- Chat Flow:
1. User types message in frontend → WS API Gateway → websocket_lambda
2. chat_input Lambda publishes message to Kafka “chat-input”
3. Flink pipeline reads “chat-input” → embed query → issues vector search to MongoDB → emits search results to “chat-output”
4. chat_output Lambda receives from “chat-output”, calls Claude for response generation → emits enriched chat message back over WS → frontend displays

##### Architectural patterns & principles
- Event-driven, asynchronous pipelines (Kafka + Flink)
- Serverless compute (Lambda + API Gateway) with auto-scaling
- Retrieval-Augmented Generation (RAG) using vector search + LLMs
- Infrastructure as Code for reproducible deployments
- Clear separation of concerns: UI, API, streaming, storage, AI services

##### Details to capture
- Frontend build/deploy via terraform module → CloudFront/S3
- API Gateway WebSocket → Lambda handlers
- Kafka topics & Flink SQL jobs (summarization, embedding, search)
- MongoDB Atlas vector search index
- Bedrock model calls (Anthropic Claude + Titan Embed)
- Deployment scripts (deploy.sh/destroy.sh)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Client/UI Layer
    subgraph "Client/UI" 
        direction TB
        Browser["Browser"]:::frontend
        FrontendApp["Astro+React App"]:::frontend
        Pages["Astro Pages"]:::frontend
        Components["React Components"]:::frontend
        WebSocketLogic["WebSocket Client Logic"]:::frontend
        Browser -->|HTTPS| FrontendApp
        FrontendApp --> Pages
        FrontendApp --> Components
        Components --> WebSocketLogic
        Browser -->|WebSocket| APIGW
    end

    %% API & Compute Layer
    subgraph "API & Compute" 
        direction TB
        APIGW["API Gateway (WebSocket)"]:::compute
        WebsocketLambda["websocket_lambda Lambda"]:::compute
        ChatInput["chat_input Lambda"]:::compute
        ChatOutput["chat_output Lambda"]:::compute
        KafkaTrigger["kafkatrigger_lambda Lambda"]:::compute

        APIGW -->|connect/disconnect/message| WebsocketLambda
        WebsocketLambda -->|invoke| ChatInput
        WebsocketLambda -->|invoke| ChatOutput
        APIGW -->|optional trigger| KafkaTrigger
    end

    %% Streaming Layer
    subgraph "Event Streaming (Confluent Cloud + Flink)" 
        direction TB
        KafkaCluster["Confluent Cloud Cluster"]:::streaming
        ProductsTopic["products Topic"]:::streaming
        ChatInputTopic["chat-input Topic"]:::streaming
        ChatOutputTopic["chat-output Topic"]:::streaming
        IndexingJob["Flink Indexing Job"]:::streaming
        QueryJob["Flink Query Job"]:::streaming

        KafkaCluster --> ProductsTopic
        KafkaCluster --> ChatInputTopic
        KafkaCluster --> ChatOutputTopic

        ProductsTopic --> IndexingJob
        IndexingJob -->|summarize via Claude & embed via Titan| ProductsTopic
        ChatInputTopic --> QueryJob
        QueryJob --> ChatOutputTopic
    end

    %% Data & AI Services Layer
    subgraph "Data & AI Services" 
        direction TB
        MongoDB["MongoDB Atlas Vector Store"]:::datastore
        Claude["Anthropic Claude (Bedrock)"]:::ai
        TitanEmbed["Titan Embed (Bedrock)"]:::ai

        ProductsTopic -->|Confluent Sink| MongoDB
        QueryJob -->|vector search| MongoDB
        IndexingJob -->|call Claude| Claude
        IndexingJob -->|call Titan Embed| TitanEmbed
        ChatOutput -->|call Claude| Claude
    end

    %% Deployment (Terraform) Layer
    subgraph "Deployment (Terraform)" 
        direction TB
        TFRoot["infrastructure/"]:::infra
        TFBackend["backend Module"]:::infra
        TFFrontend["frontend Module"]:::infra
        TFConfluent["confluent-cloud-cluster Module"]:::infra
        TFMongo["mongodb Module"]:::infra
        TFModels["Bedrock Model SQL Scripts"]:::infra
        TFScripts["deploy.sh / destroy.sh"]:::infra

        TFRoot --> TFBackend
        TFRoot --> TFFrontend
        TFRoot --> TFConfluent
        TFRoot --> TFMongo
        TFRoot --> TFModels
        TFRoot --> TFScripts
    end

    %% Connections across layers
    ChatInput -->|publish message| ChatInputTopic
    ChatOutputTopic -->|consume| ChatOutput
    WebSocketLogic -->|display| Browser

    %% Click Events
    click FrontendApp "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/tree/main/frontend/"
    click Pages "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/tree/main/frontend/src/pages/"
    click Components "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/tree/main/frontend/src/components/"
    click WebSocketLogic "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/frontend/src/components/WebSocketServerMessage.tsx"
    click WebsocketLambda "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/modules/backend/functions/websocket_lambda.py"
    click ChatInput "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/modules/backend/functions/chat_input.py"
    click ChatOutput "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/modules/backend/functions/chat_output.py"
    click KafkaTrigger "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/modules/backend/functions/kafkatrigger_lambda.py"
    click TFConfluent "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/tree/main/infrastructure/modules/confluent-cloud-cluster/"
    click TFBackend "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/tree/main/infrastructure/modules/backend/"
    click TFFrontend "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/tree/main/infrastructure/modules/frontend/"
    click TFMongo "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/modules/mongodb/main.tf"
    click TFModels "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/statements/create-models/bedrock-general-model.sql"
    click TFModels "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/statements/create-models/bedrock-titan-embed.sql"
    click TFRoot "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/infrastructure/main.tf"
    click TFScripts "https://github.com/confluentinc/mongodb-cflt-genai-quickstart/blob/main/deploy.sh"

    %% Styles
    classDef frontend fill:#D0E8FF,stroke:#4098D7,color:#000;
    classDef compute fill:#DFF7DF,stroke:#33A02C,color:#000;
    classDef streaming fill:#FFE7B3,stroke:#FF8C00,color:#000;
    classDef datastore fill:#E8D0FF,stroke:#7D3ACC,color:#000;
    classDef ai fill:#FFD0D0,stroke:#E03E3E,color:#000;
    classDef infra fill:#F0F0F0,stroke:#A0A0A0,stroke-dasharray: 5 5,color:#000;
```
