# gopeshkhandelwal/nextgen-ai architecture

> Architecture diagram of the GitHub repository gopeshkhandelwal/nextgen-ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gopeshkhandelwal/nextgen-ai
- Repository: https://github.com/gopeshkhandelwal/nextgen-ai

## Overview

##### Project Type and Purpose
- This is a full-stack, microservice-style LLM Agent system that combines:
- A client library (`mcp_client`) for invoking an “LLM + MCP” agent
- A backend server (`mcp_server`) that exposes secure tool-calling endpoints
- A retrieval-augmented generation (RAG) pipeline using FAISS vectorstores
- Pluggable local or API-based LLMs (via vLLM/HuggingFace or OpenAI API)
- A Postgres long-term memory store for conversation history
- Deployment artifacts (Docker, Kubernetes)

##### File Structure Analysis
- Top level:
- `common_utils/`: shared code for config, vectorstore build, RAG tests, DB setup
- `mcp_client/`: client code to call the agent and wrap tools via LangGraph
- `mcp_server/`: server code, tool implementations (`idc_images`, `idc_pools`, `weather`, `rag`), HTTP utility
- `deployment/`: Dockerfiles and K8s YAML for vLLM container
- `docs/`: API guides and SQL schema
- `Makefile` & `run.sh`: build, download, run targets
- Patterns:
- Agent orchestrator (LangGraph) in the client
- Tool registry & execution in server
- RAG as a standalone retrieval component
- Async streaming, environment‐based config

##### Key README Insights
- Multi‐step, extensible tool workflow via LangGraph
- Dual LLM support (local HuggingFace & OpenAI API)
- FAISS vectorstore management for RAG document QA
- PostgreSQL for conversation memory
- Secure tool execution through the MCP server with logging

##### Main Components
a. Frontend / Client Layer (`mcp_client`):
- LangGraph orchestrator
- LangChain tool‐calling wrapper
- Conversation input/output
b. Agent Core (could be co‐located with client or separate service):
- Prompt manager
- RAG retriever (FAISS vectorstore)
- LLM interface (vLLM container or OpenAI API)
- Memory manager (short-term in-process + long-term in Postgres)
c. Tool Execution Backend (`mcp_server`):
- HTTP/gRPC server
- Tool registry & dispatcher (IDC pools, IDC images, Weather, RAG tool)
- Auth & audit logging
d. Data Stores & External Services:
- PostgreSQL DB (conversation_history)
- FAISS index files on disk (`RAG_INDEX_DIR`)
- Local HF model cache & vLLM container
- External APIs: OpenAI, IDC API endpoints, Weather API
e. Deployment Environments:
- Docker container for vLLM (HuggingFace Llama/Mistral)
- Kubernetes manifests for scaling vLLM

##### Relationships & Data Flows
- User → Client (LangGraph orchestrator)
- Client → Agent Core
- Query forwarded to RAG retriever
- Retriever pulls from FAISS index → context snippets
- Core concatenates context + user prompt → LLM interface
- LLM call to OpenAI API or to vLLM container
- Agent Core → Memory manager → store to Postgres
- Agent Core → Client
- Client → MCP Server (HTTP/gRPC) for tool calls → Tools
- Tools → External services (IDC API, weather) → Response
- All calls logged & authenticated via MCP Server

##### Architectural Patterns & Principles
- Microservices: decouple LLM serving, tool execution, retrieval, memory
- Plugin architecture: add new tools by dropping Python modules + registering
- Separation of concerns: RAG, LLM inference, tool calls, memory persistence
- Async streaming: non‐blocking I/O for high throughput

##### Technologies to Display
- Python 3, FastAPI or Flask (MCP Server), gRPC/HTTP
- LangGraph, LangChain
- FAISS (vectorstores), sentence-transformers (embeddings)
- PostgreSQL
- Docker, Kubernetes (K8s YAML for vLLM)
- HuggingFace cache, vLLM.Dockerfile
- OpenAI REST API

##### Recommendations
- Data persistence (disk for FAISS, Postgres for long‐term memory)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Frontend / Client Layer
    subgraph "Frontend / Client Layer"
        direction TB
        User["User"]:::external
        CI["Client I/O"]:::frontend
        LG["LangGraph Orchestrator"]:::frontend
        LC["LangChain Tool Wrappers"]:::frontend
        CDB["Short-term Memory Utils"]:::frontend
    end

    %% Agent Core
    subgraph "Agent Core"
        direction TB
        PM["Prompt Manager & Config"]:::core
        RAG["RAG Retriever (FAISS)"]:::core
        LLM["LLM Interface (vLLM/OpenAI)"]:::core
        MEM["Memory Manager"]:::core
        SU["Shared Utilities"]:::core
    end

    %% MCP Server
    subgraph "MCP Server"
        direction TB
        Server["MCP Server (HTTP/gRPC)"]:::core
        Auth["Auth & Audit Logging"]:::core
        subgraph "Tool Registry & Dispatcher"
            direction TB
            IDCImg["IDC Images Tool"]:::core
            IDCPool["IDC Pools Tool"]:::core
            RAGTool["RAG Tool"]:::core
            Weather["Weather Tool"]:::core
        end
    end

    %% Data Stores & External Services
    subgraph "Data Stores & External Services"
        direction TB
        PG["Postgres DB"]:::datastore
        FAISSDisk["FAISS Index (Disk)"]:::datastore
        HFCache["HF Model Cache"]:::external
        OpenAI["OpenAI API"]:::external
        IDCAPI["IDC API"]:::external
        WeatherAPI["Weather API"]:::external
    end

    %% Deployment Artifacts
    subgraph "Deployment"
        direction TB
        VLLMDocker["vLLM Dockerfile"]:::deployment
        K8sDep["vLLM K8s Deployment"]:::deployment
        K8sSvc["vLLM K8s Service"]:::deployment
        BuildRun["Makefile & run.sh"]:::deployment
    end

    %% Connections
    User -->|inputs| CI
    CI -->|"invokes"| LG
    LG -->|"uses"| PM
    LG -->|"calls"| RAG
    RAG -->|reads/writes| FAISSDisk
    RAG -->|"uses"| SU
    SU -->|"utils"| PM
    LG -->|"sends prompt to"| LLM
    LLM -->|"calls REST/socket"| HFCache
    LLM -->|"calls API"| OpenAI
    LLM -->|"returns response"| MEM
    MEM -->|"stores conversation"| PG
    CI -->|"tool call (HTTP/gRPC)"| Server
    Server -->|auth/log| Auth
    Server -->|"dispatches to"| IDCImg
    Server -->|"dispatches to"| IDCPool
    Server -->|"dispatches to"| RAGTool
    Server -->|"dispatches to"| Weather
    IDCImg -->|"calls API"| IDCAPI
    IDCPool -->|"calls API"| IDCAPI
    Weather -->|"calls API"| WeatherAPI
    RAGTool -->|"reads index"| FAISSDisk

    %% Deployment flows
    VLLMDocker -->|"build image"| K8sDep
    K8sDep -->|"exposes service"| K8sSvc
    BuildRun -->|"automation"| VLLMDocker

    %% Click Events
    click CI "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_client/client.py"
    click LG "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_client/langgraph_graph.py"
    click LC "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_client/langgraph_tool_wrappers.py"
    click CDB "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_client/db_utils.py"
    click PM "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/common_utils/config.py"
    click RAG "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/common_utils/build_vectorstore.py"
    click RAGTool "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/common_utils/rag_test.py"
    click LLM "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/common_utils/download_model.py"
    click SU "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/common_utils/utils.py"
    click PG "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/common_utils/database/conversation_history.sql"
    click Server "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_server/server.py"
    click IDCImg "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_server/tools/idc_images.py"
    click IDCPool "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_server/tools/idc_pools.py"
    click RAGTool "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_server/tools/rag.py"
    click Weather "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_server/tools/weather.py"
    click Auth "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/mcp_server/utils/http.py"
    click VLLMDocker "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/deployment/docker/vllm.Dockerfile"
    click K8sDep "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/deployment/k8s/vLLM/deployment-vllm-fork-with-llama-2-7b-chat-hf.yaml"
    click K8sSvc "https://github.com/gopeshkhandelwal/nextgen-ai/blob/main/deployment/k8s/vLLM/service-vllm-fork-with-llama-2-7b-chat-hf.yaml"
    click BuildRun "https://github.com/gopeshkhandelwal/nextgen-ai/tree/main/Makefile"

    %% Styles
    classDef frontend fill:#E0F7FA,stroke:#00838F,stroke-width:2px
    classDef core fill:#E8F5E9,stroke:#2E7D32,stroke-width:2px
    classDef datastore fill:#FFF3E0,stroke:#EF6C00,stroke-width:2px
    classDef external fill:#F3E5F5,stroke:#6A1B9A,stroke-width:2px
    classDef deployment fill:#ECEFF1,stroke:#455A64,stroke-width:2px
```
