How gopeshkhandelwal/nextgen-ai works
Project Type and Purpose
• This is a full-stack, microservice-style LLM Agent system that combines:
• A client library (mcp_client) for invoking an “LLM + MCP” agent
• A backend server (mcp_server) that exposes secure tool-calling endpoints
• A retrieval-augmented generation (RAG) pipeline using FAISS vectorstores
• Pluggable local or API-based LLMs (via vLLM/HuggingFace or OpenAI API)
• A Postgres long-term memory store for conversation history
• Deployment artifacts (Docker, Kubernetes)
File Structure Analysis
• Top level:
• common_utils/: shared code for config, vectorstore build, RAG tests, DB setup
• mcp_client/: client code to call the agent and wrap tools via LangGraph
• mcp_server/: server code, tool implementations (idc_images, idc_pools, weather, rag), HTTP utility
• deployment/: Dockerfiles and K8s YAML for vLLM container
• docs/: API guides and SQL schema
• Makefile & run.sh: build, download, run targets
• Patterns:
• Agent orchestrator (LangGraph) in the client
• Tool registry & execution in server
• RAG as a standalone retrieval component
• Async streaming, environment‐based config
Key README Insights
• Multi‐step, extensible tool workflow via LangGraph
• Dual LLM support (local HuggingFace & OpenAI API)
• FAISS vectorstore management for RAG document QA
• PostgreSQL for conversation memory
• Secure tool execution through the MCP server with logging
Main Components
a. Frontend / Client Layer (mcp_client):
• LangGraph orchestrator
• LangChain tool‐calling wrapper
• Conversation input/output
b. Agent Core (could be co‐located with client or separate service):
• Prompt manager
• RAG retriever (FAISS vectorstore)
• LLM interface (vLLM container or OpenAI API)
• Memory manager (short-term in-process + long-term in Postgres)
c. Tool Execution Backend (mcp_server):
• HTTP/gRPC server
• Tool registry & dispatcher (IDC pools, IDC images, Weather, RAG tool)
• Auth & audit logging
d. Data Stores & External Services:
• PostgreSQL DB (conversation_history)
• FAISS index files on disk (RAG_INDEX_DIR)
• Local HF model cache & vLLM container
• External APIs: OpenAI, IDC API endpoints, Weather API
e. Deployment Environments:
• Docker container for vLLM (HuggingFace Llama/Mistral)
• Kubernetes manifests for scaling vLLM
Relationships & Data Flows
• User → Client (LangGraph orchestrator)
• Client → Agent Core
• Query forwarded to RAG retriever
• Retriever pulls from FAISS index → context snippets
• Core concatenates context + user prompt → LLM interface
• LLM call to OpenAI API or to vLLM container
• Agent Core → Memory manager → store to Postgres
• Agent Core → Client
• Client → MCP Server (HTTP/gRPC) for tool calls → Tools
• Tools → External services (IDC API, weather) → Response
• All calls logged & authenticated via MCP Server
Architectural Patterns & Principles
• Microservices: decouple LLM serving, tool execution, retrieval, memory
• Plugin architecture: add new tools by dropping Python modules + registering
• Separation of concerns: RAG, LLM inference, tool calls, memory persistence
• Async streaming: non‐blocking I/O for high throughput
Technologies to Display
• Python 3, FastAPI or Flask (MCP Server), gRPC/HTTP
• LangGraph, LangChain
• FAISS (vectorstores), sentence-transformers (embeddings)
• PostgreSQL
• Docker, Kubernetes (K8s YAML for vLLM)
• HuggingFace cache, vLLM.Dockerfile
• OpenAI REST API
Recommendations
• Data persistence (disk for FAISS, Postgres for long‐term memory)