How straydragon/exp-llm-mcp-rag works
Project Type and Purpose
• This project is an experimental AI assistant system that leverages Large Language Models (LLM), the Model Context Protocol (MCP), and Retrieval-Augmented Generation (RAG). Its purpose is to demonstrate interactions between a language model and external tools (through MCP) as well as how the system uses vector retrieval to augment responses with contextual data.
File Structure
• Notice the primary source folder is “src/augmented/” which contains the core modules:
• agent.py – the central coordinator.
• chat_openai.py – handles LLM invocation through OpenAI.
• mcp_client.py & mcp_tools.py – manage the MCP interactions.
• embedding_retriever.py and vector_store.py – implement the RAG (with retrieval and vector search).
• Additional util functions in “utils/”.
• Configuration files and examples (e.g., .env.example, justfile, and rag_example.py) indicate how to run and test the system.
• The “output” directory holds some result logs and knowledge documents, hinting at how the system might process and present retrieved information.
README for Additional Insights
• The README gives a full overview, including key functionality like LLM API calls, MCP integration, and the RAG system.
• This provides excellent guidance on how components interact and which dependencies (such as OpenAI API, vector store, and MCP) are critical.
Main Components and Their Interactions
a. Major Components:
• User: the external client making queries.
• Agent: the central coordinator that interacts with LLM and tools.
• LLM (via AsyncChatOpenAI): for language model interaction.
• MCP Client: manages the invocation of external tools based on MCP.
• MCP Server (implied): performs file system operations or web content retrieval.
• EmbeddingRetriever & VectorStore: power the RAG functionality by embedding queries and retrieving knowledge.
• Logger (ALogger): records interactions and actions.
b. Relationships and Interactions:
• The user sends a query to the Agent.
• The Agent logs the operation, calls the EmbeddingRetriever to look up relevant context from the VectorStore, then combines the context with the query and sends it to the LLM.
• The LLM produces a response and may trigger a tool call via the MCP Client.
• The MCP Client interacts with the external MCP Server (which handles things like file system operations or web fetches) and returns the output.
• Results from MCP are fed back into the LLM for refining the final answer that is sent back to the user.
c. Patterns & Design Principles:
• The architecture follows a modular design where the Agent orchestrates many independent modules (LLM, MCP, RAG components).
• Interaction flows are based on request-response patterns, with clear separation of concerns (communication, logging, retrieval, and external tool invocation).